Subject: 自身の価値をご存知ですか？

From: dlywb3 <[email-masked]>

【PR:最高額出ましたよ〜】
　　　┏━┓┏━━┳━━┳━━┓┏━━━━┓
　　　┗┓┃┃┏┓┃┏┓┃┏┓┃┗┓┏━━┛
　　　　┃┃┃┃┃┃┃┃┃┃┃┃　┃┗━━┓
　　　　┃┃┃┃┃┃┃┃┃┃┃┃　┃┏━┓┃
　　　　┃┃┃┃┃┃┃┃┃┃┃┃　┃┃　┃┃
　　　┏┛┗┫┗┛┃┗┛┃┗┛┃┏┛┃┏┛┃
　　　┗━━┻━━┻━━┻━━┛┗━┛┗━┛
     ━━━━━━━━━●━━━━━━━━━
○o.今雑誌などで話題の童貞オークション開催.o○
　　　　◆◆◆ 　 多種女性開催    ◆◆◆
     ━━━━━━━━━●━━━━━━━━━
2007年最初のオークション最高額一千万円の高額落札も有！
　　　　≪童貞≫自己申告制になります！
　　　　　　　　　　現在開催中
      http://orange-candy.com/kda06/?1070

o○ .。o○ .。o○高額現金獲得その方法○o。. ○o。. ○o
　　　　利用の際に必ず、希望落札価格を書く
　　　　自分の長所短所含む自己紹介も必須
　　　　　　　┏━━━━━━━━┓
　　　　　　　┃オークション女性┃
　　　　　　　┗━━━━━━━━┛
雑誌などの広告・セレブパーティ広告により参加！
女性オークション参加条件として『金』『時間』にゆとりのある女性のみを集めました 

開催するには今心動くあなたの参加が必須です！！
　　　　　　　　　　￣￣￣￣￣￣
　　　　　　　　今すぐ参加
　　http://orange-candy.com/kda06/?1070

　　　　　　　┏━━━━━━━━┓
　　　　　　　┃ちょっと待った！┃
　　　　　　　┗━━━━━━━━┛
　　　このメール消す前にもう一度考えて！
　　　　　　　　後で参加する
　　　http://orange-candy.com/kda06/?1070

配信拒否の方はこちら
[email-masked]

To: [email-masked]
Xref: clerew local.usefor:24500
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: application/news-message-id
Date: Tue, 27 Feb 2007 17:12:14 GMT
Lines: 32
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>>> http://tools.ietf.org/html/draft-ellermann-news-nntp-uri-05#section-8.3

>> Ah! You are wanting to change it from an RFC 4288 registrartion to an
>> RFC 4289 registration. That makes sense.

>Yes, see http://permalink.gmane.org/gmane.ietf.types/799

>> But you would still have to write a proper 4289 template for it.

>What's a "proper 4289 template" ?  I've posted a pro forma review request
>on the ietf-types mailing list as specified in chapter 2.2.2 of RFC 4289:

Hmmm! RFC 4289 doesn't seem to provide for any templates. So when IANA
accepts a registration how does it record it? 

Ah! I see than an RFC MUST exist. But there is no standard format
(template) in which it must be described, which is unusual.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24501
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Date: Tue, 27 Feb 2007 21:23:04 GMT
Lines: 162
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> But if an injecting agent is aware of the cutoff of its immediately
>> following relaying agent (which is a reasonable assumption, because they
>> are often two aspects of the same agent), then it will also be aware
>> whether said relaying agent understands Injection-Date. In which case
>> that SHOULD should not apply. IOW, what you are proposing should be
>> regarded as only a temporary measure for use when Injection-Date is not
>> observed (i.e.  a continuation of what some agents are currently doing
>> anyway).

>Yes, exactly.  I think that the root of our disagreement is again that I
>believe said transition period until Injection-Date is widely enough
>observed that an article with a very stale Date will propagate usefully is
>on the order of five to ten years.

It only needs to be implemented by those news servers with exceptionally
short history retention times for its benefits to be felt, and the number
of such servers is relatively small.

>I believe that some posting agents deliberately add the Message-ID and
>Date header so that they can inject the article with the same identity to
>multiple servers simultaneously.  I know that at least some proxies have
>that capability.  If each injecting agent then adds their own
>Injection-Date, we break the identity model.

OK, so the issue only arises with agents that multi-inject, and harm will
only arise if those injections are significantly separated in time
(and by a period measured in hours rather than minutes).

>Sure.  Suppose that I'm using a local news proxy which talks to three
>different remote servers for me and presents me with a combined view of
>newsgroups from those servers.  (This is an increasingly common substitute
>for running a full-fledged local news server, and in some ways works much
>nicer.)  Suppose that for maximum propagation (for exactly the same sorts
>of reasons that you do something similar with C News), I have it
>configured to post my messages to all of those remote servers at the same
>time.

>If one of those servers is down at the time of the posting, the safest and
>most reliable behavior, and the one that I believe is currently
>implemented in at least some software of this sort, is to queue the post
>and keep trying.

And that would seem to be the only situation in which a significant time
separation could occur during multi-injection.

And that has to be set against my concern about the man who prepares
articles offline and then injects them much later.

So let us examine the two competing strategies.

I think we are agreed that

. Injecting agents MUST insert Date and Message-ID when absent.
. Injecting agents MUST insert Injection-Date when Injection-Date is
absent AND (either Date OR Message-ID is absent).
. Injecting agents MUST NOT insert Injection-Date if it is already
present.

We disagree on the following two possibilities:

C. Injecting agents MUST insert Injection-Date when it is absent.

R. Injecting agents MUST NOT insert Injection-Date if both (Date AND
   Message-ID are already present).

So now we can examine the consequences of those two rules.

Consequences of Rule C:

Posting agents (or proxies etc) which multi-inject AND where one of the
injections sits around in a queue because a server is down, and which
inserts an Injection-Date when it eventually wakes up, may cause relaying
agents further on which understand Injection-Date to re-accept articles
that they have already accepted before.

But note that this problem is not permanent - it will go away as and when
those multi-injecting agents start inserting Injection-Date themselves
(which they would have to do to claim compliance with our new standard).

OTOH, posting agents which prepare articles offline, adding the Date at
composition time, and then failing to inject until hours/days later, and
who then inject to a single injecting agent (either adding Injection-Date
themselves or letting the injecting agent do it) will find that their
articles propagate well amongst servers that understand Injection-Date. So
their situation will improve as Injection-Date becomes more widely
adopted.

Consequences of Rule R:

Posting agents (or proxies etc) which multi-inject AND where one of the
injections sits around in a queue because a server is down will not cause
articles to be re-accepted.

OTOH, posting agents which prepare articles offline, adding the Date at
composition time, and then failing to inject until hours/days later, and
who then inject to a single injecting agent and do not add Injection-Date
themselves will find that their articles propagate as badly as they do at
present.

Note that this problem is permanent (or at least until those posting
agents learn to insert Injection-Date themselves). Moreover, the problem
will get worse as more and more posting agents accept our recommendation
to add the Date header at composition time.

So now we have to consider which of those scenarios is the more
troublesome. For this, we have to make some guesses:

1. How many multi-injecting agents are there around, and how often will
there be disruption because one of their injectees is down; and how does
this compare with the number of offline posting agents around and how
often their injecting get delayed?

2. How will the deployment of Injection-Date go? My expectation is that
it will be taken up first by those relaying agents which currently have
short expiries on their history files (because they have the most to
gain). It will then be taken up by smaller sites with clueful
administrators, and that probably includes people running proxies and
doing multi-injecting. And finally, it will be taken up by ordinary user
agents, and indeed that will be a _very_ slow uptake.

3. How rapidly will the practice of constructing the Date at composition
time spread? Probably faster than the spread of posting agents that know
how to insert Injection-Date themselves (which I regard as normally
unnecessary, although we seem to disagree there). But this is a guess
where we have some evidence to go on, because this practice is already
widespread.

I performed an experiment on 350 articles in the group uk.misc (where I
would expect to find a pretty average bunch of posters). I compared the
NNTP-Posting-Date with the Date header. I would expect these to be
identical (to the same second) where they were both created on the
injecting agent. Where they were different, they were often surprisingly
close (indicating that lots of people were actually online while they
composed articles, though differences between the two dates were both
positive and negative, indicating differences between the clocks on the
two systems). But the interesting fact to emerge was that nearly 70% of
posting agents were apparently already constructing their own date
headers. OK, the sample was small, and I will happily perform a larger
experiment if people wish. But that 70% was considerably higher than I
would have expected.

So my conclusion is that it will soon be commonplace for Date to be already
present at injection time (indeed we are almost there), and so to make
assumptions on the basis that this indicates a multi-injecting proxy is
unwise. On balance, it sees that rule C is going to provide fewer
problems, especially in the longer term.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Organization: The Eyrie
Date: Tue, 27 Feb 2007 11:21:07 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> 3b: It MAY reject articles with dates more than 24 hours in the future.

> Normally magic numbers like "24 hours" need a justification, but here it
> should be obvious that this is about a plausible timezone confusion.
> (If that's not the real reason for "24 hours" better say what it is.)

That's exactly the reason for 24 hours (and I'm not adverse to adding a
comment saying so).

> The "articles of which it keeps records" isn't clear.  It's not about
> the oldest available article (that could be months or more), it's the
> oldest unavailable article (only recorded in the history).

Yes.  Exactly.

> Maybe it's better if you define and use a term like "retention time" or
> "cut off time" as you've done it elsewhere.  I'm not sure what the
> difference is, "cut off => reject as too old, retention => could be
> removed from the history" (?)

Right.

I think we're converging on a desire to rephrase things that way and
introduce those terms.

> For the example mentioned above:  groups with very short expirations, an
> extreme case would be a server only relaying some groups without keeping
> the articles at all.  It would still note the Message-IDs as "seen" in
> its history for some days.  Or rather that's what I think how it works
> after long discussions more than a decade ago where some folks finally
> convinced me that NetNews doesn't rely on the Path for this purpose
> (unlike Fido echomail).

You're entirely correct in your understanding.  I think I understand your
point now (and agree with it).

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Date:  Tue, 27 Feb 2007 19:32:57 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 53
Sender: [email-masked]

Russ Allbery wrote:

>    3.  It MUST examine the Injection-Date header field or, if absent,
>        the Date header field, and reject the article if that date
>        predates the earliest articles of which it keeps record or if
>        that date is more than 24 hours into the future.  It MAY reject
>        articles with dates in the future with a smaller margin than 24
>        hours.

I'd split that - future timestamps can get their own (small) paragraph:

3b: It MAY reject articles with dates more than 24 hours in the future.

Normally magic numbers like "24 hours" need a justification, but here
it should be obvious that this is about a plausible timezone confusion.
(If that's not the real reason for "24 hours" better say what it is.)

The "articles of which it keeps records" isn't clear.  It's not about
the oldest available article (that could be months or more), it's the
oldest unavailable article (only recorded in the history).  Maybe it's
better if you define and use a term like "retention time" or "cut off
time" as you've done it elsewhere.  I'm not sure what the difference
is, "cut off => reject as too old, retention => could be removed from
the history" (?)

What I have in mind is a server keeping articles in some groups for
long periods, e.g. for "answers" groups, and articles in other groups
only shortly, e.g. in binary groups.

> As for the reasonable minimum on history retention, you have repeatedly
> raised this and repeatedly lost this argument every time that you have.

IMO the argument in s-o-1036 is compelling.  Its UUCP considerations are
not more as important as in 1994.  On the other hand we have this "Date
to Injection-Date transition" now, that's not the moment to lower the
s-o-1036 minimum.   If a news server hasn't the disk space to keep a 
reasonable history it carries too many groups or something else is odd.

>> Implementors need to know that "listed in the history" isn't the same
>> as "we still have that article".
 
> Why?

For the example mentioned above:  groups with very short expirations, an
extreme case would be a server only relaying some groups without keeping
the articles at all.  It would still note the Message-IDs as "seen" in
its history for some days.  Or rather that's what I think how it works
after long discussions more than a decade ago where some folks finally
convinced me that NetNews doesn't rely on the Path for this purpose 
(unlike Fido echomail).

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Organization: The Eyrie
Date: Tue, 27 Feb 2007 09:23:46 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> OTOH, one would hope that it will be those dedicated relayers who will
> be the early adopters of Injection-Date.

Given that development of NNTPRelay has essentially stopped, the Highwinds
support forums appear to be down, and Highwinds hasn't issued a press
release on their site for nearly four years, I wouldn't hold my breath.
Highwinds did just release new builds last November for the first time in
two years, though, so they don't appear to have disappeared entirely.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24498
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Date: Tue, 27 Feb 2007 17:04:17 GMT
Lines: 35
Sender: [email-masked]

In <[email-masked]> Harald Alvestrand <[email-masked]> writes:

>Should we add an appendix with a list of "recommended timers"?

>There, we could say

>   Stale article time (section x.x): RECOMMENDED to be larger than 7 days
>        MUST (SHOULD?) be less than Message-ID retention time.

I am not sure that "stale article time" is a well-defined concept. A news
server that is both a relaying agent and a serving agent (and that is
nearly all of them, except for the dedicated
relay-only-as-fast-as-possible sites which act as a Usenet backbone) will
keep an article in its history file as long as it stores the article and
is prepared to deliver it to reading agents, and that will usually by much
longer than 7 days. It is only those dedicated fast relayers that are
likely to purge their history files early (even after 3 days) and who are
at risk of accepting a delayed article a second time.

OTOH, one would hope that it will be those dedicated relayers who will be
the early adopters of Injection-Date.

>   Message-ID retention time (section x.y): RECOMMENDED to be larger 
>than 7 days

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: Usenet <[email-masked]>
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Organization: The Eyrie
Date: Tue, 27 Feb 2007 08:45:13 -0800
Sender: [email-masked]

Harald Alvestrand <[email-masked]> writes:
> Russ Allbery wrote:

>> The retention time is specifically tied to the stale article time and
>> shouldn't be set independently of it, so we need to capture that somehow.

> Should we add an appendix with a list of "recommended timers"?

> There, we could say

>   Stale article time (section x.x): RECOMMENDED to be larger than 7 days
>        MUST (SHOULD?) be less than Message-ID retention time.

>   Message-ID retention time (section x.y): RECOMMENDED to be larger than 7
> days

> I seem to remember seeing such lists in other specs. "When in doubt, be
> explicit".

We could, but are there any other timers that would go there?

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Organization: The Eyrie
Date: Tue, 27 Feb 2007 08:43:30 -0800
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> I still don't believe that this "effective identity" concept is a Good
> Thing.

> The Netnews environment includes, among other things:
> - Current spools (with different expiry times)
> - History databases (with different expiry times)
> - Long-term archive (which don't expire at all)

> Introducing a concept of "effective identity" that involves Date: clouds
> the issue for me.

Note that it's just a terminology for discussion purposes and not
something that I wanted to put into the wording of the draft.

> An alternative formulation that I think results in the same semantics:

> - News servers want to serve messages only once.

They want to accept messages only once.  Usually messages are served many
times.

> - The primary means of ensuring this is to keep a database of message-IDs
> they have already served. This database usually contains message-ID +
> arrival time.
> - This creates three different classes of message:
>  - Those that are in the database
>  - Those that are not in the database, and never have been
>  - Those that have been removed from the database because it's been a long
> time since they entered it.

Right.

> - Servers will try to discard messages of the first and third classes
> - To see the difference between the second and third class, they look at
> the age of the message, to see if they would have already removed it from
> the database if they had seen it before.

Right, although that's a rather weak way of stating it.  I would rather
say that servers MUST discard messages that, had they seen the message
before, they would have already removed it from their database.

> - They use two hints to the age of the message: Date and Injection-Date
> - Of these, Date: is (usually) older than Injection-Date

True, although not guaranteed by anything, and there are cases where this
won't be true (a client with a bad clock, for instance).

> - Since most messages arrive shortly after injection, it is reasonable to
> assume that a message "has been here and gone" if Injection-Date is such
> that if it arrived here shortly after the injection-date, it would have
> been discarded.

The assumpion is that if the article is not mentioned in history and the
Injection-Date is older than the cut-off date, the server has no way of
being sure that it's a new article and therefore MUST reject the article
to prevent duplicates.  It's not so much an assumption about the article
as it is a correctness guarantee in the news server.

Minor digression: Usually, a news server implements a cut-off date that
matches the history retention time.  History entries for unwanted or
expired articles are maintained until the retention time has passed since
the history entry was written.  Articles are rejected if Injection-Date is
older than the cut-off date.  The cut-off date MUST be more recent than or
the same as the retention date.

So, if the cut-off date is seven days in the past, and an article arrives
on the server with an Injection-Date that's six days in the past, the
article will be accepted and, normally, the history entry will be retained
for *another* seven days past that.  That configuration isn't mandatory
and a configuration that does article and history expiration based on the
Injection-Date is also possible and probably conforming.

> - If Injection-date: isn't there, use Date:
> - Since some servers will use Date: anyway, injecting-agents should police
> injection based on Date:

Yup.

> This reformulation (which is hardly English...) positions Date and
> Injection-Date as a *hint* that allows one to heuristically guess at
> whether the article has been here or not, without introducing a new
> identity concept. I'm not sure if that's a better way to attack it, but
> I'd like to try.

As long as we're strong enough about a news server having to reject a
message that it cannot prove it's not seen before, I'm okay with that.

> (parenthesis: has anyone tried using Bloom filters for history
> databases? They're constant size, so they don't need to be
> trimmed.... and can have a fairly low false-positive hitrate...)

Interesting.  I haven't, in that form, although I've used various
variations on hash tables that reduce to k = 1 Bloom filters.  A Bloom
filter can't serve some of the ancillary purpose of the history file, such
as being the message ID to article mapping database, but those are only
needed on a serving agent and can be handled other ways.  You'd probably
have to rebuild the Bloom filter periodically from a simple list of
message IDs once the false positive rate became worrisome, but this might
still be faster.

You're potentially doing a lot of random-access writes for each incoming
message, which is the place I'd worry about performance.  The traditional
history database algorithm requires a ton more space and a very
time-consuming expiration run, but storing a new message ID is two pwrites
and a buffered write.  I wonder how many disk pages you'd make dirty when
writing a message ID to a memory-mapped Bloom filter with an ideal k.

INN's current history algorithm uses a Bloom filter with k = 1 as an
initial check for unknown articles, but it does a full hash-table lookup
if the Bloom filter lookup succeeds to eliminate the false positives.

This would be an interesting experiment.  INN does have a pluggable
history mechanism precisely so that people can write new ones and see how
they work.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Tue, 27 Feb 2007 11:06:16 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Russ Allbery <[email-masked]>, [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Sender: [email-masked]

--On 24. februar 2007 11:33 -0800 Russ Allbery <[email-masked]> wrote:

>
> Frank Ellermann <[email-masked]> writes:
>
>> I don't believe in this "identity = ID + timestamp" theory.  The ID is
>> the identity, the timestamp is a kludge because histories are limited,
>> and because accepting stale articles isn't in the interest of the
>> poster.
>
> I think that's a wording distinction without any semantic difference.
> Kludge or identity, history files are limited and therefore the timestamp
> is a component of the effective identity of the article.

I still don't believe that this "effective identity" concept is a Good 
Thing.

The Netnews environment includes, among other things:
- Current spools (with different expiry times)
- History databases (with different expiry times)
- Long-term archive (which don't expire at all)

Introducing a concept of "effective identity" that involves Date: clouds 
the issue for me.

I feel that we're better off sticking to the words in USEFOR that two 
messages with the same message-ID *are* the same message (albeit one or 
both may have been damaged in transit, such as by having its entire 
contents replaced :-)

An alternative formulation that I think results in the same semantics:

- News servers want to serve messages only once.
- The primary means of ensuring this is to keep a database of message-IDs 
they have already served. This database usually contains message-ID + 
arrival time.
- This creates three different classes of message:
  - Those that are in the database
  - Those that are not in the database, and never have been
  - Those that have been removed from the database because it's been a long 
time since they entered it.

- Servers will try to discard messages of the first and third classes
- To see the difference between the second and third class, they look at 
the age of the message, to see if they would have already removed it from 
the database if they had seen it before.
- They use two hints to the age of the message: Date and Injection-Date
- Of these, Date: is (usually) older than Injection-Date
- Since most messages arrive shortly after injection, it is reasonable to 
assume that a message "has been here and gone" if Injection-Date is such 
that if it arrived here shortly after the injection-date, it would have 
been discarded.
- If Injection-date: isn't there, use Date:
- Since some servers will use Date: anyway, injecting-agents should police 
injection based on Date:

This reformulation (which is hardly English...) positions Date and 
Injection-Date as a *hint* that allows one to heuristically guess at 
whether the article has been here or not, without introducing a new 
identity concept. I'm not sure if that's a better way to attack it, but I'd 
like to try.

(parenthesis: has anyone tried using Bloom filters for history databases? 
They're constant size, so they don't need to be trimmed.... and can have a 
fairly low false-positive hitrate...)

               Harald

Date: Mon, 26 Feb 2007 09:26:39 +0100
From: Harald Alvestrand <[email-masked]>
To: Russ Allbery <[email-masked]>
Cc: Usenet <[email-masked]>
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Sender: [email-masked]

Russ Allbery wrote:
> Bill McQuillan <[email-masked]> writes:
>   
>
>>    2.  It MUST reject any article that has already been successfully
>>        sent to it, based on the Message-ID header field of the article.
>>        To satisfy this requirement, a relaying agent normally keeps a
>>        database of message identifiers it has already accepted. Since
>>        this database may grow without limit, message identifiers that
>>        have been stored for more than a certain amount of time
>>        (often 7 days) may be purged.
>>     
>
> The retention time is specifically tied to the stale article time and
> shouldn't be set independently of it, so we need to capture that somehow.
>
>   
Should we add an appendix with a list of "recommended timers"?

There, we could say

   Stale article time (section x.x): RECOMMENDED to be larger than 7 days
        MUST (SHOULD?) be less than Message-ID retention time.

   Message-ID retention time (section x.y): RECOMMENDED to be larger 
than 7 days

I seem to remember seeing such lists in other specs. "When in doubt, be 
explicit".

                   Harald

From: Russ Allbery <[email-masked]>
To: Usenet <[email-masked]>
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Organization: The Eyrie
Date: Sun, 25 Feb 2007 17:19:41 -0800
Sender: [email-masked]

Bill McQuillan <[email-masked]> writes:

> To me this reads like:

>      2 - Keep all message identifiers FOREVER!
> and
>      3 - Injection-Date helps reject articles that were injected before
>          the agent was ever run for the first time!

> Since this is clearly a mistaken notion, I think that #2 should be modified
> to include a note that the history may be pruned after a "decent interval".
> Perhaps something like:

>    2.  It MUST reject any article that has already been successfully
>        sent to it, based on the Message-ID header field of the article.
>        To satisfy this requirement, a relaying agent normally keeps a
>        database of message identifiers it has already accepted. Since
>        this database may grow without limit, message identifiers that
>        have been stored for more than a certain amount of time
>        (often 7 days) may be purged.

The retention time is specifically tied to the stale article time and
shouldn't be set independently of it, so we need to capture that somehow.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Sun, 25 Feb 2007 15:25:48 -0800
From: Bill McQuillan <[email-masked]>
To: Usenet <[email-masked]>
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Sender: [email-masked]

On Sun, 2007-02-25, Russ Allbery wrote:

> Currently, we have:

>    2.  It MUST reject any article that has already been successfully
>        sent to it, based on the Message-ID header field of the article.
>        To satisfy this requirement, a relaying agent normally keeps a
>        database of message identifiers it has already accepted.

>    3.  It MUST examine the Injection-Date header field or, if absent,
>        the Date header field, and reject the article if that date
>        predates the earliest articles of which it keeps record or if
>        that date is more than 24 hours into the future.  It MAY reject
>        articles with dates in the future with a smaller margin than 24
>        hours.

To me this reads like:

     2 - Keep all message identifiers FOREVER!
and
     3 - Injection-Date helps reject articles that were injected before
         the agent was ever run for the first time!

Since this is clearly a mistaken notion, I think that #2 should be modified
to include a note that the history may be pruned after a "decent interval".
Perhaps something like:

   2.  It MUST reject any article that has already been successfully
       sent to it, based on the Message-ID header field of the article.
       To satisfy this requirement, a relaying agent normally keeps a
       database of message identifiers it has already accepted. Since
       this database may grow without limit, message identifiers that
       have been stored for more than a certain amount of time
       (often 7 days) may be purged.

-- 
Bill McQuillan <[email-masked]>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Organization: The Eyrie
Date: Sun, 25 Feb 2007 10:19:16 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:
> Russ Allbery wrote:

>> Again here it would be more useful if you would present an argument
>> rather than making an assertion.

> The draft has to:

> (1) explain why Date can't be used anymore in the sense of s-o-1036
>     without getting into conflicts with the RFC 2822 usage
> (2) state that Injection-Date is introduced to fill that gap

This seems like a reasonable addition to the point on Injection-Date in
appendix A where changes to the protocol are discussed.  Please feel free
to propose text.

> (3) explain what a history is, and why a reasonable minimum like N=7
>     in s-o-1036 is required for its purpose

Currently, we have:

   2.  It MUST reject any article that has already been successfully
       sent to it, based on the Message-ID header field of the article.
       To satisfy this requirement, a relaying agent normally keeps a
       database of message identifiers it has already accepted.

   3.  It MUST examine the Injection-Date header field or, if absent,
       the Date header field, and reject the article if that date
       predates the earliest articles of which it keeps record or if
       that date is more than 24 hours into the future.  It MAY reject
       articles with dates in the future with a smaller margin than 24
       hours.

I agree that an explicit link between staleness and the lack of a
requirement to keep an unbounded database isn't drawn there as clearly as
it could be.  Specific text recommendations are welcome.

As for the reasonable minimum on history retention, you have repeatedly
raised this and repeatedly lost this argument every time that you have.  I
have no interest in opening the discussion again.

> If the principles of operation are clear the consequences are also
> clear.  Small differences of the Injection-Date of multiple injections
> are IMO no problem.  If you think it's a problem explain what that is.

I already did.  I don't have anything further to add.

> Implementors need to know that "listed in the history" isn't the same as
> "we still have that article".

Why?

> They also need to know when they can get rid of a history entry (based
> on arrival date or Injection-Date).

I think this is a reasonable point.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Date:  Sun, 25 Feb 2007 10:09:29 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

Russ Allbery wrote:

> Again here it would be more useful if you would present an argument
> rather than making an assertion.

The draft has to:

(1) explain why Date can't be used anymore in the sense of s-o-1036
    without getting into conflicts with the RFC 2822 usage
(2) state that Injection-Date is introduced to fill that gap
(3) explain what a history is, and why a reasonable minimum like N=7
    in s-o-1036 is required for its purpose

If the principles of operation are clear the consequences are also
clear.  Small differences of the Injection-Date of multiple injections
are IMO no problem.  If you think it's a problem explain what that is.

Implementors need to know that "listed in the history" isn't the same
as "we still have that article".  They also need to know when they can
get rid of a history entry (based on arrival date or Injection-Date).

If all else fails just copy the s-o-1036 text replacing its "Date".

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Organization: The Eyrie
Date: Sat, 24 Feb 2007 11:33:37 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> I don't believe in this "identity = ID + timestamp" theory.  The ID is
> the identity, the timestamp is a kludge because histories are limited,
> and because accepting stale articles isn't in the interest of the
> poster.

I think that's a wording distinction without any semantic difference.
Kludge or identity, history files are limited and therefore the timestamp
is a component of the effective identity of the article.

> Either the ID is still in the history, or it isn't.  It's already odd to
> post articles days after they were written.  If somebody manages to post
> the same article again elsewhere after a significant delay that's just a
> very bad idea, add a MUST NOT or something.

We just had a long thread discussing the reasons why this is useful to do
when going between Netnews networks that may or may not be disjoint.
Those uses don't go away just because you think they're a bad idea.

> How about "posting agent MUST generate an Injection-Date at the time of
> the 1st injection if it intends to inject this article later elsewhere"?

I think everyone's already agreed to this and we've moved on to discussing
the backward-compatibility impact on agents that have not updated to
support this draft.

> Multiple injection is gateway magic, "above all avoid loops" etc,

That's where we started this whole discussion at.  I think several cogent
arguments have been presented since for why just leaving all of that to
the section on gateways isn't a good idea.  Again here it would be more
useful if you would present an argument rather than making an assertion.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Date:  Sat, 24 Feb 2007 16:04:35 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 33
Sender: [email-masked]

Russ Allbery wrote:

> I believe said transition period until Injection-Date is widely enough
> observed that an article with a very stale Date will propagate usefully
> is on the order of five to ten years.

Or more.  But at least it will be _possible_ to get it right for those
who care about it.  You need a rationale pointing out that RFC 2822 has
its own concept for Date now also adopted for NetNews.  Injection-Date
is a simple workaround for this issue because histories are limited.

> I believe that some posting agents deliberately add the Message-ID and
> Date header so that they can inject the article with the same identity
> to multiple servers simultaneously.

I don't believe in this "identity = ID + timestamp" theory.  The ID is
the identity, the timestamp is a kludge because histories are limited,
and because accepting stale articles isn't in the interest of the poster.

> If each injecting agent then adds their own Injection-Date, we break
> the identity model.

Either the ID is still in the history, or it isn't.  It's already odd to
post articles days after they were written.  If somebody manages to post
the same article again elsewhere after a significant delay that's just a
very bad idea, add a MUST NOT or something.

How about "posting agent MUST generate an Injection-Date at the time of
the 1st injection if it intends to inject this article later elsewhere" ?
Multiple injection is gateway magic, "above all avoid loops" etc,

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Organization: The Eyrie
Date: Fri, 23 Feb 2007 21:53:13 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> But if an injecting agent is aware of the cutoff of its immediately
> following relaying agent (which is a reasonable assumption, because they
> are often two aspects of the same agent), then it will also be aware
> whether said relaying agent understands Injection-Date. In which case
> that SHOULD should not apply. IOW, what you are proposing should be
> regarded as only a temporary measure for use when Injection-Date is not
> observed (i.e.  a continuation of what some agents are currently doing
> anyway).

Yes, exactly.  I think that the root of our disagreement is again that I
believe said transition period until Injection-Date is widely enough
observed that an article with a very stale Date will propagate usefully is
on the order of five to ten years.

>> I don't agree with this at all.  I think that you're selling posting
>> agents short.  Many posting agent authors know very well what identity
>> model they're using and know that Date plus Message-ID make up the
>> message's identity, and provide header fields appropriately.  We will
>> cause problems if we break this.

> If you are saying that some posting agents deliberately don't add the
> Date header until they are actually about to inject, then that is fine,
> and in future they will be doing the same with Injection-Date. But I
> suspect the majority of current agents either create the Date at
> composition time, or else don't create it at all.

I believe that some posting agents deliberately add the Message-ID and
Date header so that they can inject the article with the same identity to
multiple servers simultaneously.  I know that at least some proxies have
that capability.  If each injecting agent then adds their own
Injection-Date, we break the identity model.

>> This will continue to work as expected in my proposal (just to be
>> clear) since in that case the injecting agent can add an Injection-Date
>> as well (and indeed MUST).

> Yes, but what I want is MUST when Injection-Date is absent, irrespective
> of whether Date is present or not. I still do not see what can go wrong
> in that situation. Can you construct an example?

Sure.  Suppose that I'm using a local news proxy which talks to three
different remote servers for me and presents me with a combined view of
newsgroups from those servers.  (This is an increasingly common substitute
for running a full-fledged local news server, and in some ways works much
nicer.)  Suppose that for maximum propagation (for exactly the same sorts
of reasons that you do something similar with C News), I have it
configured to post my messages to all of those remote servers at the same
time.

If one of those servers is down at the time of the posting, the safest and
most reliable behavior, and the one that I believe is currently
implemented in at least some software of this sort, is to queue the post
and keep trying.

So you potentially get multiple injection separated by time, which if
Injection-Date is added separately by each injection agent creates the
possibility of duplicate articles via the means that we've just been
discussing in this thread.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1415 USEPRO 3.2.1 - Number of path entries per site
Organization: The Eyrie
Date: Fri, 23 Feb 2007 21:47:50 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> If I'm in the midst of such a transition and I've informed some of my
>> peers of the new Path identity and haven't been able to reach others,
>> there's no way for me to satisfy a MUST here without generating
>> different Path headers for different peers.  Some peers will be
>> expecting one thing and some will be expecting another.

> If you start putting foo.bigbar.com as your leftmost <path-identity>
> (and if you don't go to the complication of generating different Paths
> for different peers), then the peers which haven't been informed yet are
> going to start putting !.MISMATCH.fooco.com! to the left of your Path,
> and there is nothing you can do to stop them without speaking to them. I
> think that is inevitable.

Right, which is why doing so can't be a violation of a MUST.  I think
we're actually agreeing here (unless you feel that using the identity that
you advertised to your peer should still be MUST instead of SHOULD).

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24485
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Date: Fri, 23 Feb 2007 17:35:48 GMT
Lines: 82
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> Yes, that's a fair summary of my position. For sure, a 3-day limit is
>> far too short. Current practice appears to be much longer than that, and
>> I can easily imagine situations where injection might be delayed that
>> long after composition of the article. And currently (so far as I can
>> tell) propagation does not deteriorate until something closer to 8 days.

>3-day limit?  Oh, hm:

>   3.   It SHOULD reject any article whose Date header field is more
>        than 24 hours into the future (and MAY use a margin less than 24
>        hours).  It SHOULD reject any article whose Date header appears
>        to be stale (more than 72 hours into the past, for example, or
>        too old to still be recorded in the database of a relaying agent
>        the injecting agent will be using) since not all news servers
>        support Injection-Date.

>That's really not saying what I wanted to say.  I think I borrowed that
>language from something else (probably the staleness checks for
>Injection-Date).

>What I really wanted to say there was something more like:

>    It SHOULD reject any article whose Date header is too far into the
>    past (too old to still be recorded in the database of a relaying agent
>    the injecting agent is using, for example).  This cutoff SHOULD NOT be
>    any shorter than 72 hours into the past.

>In other words, the 72 hours was intended to be a minimum, not a suggested
>value.

Well that is certainly better.

>I know you probably still disagree, but I didn't intend to be that harsh.

But if an injecting agent is aware of the cutoff of its immediately
following relaying agent (which is a reasonable assumption, because they
are often two aspects of the same agent), then it will also be aware
whether said relaying agent understands Injection-Date. In which case that
SHOULD should not apply. IOW, what you are proposing should be regarded as
only a temporary measure for use when Injection-Date is not observed (i.e.
a continuation of what some agents are currently doing anyway).

>> I disagree there. Posting agents have no concept of which identity model
>> thet are using.

>I don't agree with this at all.  I think that you're selling posting
>agents short.  Many posting agent authors know very well what identity
>model they're using and know that Date plus Message-ID make up the
>message's identity, and provide header fields appropriately.  We will
>cause problems if we break this.

If you are saying that some posting agents deliberately don't add the Date
header until they are actually about to inject, then that is fine, and in
future they will be doing the same with Injection-Date. But I suspect the
majority of current agents either create the Date at composition time, or
else don't create it at all.

>> Current practice has often been that they do not set the Date, and leave
>> it to the injecting agent to set it.

>This will continue to work as expected in my proposal (just to be clear)
>since in that case the injecting agent can add an Injection-Date as well
>(and indeed MUST).

Yes, but what I want is MUST when Injection-Date is absent, irrespective
of whether Date is present or not. I still do not see what can go wrong
in that situation. Can you construct an example?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24486
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1415 USEPRO 3.2.1 - Number of path entries per site
Date: Fri, 23 Feb 2007 17:48:21 GMT
Lines: 74
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>>> This language requires mind reading; you may have no idea what other sites
>>> expect.

>> Presuably they "expect" whatever was negotiated at the time the peering
>> arrangement was set up (which may, of course, amount to more than one
>> IP-address/domain-name/whatever).

>Right, but let's just say that rather than leaving it as a presumption.
>In other words, let's make the language entirely about what that site
>does, rather than about what other sites will do.

Yes, but what a site does is heavily constrained by what it has agreed
with its peers.

>Suppose that my current Path identity is fooco.com.  fooco just got bought
>out by bigbar.com and for various corporate political reasons is being
>absorbed.  So my news server and everything else is switching from being
>in fooco.com to being in foo.bigbar.com.

>If I'm in the midst of such a transition and I've informed some of my
>peers of the new Path identity and haven't been able to reach others,
>there's no way for me to satisfy a MUST here without generating different
>Path headers for different peers.  Some peers will be expecting one thing
>and some will be expecting another.

If you start putting foo.bigbar.com as your leftmost <path-identity> (and
if you don't go to the complication of generating different Paths for
different peers), then the peers which haven't been informed yet are going
to start putting !.MISMATCH.fooco.com! to the left of your Path, and there
is nothing you can do to stop them without speaking to them. I think that
is inevitable.

Obviously you try to avoid getting into that situation, but if your peers
just aren't interested in reconfiguring what they expect, then either you
live with it, or you stop feeding them.

>>> The present wording doesn't consider "!" to belong to any identity and
>>> instead treats it as a delimiter, so it's not following either of those
>>> conventions.  The precise steps seemed to read more clearly to me doing
>>> it that way.

>> I found that I had to read the wording *very* carefully, and more than
>> once, to convince myself that you had got your delimiters in exactly the
>> right places (as indeed you had).

>That's probably the case.  However, that reading isn't the one that I'm
>the most concerned with, since it's only us in this working group who have
>to do that careful check.  I'm more concerned with an implementor blindly
>following the text ending up with the correct result without stumbling
>over or missing some delimiter.

>I intentionally sacrificed minor difficulty of verifying the correctness
>of the text for what felt to me like more straightforward and harder to
>misread text from an implementor perspective.

>Of course, something that works for either is the best.

OK, I shall try and suggest a rewording as I said, but not this week :-( .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: application/news-message-id
Date:  Fri, 23 Feb 2007 19:32:38 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 18
Sender: [email-masked]

Charles Lindsey wrote:

>> http://tools.ietf.org/html/draft-ellermann-news-nntp-uri-05#section-8.3

> Ah! You are wanting to change it from an RFC 4288 registrartion to an
> RFC 4289 registration. That makes sense.

Yes, see http://permalink.gmane.org/gmane.ietf.types/799

> But you would still have to write a proper 4289 template for it.

What's a "proper 4289 template" ?  I've posted a pro forma review request
on the ietf-types mailing list as specified in chapter 2.2.2 of RFC 4289:

http://permalink.gmane.org/gmane.ietf.types/797

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Organization: The Eyrie
Date: Fri, 23 Feb 2007 09:31:03 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> That is fine for well-written agents, but there are an awful lot of
> people out there writing posting agents wo think that news is just a
> minor variant of email, and I would not trust them to do it right. They
> need to ensure that the Injection-Date header is not added until they
> actually have a connection to an NNTP server open and ready to accept
> the injection.

Well, I think you've already proven by experiment that if they miss by a
day, it's not a big deal.  It's only a problem if they miss by three or
four, at which point we're mostly talking about off-line readers, which
are more sophisticated clients to start with.  I'm more confident in the
ability of people who write off-line readers to get this right than people
who write a simple normal client.

> As far as USEPRO is concerned, we have to allow either party to add the
> Injection-Date. As to what Best Practice might be, that may be more or
> USEAGE.

In the absence of any stronger arguments than "some clients are stupid"
(and right now people don't seem to be presenting any), I'm opposed to
breaking the existing identity model for existing clients written before
the introduction of Injection-Date.  I don't think that's the sort of
non-backward-compatible change that's okay for us to make just because
some clients get it wrong.  I think we can provide mechanisms for clients
that want to do the right thing to do it right, but potentially breaking
the identity model of currently well-behaved clients to work around
problems with others doesn't make me comfortable.

So I only agree with this statement if there is no Date header in the
proto-article.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24483
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: application/news-message-id
Date: Fri, 23 Feb 2007 16:57:40 GMT
Lines: 22
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:
> 
>> But Frank seems to have found a text. Could someone post it all?

>http://tools.ietf.org/html/draft-ellermann-news-nntp-uri-05#section-8.3

Ah! You are wanting to change it from an RFC 4288 registrartion to an RFC
4289 registration. That makes sense. But you would still have to write a
proper 4289 template for it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24481
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Date: Fri, 23 Feb 2007 16:45:28 GMT
Lines: 50
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>Russ Allbery wrote:
>> Harald Alvestrand <[email-masked]> writes:

>>>- We state that existing implementations use Date: to check for
>>>staleness, and that will cause discarding of articles that are too
>>>old. Period, end of story, accepted.
>> 
>> 
>> Currently we don't say this for injecting and serving agents.

>I think it is important to note this. It is important to document how it works 
>today. The discarding happens at relays independently.  Some sites see the 
>article, some do not.  That's worse than refusing it at the point of
>injection.

It's not as bad as that, since if Site A refuses to relay it, it will
likely flood around A, and still arrive (but not as rapidly) at sites
beyind A. So it will be A's direct clients who will not be able to read
the article.

But yes, it would be useful to note this present practice. But when
Injection-Date is widely implemented, things will be better (not that any
vast number of articles actually get lost this way, even at present).

>We should not suggest a specific number of days cutoff, since the typical value 
>has changed, and is not uniform across sites.

Agreed.

>Would it be appropriate to write this as "In the case of delayed injection, 
>existing relaying and serving agents' article staleness checks on Date and not 
>Injection-Date will lead to non-uniform propagation. If injection is delayed, 
>the posting agent MAY [also] place the time of injection in the Date: header field."

I would not want to encourage this. A few current agents may be doing
this, and let them continue. But I would rather then moved to using
Injection-Date for such situations.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24480
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: application/news-message-id
Date: Fri, 23 Feb 2007 16:23:16 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> "Charles Lindsey" <[email-masked]> writes:

>In <[email-masked]> Harald Alvestrand <[email-masked]> writes:

>>Frank Ellermann wrote:
>>> Hi, what's an application/news-message-id ?
>>>
>>> Frank
>>>
>>>
>>>
>>>   
>>http://www.fileformat.info/info/mimetype/application/news-message-id/index.htm 
>>as well as http://www.iana.org/assignments/media-types/application/ sems 
>>to claim it's a Spencer-defined son-of-1036-ism.

>Well that first site seems to be down, and the second seems to indicate
>that IANA have now "lost" the official definition of that media type
>(there was a time when IANA used to store the actual templates. Now they
>just give references which are sometime broken).

That first site is up again, but all it does is to repeat what is on the
IANA site. SO it does indeed appear that the actual registration template
for that media type is pemanently lost - at least so far as trying to find
it via IANA is concerned.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24482
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Date: Fri, 23 Feb 2007 16:54:13 GMT
Lines: 46
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> Agreed. The posting agent is in a better position to know exactly what
>> is going on in the more unusual cases, and if he has available an
>> earlier Injection-Date than the one the injecting agent is likely to
>> apply, then it is safer for the injecting agent to accept his version
>> (and, after all, if the posting agent make the wrong call, then it(he)
>> has only it(him)self to blame if propagation suffers). This feature it
>> _not_ for normal straightforward posters to use.

>I don't agree with the last sentence.  I don't think there's any reason
>for a posting agent not to just always supply an Injection-Date similar to
>how many posting agents already always supply Date.

That is fine for well-written agents, but there are an awful lot of people
out there writing posting agents wo think that news is just a minor
variant of email, and I would not trust them to do it right. They need to
ensure that the Injection-Date header is not added until they actually have
a connection to an NNTP server open and ready to accept the injection.

As far as USEPRO is concerned, we have to allow either party to add the
Injection-Date. As to what Best Practice might be, that may be more or
USEAGE.

>The original goal, after all, is to move all the protocol weight of Date
>onto a new header without fundamentally changing anything, which means
>that anything that's fair game today with Date should probably also be
>fair game with Injection-Date.

Except that "fair game" for Date today covers a variety of practices, not
all of which are appropriate for Injection-Date (though 99% of the time
injectin follows composition so closely that it hardly matters - it is the
other 1% we are trying to make better).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24484
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1415 USEPRO 3.2.1 - Number of path entries per site
Date: Fri, 23 Feb 2007 17:07:14 GMT
Lines: 61
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>In message <[email-masked]>, Charles Lindsey
><[email-masked]> writes
>>
>>In <[email-masked]> Russ Allbery <rra@xxxxxxxxxxxx> writes:
>>
>>>Charles Lindsey <chl@xxxxxxxxxxxxxxxx> writes:
>>
>>>>    Except possibly when relaying to other hosts on the same site, every
>>>>    injecting, relaying, or serving agent that accepts an article MUST
>>>>    update the Path header field ....
>>
>>>I don't think this is necessary.  There are two possibilities: the site is
>>>using the Netnews protocol within the site or not.  If it is, it needs to
>>>fill out the Path header field for the same reasons that any other Netnews
>>>servers need to.
>>
>>Not necessarily, if it leads to a long string of identities for machines
>>on the one site which takes up much Path space without assisting in
>>preventing loops or enabling the detection of anomalies. 

>Russ is, I believe, saying it prevents loops within the "site" if
>they're using NNTP in an off-the-shelf way.

OK, we are talking about a site which passes articles through a succession
of machines:
 s3.example.com!s2.example.com!s1.example.com!...

If there is some risk of looping inside that site, then a newsmaster would
be stupid to omit s2.example.com from the list. But that is really his own
(internal) problem. My point was that s2.example.com was unlikely to be of
interest for preventing loops in the rest of the net, and hence that site
should be allowed to omit it (MAY) if it was not otherwise useful for its
own purposes.

>However, it may also help when the site has s1.example.com and
>s2.example.com for peering purposes. Adding s2.example.com into the Path
>header field, even it was only handled by the s1 machine, ought to
>prevent it being offered back by a peer that only talks to s2.

Sure. Newsmasters need to be aware of all such matters, and to set their
Paths accordingly.

>It would be better to add a generic site-wide name (example.com) and
>ensure that peers understood that s2 was also "example.com" 

And that also may be useful, again depending on the particular
circumstances. Your 'demon' case was a good example of that.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: application/news-message-id
Date:  Wed, 21 Feb 2007 17:25:32 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 8
Sender: [email-masked]

Charles Lindsey wrote:
 
> But Frank seems to have found a text. Could someone post it all?

http://tools.ietf.org/html/draft-ellermann-news-nntp-uri-05#section-8.3

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: content of Approved header
Date:  Wed, 21 Feb 2007 17:20:51 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 13
Sender: [email-masked]

Charles Lindsey wrote:

>> There's no "@" in foobar.com, it's a syntax error.
 
> Oops! "Approved: [email-masked]".

Oh, only a typo.  Something with your references is broken,
 
I still think that the USEFOR wording is good enough, you
quoted it in <[email-masked]>

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Organization: The Eyrie
Date: Tue, 20 Feb 2007 21:27:10 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Yes, that's a fair summary of my position. For sure, a 3-day limit is
> far too short. Current practice appears to be much longer than that, and
> I can easily imagine situations where injection might be delayed that
> long after composition of the article. And currently (so far as I can
> tell) propagation does not deteriorate until something closer to 8 days.

3-day limit?  Oh, hm:

   3.   It SHOULD reject any article whose Date header field is more
        than 24 hours into the future (and MAY use a margin less than 24
        hours).  It SHOULD reject any article whose Date header appears
        to be stale (more than 72 hours into the past, for example, or
        too old to still be recorded in the database of a relaying agent
        the injecting agent will be using) since not all news servers
        support Injection-Date.

That's really not saying what I wanted to say.  I think I borrowed that
language from something else (probably the staleness checks for
Injection-Date).

What I really wanted to say there was something more like:

    It SHOULD reject any article whose Date header is too far into the
    past (too old to still be recorded in the database of a relaying agent
    the injecting agent is using, for example).  This cutoff SHOULD NOT be
    any shorter than 72 hours into the past.

In other words, the 72 hours was intended to be a minimum, not a suggested
value.

I know you probably still disagree, but I didn't intend to be that harsh.

> I disagree there. Posting agents have no concept of which identity model
> thet are using.

I don't agree with this at all.  I think that you're selling posting
agents short.  Many posting agent authors know very well what identity
model they're using and know that Date plus Message-ID make up the
message's identity, and provide header fields appropriately.  We will
cause problems if we break this.

> Current practice has often been that they do not set the Date, and leave
> it to the injecting agent to set it.

This will continue to work as expected in my proposal (just to be clear)
since in that case the injecting agent can add an Injection-Date as well
(and indeed MUST).

> But that preactice is already changing, since mail user agents tend to
> write the Date at composition time, and lots of mail user agents are
> also news user agents (or think they are :-( ), and hence do the same
> thing.

I think the point of disagreement here is my belief that maintaining the
current identity model and not breaking posting agents that are using it
intentionally is more important than making use of Date for composition
date quietly work without changing posting agents.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24460
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Date: Tue, 20 Feb 2007 17:17:58 GMT
Lines: 109
Sender: [email-masked]

In <[email-masked]> Russ Allbery <rra@xxxxxxxxxxxx> writes:

>So this proposal as stated doesn't make sense to me, since it talks about
>keeping Date consistent when the only reason to keep Date consistent
>(beyond generally not wanting to modify articles and keeping the human
>information component) is something that we're phasing out.

Well "not wanting to modify articles" is an important principle of Usenet,
and so one should not be tinkering with the Date for the same reason that
one should not be tinkering with the Subject, or any other header, and for
that matter the body.

>One of the reasons why I believe injecting agents SHOULD continue to check
>Date for staleness is that I believe no propagation plus a clear error
>message is better than spotty propagation and no diagnostics.  I believe,
>from other messages, that Charles disagrees and would prefer spotty
>propagation improving as (hopefully) more hosts support Injection-Date.
>The disagreement may to some extent depend on one's beliefs about how fast
>Injection-Date adoption will happen.

Yes, that's a fair summary of my position. For sure, a 3-day limit is far
too short. Current practice appears to be much longer than that, and I can
easily imagine situations where injection might be delayed that long after
composition of the article. And currently (so far as I can tell)
propagation does not deteriorate until something closer to 8 days.

>It's not the sort of loop that melts down a newsgroup, so it's not as big
>of a deal in that sense.  The loop has to be time-delayed.  But it does
>feel like a fairly fundamental failure of the protocol to me.

Indeed.

>I think the main alternative (assuming we're keeping Injection-Date) looks
>like this:

> * Servers are required to check Injection-Date in preference to Date for
>   determining staleness.

> * Posting agents may provide Injection-Date in proto-articles.  They MUST
>   do so if doing multiple injection.  If Injection-Date is provided, the
>   injecting agent MUST NOT modify it.

Yes, I agree entirely so far (though I might have SHOULD for that first
MUST - but not for the second).

But I am not so sure abut the next bit.

> * If the proto-article contains neither a Date nor an Injection-Date
>   header field, the injecting agent MUST add both.

> * If the proto-article contains only a Date header but no Injection-Date
>   header, it's using the old article identity model.  If multiple
>   injection is done, Date is provided, and no Injection-Date header is
>   ever added, everything works as it does now.  If multiple injection is
>   done, Injection-Date is provided, and all agents check Injection-Date,
>   everything works as it does now.  We have to keep things into one of
>   those two cases as much as possible to maintain the existing guarantees
>   of the protocol, so I think an injecting agent, when presented with a
>   proto-article that contains a Date but no Injection-Date, must assume
>   the old identity model is being used and MUST NOT add Injection-Date.
>   This means that people who want to send time-delayed messages will need
>   to update their posting agents to add Injection-Date.  I think that's
>   reasonable.

I disagree there. Posting agents have no concept of which identity model
thet are using. Current practice has often been that they do not set the
Date, and leave it to the injecting agent to set it. But that preactice is
already changing, since mail user agents tend to write the Date at
composition time, and lots of mail user agents are also news user agents
(or think they are :-( ), and hence do the same thing.

So what I am hoping to see in the future is that posting agents will
routinely add a Date at composition time (following the RFC 2822 practice,
which we are now adopting) but that they will still leave it to the injecting
date to add the Injection-Date. So that will not imply an "old article
identity model", but rather the Best Current Practice. But that then means
that an injecting agent MUST supply an Injection-Date if none was present
in the article as submitted.

>Remaining issues are to what degree we want to standardize serial multiple
>injection -- I think it may be worthwhile describing the specific steps
>one takes to turn an article into a proto-article somewhere -- and whether
>we need to deal with stale pre-existing Date/Injection-Date headers or
>just ignore the issue and let people make it up as they go if they end up
>in a situation where stale Date/Injection-Date headers are preventing them
>from doing some gatewaying they want to do.

People who know what they are doing (e.g. they are running their own
private servers, or are arranging for multipoint injection) are the ones
who are likely to provide their own Injection-Dates, or even to remove
existing ones if they really REALLY know that the article cannot already
have leaked out onto Usenet. How much we say/recommend/warn-against (or
don't say/etc) is a matter for discussion after some further examination
of the risks. But I don't think that need affect the broad picture, which
I think should be as outlined above 

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1415 USEPRO 3.2.1 - Number of path entries per site
Organization: The Eyrie
Date: Tue, 20 Feb 2007 09:48:37 -0800
Sender: [email-masked]

Richard Clayton <[email-masked]> writes:

> It would be better to add a generic site-wide name (example.com) and
> ensure that peers understood that s2 was also "example.com" and
> shouldn't be offered articles with that generic identity as well as a
> specific one, but it seems to me that's merely a matter of what is
> neater, not an interop issue

Note that this is sometimes the case and sometimes isn't.  For example, I
have two external relaying agents, and I want each to receive articles
even if they'd been processed by the other since it gives me more
redundancy.  If something goes wrong between the two relaying agents, this
gives me more assurance that I won't miss articles.

In nearly every case, the other relaying agent will immediately decline
the message anyway (since it will have already gotten it from the first
relaying agent), so the additional work required for my peers is minimal
and since the two relaying agents don't feed the same peers, it would
rarely even be noticable.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Tue, 20 Feb 2007 17:41:02 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1415 USEPRO 3.2.1 - Number of path entries per site
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes
>
>In <[email-masked]> Russ Allbery <rra@xxxxxxxxxxxx> writes:
>
>>Charles Lindsey <chl@xxxxxxxxxxxxxxxx> writes:
>
>>>    Except possibly when relaying to other hosts on the same site, every
>>>    injecting, relaying, or serving agent that accepts an article MUST
>>>    update the Path header field ....
>
>>I don't think this is necessary.  There are two possibilities: the site is
>>using the Netnews protocol within the site or not.  If it is, it needs to
>>fill out the Path header field for the same reasons that any other Netnews
>>servers need to.
>
>Not necessarily, if it leads to a long string of identities for machines
>on the one site which takes up much Path space without assisting in
>preventing loops or enabling the detection of anomalies. 

Russ is, I believe, saying it prevents loops within the "site" if
they're using NNTP in an off-the-shelf way.

Preventing loops within the site seems like a good idea to me, and so it
will seem to the newsadmins, who would ignore instructions not to add
all relevant names :)  Essentially it's not a site at all, just
ownership of multiple machines.

However, it may also help when the site has s1.example.com and
s2.example.com for peering purposes. Adding s2.example.com into the Path
header field, even it was only handled by the s1 machine, ought to
prevent it being offered back by a peer that only talks to s2.

It would be better to add a generic site-wide name (example.com) and
ensure that peers understood that s2 was also "example.com" and
shouldn't be offered articles with that generic identity as well as a
specific one, but it seems to me that's merely a matter of what is
neater, not an interop issue

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1415 USEPRO 3.2.1 - Number of path entries per site
Organization: The Eyrie
Date: Tue, 20 Feb 2007 09:33:29 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <rra@xxxxxxxxxxxx> writes:

>> I don't think this is necessary.  There are two possibilities: the site
>> is using the Netnews protocol within the site or not.  If it is, it
>> needs to fill out the Path header field for the same reasons that any
>> other Netnews servers need to.

> Not necessarily, if it leads to a long string of identities for machines
> on the one site which takes up much Path space without assisting in
> preventing loops or enabling the detection of anomalies.

If having accurate Path entries doesn't assist in preventing loops or
enabling the detection of anomalies, wouldn't that be the definition of
"not using the Netnews protocol"?  Since when using the Netnews protocol,
that's exactly what the Path header does do.

> I understood that the WG had agreed to that principle, which is why it
> was in Usepro-06. But it is not a huge deal if the WG prefers
> otherwise. So we need other opinions (so far I count just +1 from me and
> -1 from you).

I agree that other opinions would be useful.

>> Nothing *requires* it, either, so we should look at that.

> OK, I think we are agreed that the wording needs to change.

Well, that's stronger than my feeling about it, but I'm not adverse to the
addition of some statement about this.

>>> My suggested fix was to replace both those steps by

>>>    4. The agent MUST then prepend one or more <path-identity>s
>>>    identifying itself (as set out in section 3.2) to the Path header
>>>    field content, separated by either "!!" or "!". However, the last
>>>    (leftmost) such <path-identity> so appended MUST be one that is
>>>    expected by the destination site when it in turn comes to apply Step
>>>    3 above.

>> This language requires mind reading; you may have no idea what other sites
>> expect.

> Presuably they "expect" whatever was negotiated at the time the peering
> arrangement was set up (which may, of course, amount to more than one
> IP-address/domain-name/whatever).

Right, but let's just say that rather than leaving it as a presumption.
In other words, let's make the language entirely about what that site
does, rather than about what other sites will do.

>> It's also hard to satisfy this MUST during a transition where some
>> peers may expect one ordering and others may be expecting a different
>> one.

> I think all sites will currently be assuming (insofar as they bother to
> assume anything) that the leftmost identity in the Path as received is
> the place the article claims to have come from.

I think you missed my point, since as near as I can tell your sentence
isn't related to my sentence.  So let me expand a little.

Suppose that my current Path identity is fooco.com.  fooco just got bought
out by bigbar.com and for various corporate political reasons is being
absorbed.  So my news server and everything else is switching from being
in fooco.com to being in foo.bigbar.com.

If I'm in the midst of such a transition and I've informed some of my
peers of the new Path identity and haven't been able to reach others,
there's no way for me to satisfy a MUST here without generating different
Path headers for different peers.  Some peers will be expecting one thing
and some will be expecting another.

>> The present wording doesn't consider "!" to belong to any identity and
>> instead treats it as a delimiter, so it's not following either of those
>> conventions.  The precise steps seemed to read more clearly to me doing
>> it that way.

> I found that I had to read the wording *very* carefully, and more than
> once, to convince myself that you had got your delimiters in exactly the
> right places (as indeed you had).

That's probably the case.  However, that reading isn't the one that I'm
the most concerned with, since it's only us in this working group who have
to do that careful check.  I'm more concerned with an implementor blindly
following the text ending up with the correct result without stumbling
over or missing some delimiter.

I intentionally sacrificed minor difficulty of verifying the correctness
of the text for what felt to me like more straightforward and harder to
misread text from an implementor perspective.

Of course, something that works for either is the best.

>> I don't think we gain much from this.  This checking is not
>> particularly formal, so I think more specification implies more
>> rigorous verification than we're actually specifying.  "Expected" feels
>> to me like it captures about the level of checking and formality we're
>> getting.  The definition of the term is the standard English definition
>> of the word.

> I think my main concern here is to make it clear that relaying agents
> have an obligation to know where articles have come from and to insert
> diagnostics accordingly. I think it is now agreed that this is at a
> SHOULD level.

I'm not adverse to sticking something in, although I'm not really sure
what to say.

To give others an idea of how this generally goes, I'd estimate (back of
the envelope guess) that at around half of my peers are using a Path
identity other than the FQDN of the news server from which I get the
article, but 90% of my peers told me their Path identity when
communicating with me about peering.  So in practice, the best way (and
probably the only feasible way) to establish these Path identities is
through out-of-band communication.  If you try to guess them based on
FQDN, you'll get it wrong about half the time.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Organization: The Eyrie
Date: Tue, 20 Feb 2007 09:21:33 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Agreed. The posting agent is in a better position to know exactly what
> is going on in the more unusual cases, and if he has available an
> earlier Injection-Date than the one the injecting agent is likely to
> apply, then it is safer for the injecting agent to accept his version
> (and, after all, if the posting agent make the wrong call, then it(he)
> has only it(him)self to blame if propagation suffers). This feature it
> _not_ for normal straightforward posters to use.

I don't agree with the last sentence.  I don't think there's any reason
for a posting agent not to just always supply an Injection-Date similar to
how many posting agents already always supply Date.

The original goal, after all, is to move all the protocol weight of Date
onto a new header without fundamentally changing anything, which means
that anything that's fair game today with Date should probably also be
fair game with Injection-Date.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24456
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: content of Approved header
Date: Tue, 20 Feb 2007 14:51:44 GMT
Lines: 24
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx> writes:

>Charles Lindsey wrote:
> 
>> consider "Approved: foobar.com", which is syntactically allowed but
>> still discouraged by USEFOR.

>Are we reading the same copy of RFC 2822 ?  The article standard says
><mailbox-list>.  RFC 2822 says that this is one or more <mailbox>, 
>and a <mailbox> is either <addr-spec> or <name-addr>.  There's no "@"
>in foobar.com, it's a syntax error.

Oops! "Approved: [email-masked]".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24455
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: application/news-message-id
Date: Tue, 20 Feb 2007 14:50:53 GMT
Lines: 34
Sender: [email-masked]

In <[email-masked]> Harald Alvestrand <[email-masked]> writes:

>Frank Ellermann wrote:
>> Hi, what's an application/news-message-id ?
>>
>> Frank
>>
>>
>>
>>   
>http://www.fileformat.info/info/mimetype/application/news-message-id/index.htm 
>as well as http://www.iana.org/assignments/media-types/application/ sems 
>to claim it's a Spencer-defined son-of-1036-ism.

Well that first site seems to be down, and the second seems to indicate
that IANA have now "lost" the official definition of that media type
(there was a time when IANA used to store the actual templates. Now they
just give references which are sometime broken).

But Frank seems to have found a text. Could someone post it all?

But this looks like something we ought to have obsoleted in USEFOR at the
same time as we obsoleted message/news.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24457
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: Possibility to use non-resolvable domain name as path-identity
Date: Tue, 20 Feb 2007 15:19:00 GMT
Lines: 55
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx> writes:

>Charles Lindsey wrote:

>> if a site has outgoing relayers "news1.site.example",
>> "news2.site.example", etc, it still might use "site.example" as
>> its <path-identity>, but in that case I would like to see at
>> least an MX record for "site.example".

>That's already covered by RFC 2142, abuse@xxxxxxxxxxxxxxx has to
>work, they need either an MX or an address with an smtpd.

But we decided not to go that route (see #1093 below).

>>> say that "a path-identity SHOULD be the fully qualified domain
>>> name of the corresponding host (here news server)", and be done
>>> with it.  If it has no IP it has a good excuse to pick 'something'
>>> else, and then 'something' with an MX might be the second best
>>> choice, but maybe reality is weirder than we can imagine.
> 
>> Well, since that's almost exactly what I want it to say, can I
>> take it that you 'second' the Issue?

>As continuation of the #1093 saga, yes, it makes no sense to track
>the in essence same issue twice.

#1093 related to the 2142 issue, not the domain-name issue. So I request
Harald either to include the domain-name issus within #1093, or (better)
to create a new Issue for it.

As for #1093, I am surprised to find it still open. It reached a clear
consensus that we were not going to mandate any RFC 2142 adherence, but
Harald did agree that it would still be in order to draw attention to the
matter in a NOTE, and that is all I am currently asking for (indeed, the
NOTE text I have proposed was partly written by Harald).

>>>> we have agreed that we do not intend either to supersede/obsolete/
>>>> commend it.
> 
>>> TINW.  I certainly don't agree.

#1093 seems to have been resolved otherwise. You can, off course, request
it be reopened, but on re-reading the discussion I think you would find it
an upwards struggle.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24459
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Date: Tue, 20 Feb 2007 16:43:36 GMT
Lines: 42
Sender: [email-masked]

In <[email-masked]> Russ Allbery <rra@xxxxxxxxxxxx> writes:

>Harald Alvestrand <harald@xxxxxxxxxxxxx> writes:

>> - What you are calling "reinjection" is what I call "multipoint
>> injection, where one of the points listens to a Netnews network to get
>> hold of the proto-article".

>I think that's a reasonable simplification of terms.

Not so sure. As used hitherto, "multipoint injection" has always implied
one site (posting agent) that injected in several places at essentially
the same time. "Reinjection" implied that an article had wandered around
through assorted sites/networks/gateways/whatever and then got injected
again (whether deliberately or accidentaiiy). As such, it is inherently
more dangerous than the simpler multipoint case. So I suggest we keep both
terms going until we understand exactly what we are going to do

>> I have yet to understand any tangible benefit from your proposal to
>> allow the posting-agent to specify injection-date.

>I tried to address this (at some length, unfortunately) in my message
>about the identity model of an article.

Agreed. The posting agent is in a better position to know exactly what is
going on in the more unusual cases, and if he has available an earlier
Injection-Date than the one the injecting agent is likely to apply, then
it is safer for the injecting agent to accept his version (and, after all,
if the posting agent make the wrong call, then it(he) has only
it(him)self to blame if propagation suffers). This feature it _not_ for
normal straightforward posters to use.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24458
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1415 USEPRO 3.2.1 - Number of path entries per site
Date: Tue, 20 Feb 2007 16:17:16 GMT
Lines: 137
Sender: [email-masked]

In <[email-masked]> Russ Allbery <rra@xxxxxxxxxxxx> writes:

>Charles Lindsey <chl@xxxxxxxxxxxxxxxx> writes:

>>    Except possibly when relaying to other hosts on the same site, every
>>    injecting, relaying, or serving agent that accepts an article MUST
>>    update the Path header field ....

>I don't think this is necessary.  There are two possibilities: the site is
>using the Netnews protocol within the site or not.  If it is, it needs to
>fill out the Path header field for the same reasons that any other Netnews
>servers need to.

Not necessarily, if it leads to a long string of identities for machines
on the one site which takes up much Path space without assisting in
preventing loops or enabling the detection of anomalies. I understood that
the WG had agreed to that principle, which is why it was in Usepro-06. But
it is not a huge deal if the WG prefers otherwise. So we need other
opinions (so far I count just +1 from me and -1 from you).

>Nothing in the document should be requiring the components of a news
>server be running on a single system.

Agreed. Your wording ensured at least that. My wording went a little
further.

>> 2. The present Steps 4 and 5 of 3.2.1 are just plain wrong.

>You're reading more into steps 4 and 5 than is actually there.  :) ...

>   4.  The agent MUST then prepend its own <path-identity> to the Path
>       header field content.

>   5.  The agent MAY then prepend additional <path-identity>s for itself
>       to the Path header field content, following each <path-identity>
>       so added with either "!!" or "!".  This is permitted for agents
>       that have multiple <path-identity>s (such as during a transition
>       from one to another).  Each of these <path-identity>s MUST meet
>       the requirements set out in Section 3.2.

The wording seems (to me, and hence possibly to others) to be
distinguishing between
   "its own <path-identity>"      {i.e. the one, true identity}, and
   "additional <path-identity>s"  {i.e. less important ones}

>Nothing *requires* it, either, so we should look at that.

OK, I think we are agreed that the wording needs to change.

>> Note that such "extra" identities are only put there because that site
>> does not want to receive articles with any of those identities already
>> present from its upstreams, and for historical reasons it needed more
>> than one identity for that purpose.

>That's one possible reason.  Another reason for appending such extra
>identities is because an ISP with independent news servers wants to add a
>common identity to the Path for statistical purposes. ...

Good point.

>> My suggested fix was to replace both those steps by

>>    4. The agent MUST then prepend one or more <path-identity>s
>>    identifying itself (as set out in section 3.2) to the Path header
>>    field content, separated by either "!!" or "!". However, the last
>>    (leftmost) such <path-identity> so appended MUST be one that is
>>    expected by the destination site when it in turn comes to apply Step
>>    3 above.

>This language requires mind reading; you may have no idea what other sites
>expect.

Presuably they "expect" whatever was negotiated at the time the peering
arrangement was set up (which may, of course, amount to more than one
IP-address/domain-name/whatever).

>  It's also hard to satisfy this MUST during a transition where
>some peers may expect one ordering and others may be expecting a different
>one.

I think all sites will currently be assuming (insofar as they bother to
assume anything) that the leftmost identity in the Path as received is the
place the article claims to have come from.

>    If the agent prepends additional <path-identity>s, the leftmost
>    <path-identity> SHOULD be the one advertised to other news servers as
>    the expected <path-identity> for this agent.

Yes, something like that conveys the right idea.

>> Note. There is some confusion as to whether a "!" belongs to the
>> identity/diagnostic to its left, or to its right,...

>The present wording doesn't consider "!" to belong to any identity and
>instead treats it as a delimiter, so it's not following either of those
>conventions.  The precise steps seemed to read more clearly to me doing it
>that way.

I found that I had to read the wording *very* carefully, and more than
once, to convince myself that you had got your delimiters in exactly the
right places (as indeed you had).

>> So some further tidying up of the whole wording might be a good idea.

>Please do suggest other wording if you think there's a clearer way to
>explain it.

OK. I shall put my deputy editor's hat on and see what I can come up with.

>> And, for my final niggle, please can we have a definition of the term
>> "expected" which is used at several places in 3.2.1.

>I don't think we gain much from this.  This checking is not particularly
>formal, so I think more specification implies more rigorous verification
>than we're actually specifying.  "Expected" feels to me like it captures
>about the level of checking and formality we're getting.  The definition
>of the term is the standard English definition of the word.

I think my main concern here is to make it clear that relaying agents
have an obligation to know where articles have come from and to insert
diagnostics accordingly. I think it is now agreed that this is at a SHOULD
level.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 20 Feb 2007 08:16:46 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Sender: [email-masked]

Russ Allbery wrote:
> Harald Alvestrand <[email-masked]> writes:

>>- We state that existing implementations use Date: to check for
>>staleness, and that will cause discarding of articles that are too
>>old. Period, end of story, accepted.
> 
> 
> Currently we don't say this for injecting and serving agents.

I think it is important to note this. It is important to document how it works 
today. The discarding happens at relays independently.  Some sites see the 
article, some do not.  That's worse than refusing it at the point of
injection.

We should not suggest a specific number of days cutoff, since the typical value 
has changed, and is not uniform across sites.

Would it be appropriate to write this as "In the case of delayed injection, 
existing relaying and serving agents' article staleness checks on Date and not 
Injection-Date will lead to non-uniform propagation. If injection is delayed, 
the posting agent MAY [also] place the time of injection in the Date: header field."

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Organization: The Eyrie
Date: Mon, 19 Feb 2007 23:33:50 -0800
Sender: [email-masked]

Harald Alvestrand <[email-masked]> writes:

> - In my proposal, injection-date: identifies the time of
> injection. Period, end of story. It is, in my opinion, harmful and of no
> benefit to let the posting-agent specify this. The injecting agent knows
> its own current time, and there is no later agent that is more
> trustworthy to leave the decision to.

Well, I don't think you can say that it's of no benefit.  It's clearly of
benefit in establishing a consistent identity for the article, which will
then prevent loops or duplicates from appearing.  Whether that benefit is
worth the other costs is another question, but it's not non-existent.

> - What you are calling "reinjection" is what I call "multipoint
> injection, where one of the points listens to a Netnews network to get
> hold of the proto-article".

I think that's a reasonable simplification of terms.

> I have yet to understand any tangible benefit from your proposal to
> allow the posting-agent to specify injection-date.

I tried to address this (at some length, unfortunately) in my message
about the identity model of an article.  The advantage is that it lets the
posting agent specify the complete identity of the article, and therefore
multiply injected copies of the same article can all have the same
identity when that injection is done by a well-behaved posting agent.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Organization: The Eyrie
Date: Mon, 19 Feb 2007 23:29:10 -0800
Sender: [email-masked]

Harald Alvestrand <[email-masked]> writes:

> Relevant technical information from the thread:

> - Injecting a message (with an unique Message-ID:) several times into a
> single Usenet network is supported, and it's important that it remains
> supported ("multipoint injection").
> - Current USENET servers (when acting as relaying agents) check Date: for
> staleness (rejecting articles that have a Date: that is so old that their
> history files won't cover the message-ID).
> - Injection of articles into USENET where the client posting agent sends
> to someone acting as an injecting agent, and the article then ending up
> with someone acting as a posting agent towards another injecting agent
> ("reinjection from an isolated network"), is quite common.
> - No technical means exists today for verifyng whether a network is truly
> isolated.
> - No existing mail-to-news gateways modify Date:

Yup, all of those technical points look correct to me.

> Suggested hacks include:
> - Changing Date: at injection to be within a short distance from the
> current time
> - Allowing posting agent to set Injection-Date:, to ensure it's consistent
> between injected copies
> - Drop Injection-Date: from the spec

Also there's the current draft, which sets Injection-Date at injection to
the current time but leaves Date alone.

> This is very much on the "Keep It Simple" approach to protocol design.

> - Date: identifies the date of composition. Don't change it, ever.
> - Injection-Date: identifies the date of injection, by the last agent that
> identifies itself as an injecting agent. Proto-articles don't have
> it. Relaying and serving agents never, ever change it.

So far, this is what's in the current spec.

> - "Reinjection" doesn't exist. There is only multipoint injection. The
> means by which a proto-article (including message-ID, but NOT including
> Injection-Date:) gets from wherever it started out to the posting agent is
> not defined by the protocol; if the protocol happens to be NNTP, that's
> not an issue for this specification. (we can add warnings...)
> - When doing multipoint injection, Message-ID: MUST be consistent in all
> copies. Date: SHOULD be consistent, if it's not, Bad Things Can Happen.

As long as Injection-Date is in the specification, keeping Date consistent
is mostly meaningless, since our specification says that the header field
that goes into the identity of the article isn't Date, it's
Injection-Date.  Any reason you have for keeping Date consistent, in the
current draft, applies to Injection-Date instead, since we're replacing
the protocol use of Date with Injection-Date.

So this proposal as stated doesn't make sense to me, since it talks about
keeping Date consistent when the only reason to keep Date consistent
(beyond generally not wanting to modify articles and keeping the human
information component) is something that we're phasing out.

Anyway, your summary is basically the current draft.  It doesn't address
the identified problem in the current draft that different de facto
identities are given to the different copies of multiply injected
articles, which can potentially result in duplicates (with the chances
increasing as the time difference between the two injections increases).
I can't tell from your note whether that was intentional and you just
don't consider this much of a problem, or whether you'd missed that aspect
of the problem.

For the rest, annotating te proposal to make it clearer what the
differences are from the current draft.

> On the stale article problem:

> - We state that existing implementations use Date: to check for
> staleness, and that will cause discarding of articles that are too
> old. Period, end of story, accepted.

Currently we don't say this for injecting and serving agents.

> - We recommend (require?) that injecting agents refuse posting of "too
> old" articles by Date: - the exact acceptable Date: range should be a
> local decision.

This is what the draft currently has.

> - We recommend that the staleness check use Injection-Date: if present,
> otherwise Date:

This is what the draft currently has, except that it's mandatory, not just
recommended.

> - We state that if stale article rejection is an issue for people, they
> should:
>  - Ensure that their injecting agent adds Injection-Date:
>  - Put pressure on the discarding agent to check Injection-Date: if present

We don't currently say anything about this, but it's to a degree implicit
in the Injection-Date support.

> Case analysis:
> If a message has Date: on day 0, and is injected on day 7, into a network
> where anything older than 3 days is considered stale, but the injecting
> agents accept 14-day-old proto-articles, this will happen:

> - Servers that check Injection-Date: will relay the message. More
> propagation than today.
> - Servers that check Date: will reject the message. Unchanged from today.
> - Servers that feed from a server that checks Date: will not get the
> message in the normal feed; they may get it offered from feeds that go via
> Injection-Date: checking servers.

Agreed.

One of the reasons why I believe injecting agents SHOULD continue to check
Date for staleness is that I believe no propagation plus a clear error
message is better than spotty propagation and no diagnostics.  I believe,
from other messages, that Charles disagrees and would prefer spotty
propagation improving as (hopefully) more hosts support Injection-Date.
The disagreement may to some extent depend on one's beliefs about how fast
Injection-Date adoption will happen.

> If a message has Date: on day 0, gets injected on day 1 into the network,
> and gets injected again on day 7 into the same network, the following will
> happen:

> - Servers that check Date: will see the first copy.
> - Servers that check Injection-Date: will relay the message twice,
> possibly causing users to see it twice.

> The last looks like a loop, but can be solved by sysadmins beating up on
> the injecting agent to enforce stricter policing of posting of old
> articles.
> Is the latter a Big Deal?

Well, it's a Big Deal in the sense that it's happened in the past with
agents that discard Date, it's far more likely to happen when ignoring
Date becomes the standard way the servers are supposed to behave, and
every time it's happened that I've seen it caused a big stink and people
got rather upset that the software let it happen.

It's not the sort of loop that melts down a newsgroup, so it's not as big
of a deal in that sense.  The loop has to be time-delayed.  But it does
feel like a fairly fundamental failure of the protocol to me.

I think the main alternative (assuming we're keeping Injection-Date) looks
like this:

 * Servers are required to check Injection-Date in preference to Date for
   determining staleness.

 * Posting agents may provide Injection-Date in proto-articles.  They MUST
   do so if doing multiple injection.  If Injection-Date is provided, the
   injecting agent MUST NOT modify it.

 * If the proto-article contains neither a Date nor an Injection-Date
   header field, the injecting agent MUST add both.

 * If the proto-article contains only a Date header but no Injection-Date
   header, it's using the old article identity model.  If multiple
   injection is done, Date is provided, and no Injection-Date header is
   ever added, everything works as it does now.  If multiple injection is
   done, Injection-Date is provided, and all agents check Injection-Date,
   everything works as it does now.  We have to keep things into one of
   those two cases as much as possible to maintain the existing guarantees
   of the protocol, so I think an injecting agent, when presented with a
   proto-article that contains a Date but no Injection-Date, must assume
   the old identity model is being used and MUST NOT add Injection-Date.
   This means that people who want to send time-delayed messages will need
   to update their posting agents to add Injection-Date.  I think that's
   reasonable.

Remaining issues are to what degree we want to standardize serial multiple
injection -- I think it may be worthwhile describing the specific steps
one takes to turn an article into a proto-article somewhere -- and whether
we need to deal with stale pre-existing Date/Injection-Date headers or
just ignore the issue and let people make it up as they go if they end up
in a situation where stale Date/Injection-Date headers are preventing them
from doing some gatewaying they want to do.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1415 USEPRO 3.2.1 - Number of path entries per site
Organization: The Eyrie
Date: Mon, 19 Feb 2007 22:56:49 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> 1. To permit sites to omit path entries which merely reflect their
> internal structure without being of any benefit to the world outside. My
> suggested rewording of the first paragraph of 3.2.1 was

>    Except possibly when relaying to other hosts on the same site, every
>    injecting, relaying, or serving agent that accepts an article MUST
>    update the Path header field ....

I don't think this is necessary.  There are two possibilities: the site is
using the Netnews protocol within the site or not.  If it is, it needs to
fill out the Path header field for the same reasons that any other Netnews
servers need to.  If it's not, then those components are, from the Netnews
perspective, conceptually a single news server, and hence the existing
language already permits omitting Path entries.

Nothing in the document should be requiring the components of a news
server be running on a single system.

> 2. The present Steps 4 and 5 of 3.2.1 are just plain wrong.

> Since receiving agents are expected to do a MISMATCH check on articles
> relayed to them (inserting "!!" or "!.MISMATCH...." accordingly), and
> only look at the leftmost <path-identity> when doing that check, it is
> essential that the 'true' identity of the site (the ones its downstreams
> will be 'expecting') be the last one to be prepended, after prepending
> any extra ones (e.g. a <path-nodot> such as 'demon'). However, Steps 4
> and 5 currently prepend these things in the wrong order, which could
> cause MISMATCH checks to fail.

You're reading more into steps 4 and 5 than is actually there.  :)  That
doesn't mean that it's not a good idea to change the wording if it's
confusing, but what you're arguing for isn't contradicated by the current
language.

   4.  The agent MUST then prepend its own <path-identity> to the Path
       header field content.

   5.  The agent MAY then prepend additional <path-identity>s for itself
       to the Path header field content, following each <path-identity>
       so added with either "!!" or "!".  This is permitted for agents
       that have multiple <path-identity>s (such as during a transition
       from one to another).  Each of these <path-identity>s MUST meet
       the requirements set out in Section 3.2.

This doesn't append the identities in the wrong order.  It appends them in
an unspecified order, as there's no distinction being made here between
the identity appended in step 4 and the identities possibly appended in
step 5.  They're all just identities for the site.  So nothing prevents
the identities from being appended in the order that minimizes MISMATCH.

Nothing *requires* it, either, so we should look at that.

> Note that such "extra" identities are only put there because that site
> does not want to receive articles with any of those identities already
> present from its upstreams, and for historical reasons it needed more
> than one identity for that purpose.

That's one possible reason.  Another reason for appending such extra
identities is because an ISP with independent news servers wants to add a
common identity to the Path for statistical purposes.  I've seen that
several times.

> My suggested fix was to replace both those steps by

>    4. The agent MUST then prepend one or more <path-identity>s
>    identifying itself (as set out in section 3.2) to the Path header
>    field content, separated by either "!!" or "!". However, the last
>    (leftmost) such <path-identity> so appended MUST be one that is
>    expected by the destination site when it in turn comes to apply Step
>    3 above.

This language requires mind reading; you may have no idea what other sites
expect.  It's also hard to satisfy this MUST during a transition where
some peers may expect one ordering and others may be expecting a different
one.

I can see the argument for saying something here, but it would be nice to
say something that's a bit more clearly defined and less based on the
(unknown) behavior of one's peers.  Something like adding the below to the
end of the step five description:

    If the agent prepends additional <path-identity>s, the leftmost
    <path-identity> SHOULD be the one advertised to other news servers as
    the expected <path-identity> for this agent.

> Note. There is some confusion as to whether a "!" belongs to the
> identity/diagnostic to its left, or to its right, and the present
> wording switches betweeen these two conventions in the middle of Step 4
> (and my revision does the same switch with its "separated by").

The present wording doesn't consider "!" to belong to any identity and
instead treats it as a delimiter, so it's not following either of those
conventions.  The precise steps seemed to read more clearly to me doing it
that way.

> So some further tidying up of the whole wording might be a good idea.

Please do suggest other wording if you think there's a clearer way to
explain it.

> On top of that, my 'either "!!" or "!"' needs further rewording to
> reflect the outcome of issue #1414.

Yup.

> And, for my final niggle, please can we have a definition of the term
> "expected" which is used at several places in 3.2.1.

I don't think we gain much from this.  This checking is not particularly
formal, so I think more specification implies more rigorous verification
than we're actually specifying.  "Expected" feels to me like it captures
about the level of checking and formality we're getting.  The definition
of the term is the standard English definition of the word.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Mon, 19 Feb 2007 22:22:33 -0800 (PST)
Sender: [email-masked]

    Date: Monday, February 19, 2007 @ 22:22:32
  Author: eagle
Revision: 2817

Resolve #1413.  Make the relayer-name parameter to the ihave and sendme
control messages required, matching Son-of-1036.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2007-02-20 06:16:02 UTC (rev 2816)
+++ docs/usefor/usepro.xml	2007-02-20 06:22:32 UTC (rev 2817)
@@ -1791,8 +1791,7 @@
           <artwork>
       control-command     =/ Ihave-command
       Ihave-command       = "ihave" Ihave-arguments
-      Ihave-arguments     = 1*WSP relayer-name
-                            / 1*( 1*WSP msg-id ) [ 1*WSP relayer-name ]
+      Ihave-arguments     = 1*WSP *( msg-id 1*WSP ) relayer-name
       control-command     =/ Sendme-command
       Sendme-command      = "sendme" Sendme-arguments
       Sendme-arguments    = Ihave-arguments
@@ -1815,8 +1814,9 @@
         may be of interest to the relaying agents receiving the ihave
         message.  The sendme message requests that the agent receiving it
         send the articles having the specified message identifiers to the
-        named relaying agent.  If &lt;relayer-name> is not given, it is
-        determined from the origin of the control message.</t>
+        named relaying agent.  Contrary to <xref target="RFC1036" />, the
+        relayer-name MUST be given as the last argument in the Control
+        header field.</t>
 
         <t>Upon receipt of the sendme message (and a decision to honor
         it), the receiving agent sends the article or articles requested.
@@ -2092,6 +2092,9 @@
           <t>Cancel control messages are no longer required to have From
           and Sender header fields matching those of the target message,
           as this requirement added no real security.</t>
+
+          <t>The relayer-name parameter in the Control header field of
+          ihave and sendme control messages is now required.</t>
         </list>
       </t>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Mon, 19 Feb 2007 22:16:03 -0800 (PST)
Sender: [email-masked]

    Date: Monday, February 19, 2007 @ 22:16:02
  Author: eagle
Revision: 2816

Resolve #1417.  Prohibit injector modification of the Message-ID header
field.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2007-02-19 22:58:20 UTC (rev 2815)
+++ docs/usefor/usepro.xml	2007-02-20 06:16:02 UTC (rev 2816)
@@ -680,9 +680,10 @@
             Content-Transfer-Encoding).  It MAY add other header fields
             not already provided by the poster, but injecting agents are
             encouraged to use the Injection-Info header for such
-            information and to minimize the addition of other headers.
-            It SHOULD NOT alter, delete, or reorder any existing header
-            field except the Path header.</t>
+            information and to minimize the addition of other headers.  It
+            SHOULD NOT alter, delete, or reorder any existing header field
+            except the Path header field.  It MUST NOT alter or delete any
+            existing Message-ID header field.</t>
 
             <t>If the Newsgroups header contains one or more moderated
             groups and the proto-article does not contain an Approved

Date: Mon, 19 Feb 2007 20:09:29 +0100
From: Harald Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
Subject: Re: ISSUE: content of Approved header
Sender: [email-masked]

Charles Lindsey wrote:
> In <BBEC96AE7E2BA72374E6FF74@[192.168.1.108]> Harald Tveit Alvestrand <[email-masked]> writes:
>
>   
>> I want a second for this.
>>     
>
>   
>> If USEFOR already requires an email address, "Approved: foobar" is already 
>> prohibited. I don't see a compelling case for making USEPRO say anything 
>> about which email address to use.
>>     
>
> OK, consider "Approved: foobar.com", which is syntactically allowed but
> still discouraged by USEFOR.
>   
How does 'foobar.com' match the ABNF?

   approved        =  "Approved:" SP mailbox-list CRLF

                  Harald

Date: Mon, 19 Feb 2007 20:05:28 +0100
From: Harald Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Sender: [email-masked]

I see that you did not understand what I proposed, so I'll make the 
essential points more clear:

- Multipoint injection consists of the same article being posted 
multiple times on the same network.
There is no presumption that those injections happen at the same time, 
or are being done by the same agent.

- In my proposal, injection-date: identifies the time of injection. 
Period, end of story. It is, in my opinion, harmful and of no benefit to 
let the posting-agent specify this. The injecting agent knows its own 
current time, and there is no later agent that is more trustworthy to 
leave the decision to.

- What you are calling "reinjection" is what I call "multipoint 
injection, where one of the points listens to a Netnews network to get 
hold of the proto-article".

I have yet to understand any tangible benefit from your proposal to 
allow the posting-agent to specify injection-date.

                  Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: Possibility to use non-resolvable domain name as path-identity
Date:  Mon, 19 Feb 2007 20:03:15 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 51
Sender: [email-masked]

Charles Lindsey wrote:
 
>> How's your example supposed to work, has foo.com no NS record ?
 
> I have never regarded the existence of (only) an NS record as
> meaning "resolvable in the DNS".

Okay, but nslookup foo.com would return "nothing", not NXDOMAIN,
a similar effect as for an "UUCP domain" with only an MX record.

>> An MX record probably won't help with !! (aka 'match').
 
> 'match' is not nececessarily done by IP addresses.

Yes, that's why I wrote "probably", you or Russ mentioned some
NNTP AUTH magic some days ago instead of a gethostbyname IP match.

> if a site has outgoing relayers "news1.site.example",
> "news2.site.example", etc, it still might use "site.example" as
> its <path-identity>, but in that case I would like to see at
> least an MX record for "site.example".

That's already covered by RFC 2142, [email-masked] has to
work, they need either an MX or an address with an smtpd.

>> say that "a path-identity SHOULD be the fully qualified domain
>> name of the corresponding host (here news server)", and be done
>> with it.  If it has no IP it has a good excuse to pick 'something'
>> else, and then 'something' with an MX might be the second best
>> choice, but maybe reality is weirder than we can imagine.
 
> Well, since that's almost exactly what I want it to say, can I
> take it that you 'second' the Issue?

As continuation of the #1093 saga, yes, it makes no sense to track
the in essence same issue twice.

>>> we have agreed that we do not intend either to supersede/obsolete/
>>> commend it.
 
>> TINW.  I certainly don't agree.
 
> That seems to be the consensus, but feel free to raise an Issue.

#1093 already exists.  Any "let the users try to guess a responsible 
address, maybe with whois or by asking abuse@" approach, because it's
beyond admins to filter spam to a role account, that's IMHO arrogance,
unrelated to the C-word reserved for Chairs.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: content of Approved header
Date:  Mon, 19 Feb 2007 19:27:50 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 12
Sender: [email-masked]

Charles Lindsey wrote:
 
> consider "Approved: foobar.com", which is syntactically allowed but
> still discouraged by USEFOR.

Are we reading the same copy of RFC 2822 ?  The article standard says
<mailbox-list>.  RFC 2822 says that this is one or more <mailbox>, 
and a <mailbox> is either <addr-spec> or <name-addr>.  There's no "@"
in foobar.com, it's a syntax error.

Frank

To: [email-masked]
Xref: clerew local.usefor:24436
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: Possibility to use non-resolvable domain name as path-identity
Date: Mon, 19 Feb 2007 16:17:24 GMT
Lines: 27
Sender: [email-masked]

In <02ACD4BB8BBC699DBF0DB047@[192.168.1.108]> Harald Tveit Alvestrand <[email-masked]> writes:

>I want a second before entering this one as an issue number.

>Note: 2142 is a Proposed standard, not a Draft (of any kind).

Sorry. My typo. Should have been "proposed-standard".

>> Also, I wish to see a mention of RFC 2142, just to draw attention to its
>> existence. It does have the status of a draft-standard, although we have
>> agreed that we do not intend either to supersede/obsolete/commend it.
>> There were two wordings in Usepro-06, of which I would suggest
>>
>>    NOTE: According to [RFC 2142], the forms "usenet@<path-identity>" and
>>    "news@<path-identity>" are common addresses for a news server
>>    administrator.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24438
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: content of Approved header
Date: Mon, 19 Feb 2007 17:01:37 GMT
Lines: 25
Sender: [email-masked]

In <BBEC96AE7E2BA72374E6FF74@[192.168.1.108]> Harald Tveit Alvestrand <[email-masked]> writes:

>I want a second for this.

>If USEFOR already requires an email address, "Approved: foobar" is already 
>prohibited. I don't see a compelling case for making USEPRO say anything 
>about which email address to use.

OK, consider "Approved: foobar.com", which is syntactically allowed but
still discouraged by USEFOR.

USEPRO backs up that discouragement in the case of approvals by
Moderators, but not in the case of group control messages. That seems
inconsistent to me.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24433
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: application/news-message-id
Date: Mon, 19 Feb 2007 15:01:35 GMT
Lines: 18
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Hi, what's an application/news-message-id ?

No such beast exists AFAIK.

Or, if it does, IANA should have something to say on it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24437
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: Possibility to use non-resolvable domain name as path-identity
Date: Mon, 19 Feb 2007 16:56:47 GMT
Lines: 95
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:
> 
>> This issue originally arose from a suggestion by Harald that a domain
>> name (e.g. foo.com) could be used as a <path-identity> even though the
>> only records in the DNS were at one level lower (e.g. news.foo.com,
>> mail.foo.com).

>I don't recall "one level" and I don't recall "lower".  It could be
>also the opposite, a subdomain the.hide.out.example with "real" domain
>out.example.

Agreed. I had covered only half of the possible cases.

>How's your example supposed to work, has foo.com no NS record ?

I have never regarded the existence of (only) an NS record as meaning
"resolvable in the DNS".

>> For sure, people are likely to do it whatever we say, but I regard it
>> as an undesirable practice. I therefore wish to make it a SHOULD NOT
>> (or rather to say it SHOULD be resolvable, to MX/A/AAAA/CNAME, in the
>> DNS, which amounts to the same thing).

>Why's that the same thing ?  An MX record probably won't help with !!
>(aka 'match').  

'match' is not nececessarily done by IP addresses. And if a site has
outgoing relayers "news1.site.example", "news2.site.example", etc, it
still might use "site.example" as its <path-identity>, but in that case I
would like to see at least an MX record for "site.example".

>Let the spec. say that "a path-identity SHOULD be the fully qualified
>domain name of the corresponding host (here news server)", and be done
>with it.  If it has no IP it has a good excuse to pick 'something' else,
>and then 'something' with an MX might be the second best choice, but 
>maybe reality is weirder than we can imagine.

Well, since that's almost exactly what I want it to say, can I take it that
you 'second' the Issue?

>> Also, I wish to see a mention of RFC 2142, just to draw attention to
>> its existence.  It does have the status of a draft-standard

s/draft-standard/proposed-standard/ of course.

>No.  I'm checking facts for an implementation and interoperability
>report for its promotion to DS.

I don't think 2142 is in a fit state for DS yet, particularly regarding
what it says about Netnews. The other RFCs you mention seem to recommend
adhering to it with varying degrees of strictness. We cannot be as strict
as some of them, however, because we are not requiring <path-identity>s to
be mailable. But that is no reason for total silence.

>> we have agreed that we do not intend either to supersede/obsolete/
>> commend it.

>TINW.  I certainly don't agree.

That seems to be the consensus, but feel free to raise an Issue.

>>| According to [RFC 2142], the forms "usenet@<path-identity>" and
>>| "news@<path-identity>" are common addresses for a news server
>>| administrator.

>Yes, and according to s-o-1036 there are also two common addresses,
>only different, for a total of three...

Though I don't think 'newsmaster' is widely used in practice - not since
2142 came out, anyway. As to whether 'news' is more prevalent than
'usenet', s-o-1036 seems to think not.

>> I do not want to take any position on the relative merits of 
>> "usenet@" and "news@".

>...making it more interesting for admins and other affected parties,
>tiny little mazes, all alike.  The proposed issue could be joined
>with #1093.

As I said, feel free to raise it. I don't want to go further than I have
already indicated.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24435
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: Format of batched news in response to sendme
Date: Mon, 19 Feb 2007 16:12:32 GMT
Lines: 60
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Harald Tveit Alvestrand <[email-masked]> writes:

>> Second, please.

>Son-of-1036 says only:

>          The  ihave  message  states that the named relayer has filed
>          articles with the specified message IDs,  which  may  be  of
>          interest to the relayer(s) receiving the ihave message.  The
>          sendme message requests that the relayer receiving  it  send
>          the  articles  having the specified message IDs to the named
>          relayer.

OK, I have been poking around inside CNews to see exactly what happens
(the source code of CNews is probably the best indication of how this
protocol used to work - and probably still does in a few places).

The Ihave and Sendme control messages are normal articles, except that
they are propagated according to the dreadful "To." hack in the Newsgroups
header (we _have_ documented that). In response to Sendme, the server is
supposed to send the requested article(s), and it does so by using the
normal News Transport mechanism (the same as for the Ihave and Sendme
messages themselves. But what is the "normal News Transport mechanism"?

As far as Cnews is concerned, it is configured with a list of hosts that
it sends stuff to, together with the "mechanism" to be used for each. This
might be UUCP, NNTP, mail2news or any other hack privately agreed.
Bascially, they will all use the basic format (as now defined by USEFOR),
augmented by whatever the particular mechanisn needs. And, as I now see,
it is only those mechanisms designed for UUCP that prepend those lines
starting with "#!rnews..." of "#!cunbatch...", and possibly combining
articles into batches as well.

So, to take an absurd but instructive example, if site A sent an Ihave
control message to site B via NNTP, and it site B understood the message
and played along by returning a Sendme, then site A would respond by
sending the requested article(s) to site B by NNTP, by generating a
suitable NNTP IHAVE command (or just conceivably a POST command). So it
would all work but no "#!rnews..." line would be involved.

And indeed, I now see that the description of the "#!rnews..." stuff,
which does indeed appear in Son-of-1036, is not specifially related to use
with Ihave/Sendme, but merely describes a general transport mechanism for
all purposes (including sending the Ihave/Sendme messages in the first
place).

So, on that basis, I withdraw my Issue request.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24434
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416 Reinjection - an attempted summary, and a suggested resolution
Date: Mon, 19 Feb 2007 15:45:58 GMT
Lines: 150
Sender: [email-masked]

In <[email-masked]> Harald Alvestrand <[email-masked]> writes:

>There was some report of an experiment on propagation with different 
>Date: headers, but I've not been able to find that in my archive. Bad 
>choice of keywords to search for, I assume.

Essentially, I established that some injecting agents impose an arbitrary
cutoff (e.g. 4 days) as a matter of site policy, and some will accept any
degree os staleness (but whether they then succeed in relaying/propagating
them is another matter).

Also, it seems to be the case that articles up to 8 days stale, once
accepted by an injecting agent, seemed to propagate, and to do so without
much degradation of speed or by using noticeably different routes. But I
have insufficient data to give a definitive answer as to whether these
observations are typical (now, if Google reported arrival times, or
retained the Path, such experiments would be much easier :-( ).

>Suggested hacks include:
>- Changing Date: at injection to be within a short distance from the 
>current time
>- Allowing posting agent to set Injection-Date:, to ensure it's 
>consistent between injected copies

Yes, Russ suggested that, and I agree - modulo various niggles about what
constitutes a 'posting agent' in this particular context.

>- Drop Injection-Date: from the spec

>--------------------------------------------------------------------------------------------------------------------------
>PROPOSED RESOLUTION:

>- Date: identifies the date of composition. Don't change it, ever.

Yes.

>- Injection-Date: identifies the date of injection, by the last agent 
>that identifies itself as an injecting agent. Proto-articles don't have 
>it.

Almost. It might have been put there by a posting agent (see 'suggested
hacks above) at the time of posting (Note: that does not mean that the
proto-article has it).

> Relaying and serving agents never, ever change it.

Yes. And likewise gateways (if the outgoing medium will allow it).

>- "Reinjection" doesn't exist.

You can say that, but it will not prevent it from happening :-( . And
sometimes it is even sensible. And safe if the Injection-Date is retained
(there _may_ be some special cases where not retaining it might be OK, but
those need careful thinking about).

> There is only multipoint injection. The 
>means by which a proto-article (including message-ID, but NOT including 
>Injection-Date:) gets from wherever it started out to the posting agent 
>is not defined by the protocol; if the protocol happens to be NNTP, 
>that's not an issue for this specification. (we can add warnings...)

Eh? Surely the prime duty of a posting agent is to generate
proto-articles. What the protocol does not define is whether a
posting-agent maintains a queue of stuff waiting to be injected (lots of
them do, and those are the ones which might be allowed to insert
Injection-Date). OTOH, "normal" posting agents will not be doing
multipoint injection; that is more likely to arise from specially written
scripts.

>- When doing multipoint injection, Message-ID: MUST be consistent in all 
>copies. Date: SHOULD be consistent, if it's not, Bad Things Can Happen.

And likewise Injection-Date.

>On the stale article problem:

>- We state that existing implementations use Date: to check for 
>staleness, and that will cause discarding of articles that are too old. 
>Period, end of story, accepted.

Yes.

>- We recommend (require?) that injecting agents refuse posting of "too 
>old" articles by Date: - the exact acceptable Date: range should be a 
>local decision.

Disagree. That is a matter of site policy. It is the relayer that should
make that choice, and hopefully it will be using Injection-Date for that
putpose. OK, if the injecting agent knows that its immediately following
relaying agent is going to reject, then it would be couteous to reject it
itself (but equally well, if could look at Injection-Date itself). As I
said, this is all a matter of site policy.

>- We recommend that the staleness check use Injection-Date: if present, 
>otherwise Date:

Yes. REQUIRE.

>- We state that if stale article rejection is an issue for people, they 
>should:
>  - Ensure that their injecting agent adds Injection-Date:

Or add it themselves (e.g the man who carries his server around on his
laptop).

>  - Put pressure on the discarding agent to check Injection-Date: if present

Yes.

However, I think we should wait until Russ has produced the revised
proposal that he promised a couple of days back.

>Case analysis:
>If a message has Date: on day 0, and is injected on day 7, into a 
>network where anything older than 3 days is considered stale, but the 
>injecting agents accept 14-day-old proto-articles, this will happen:

>- Servers that check Injection-Date: will relay the message. More 
>propagation than today.
>- Servers that check Date: will reject the message. Unchanged from today.
>- Servers that feed from a server that checks Date: will not get the 
>message in the normal feed; they may get it offered from feeds that go 
>via Injection-Date: checking servers.

All correct. Injecting that late is currently a bit dodgy. It will improve
as more servers are upgraded to use Injection-Date.

>If a message has Date: on day 0, gets injected on day 1 into the 
>network, and gets injected again on day 7 into the same network, the 
>following will happen:

>- Servers that check Date: will see the first copy.
>- Servers that check Injection-Date: will relay the message twice, 
>possibly causing users to see it twice.

But not if whoever reinjected on day 7 kept the original Injection-Date
(which is what I claim SHOULD/MUST be the normal practice - though a few
exceptions might be worth looking into).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: application/news-message-id
Date:  Mon, 19 Feb 2007 13:58:13 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 7
Sender: [email-masked]

> Arrghh

Indeed.  It's no MIME subtype at all, it's a "lost" access type
for the http://www.iana.org/assignments/access-types registry.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: application/news-message-id
Date:  Mon, 19 Feb 2007 13:30:27 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

Harald Alvestrand wrote:

> http://www.iana.org/assignments/media-types/application/ sems
> to claim it's a Spencer-defined son-of-1036-ism.

Arrghh, I had checked this, but I didn't expect the line break:

    NOTE: In the specific case where it is desired  to
    essentially  make another article PART of the cur-
    rent one, e.g. for annotation of the  other  arti-
    cle, MIME's "message/external-body" convention can
    be used to do so without actual inclusion.  "news-
    message-ID" was registered as a standard external-
    body access method, with a mandatory NAME  parame-
    ter  giving  the  message  ID and an optional SITE
    parameter suggesting an NNTP site that might  have
    the  article  available  (if  it  is not available
    locally), by IANA 22 June 1993.

Sounds like a feature to be replaced by news URLs.  But excl. IETF
announce lists I know nobody using message/external-body anywhere.
No job for USEPRO.

Frank

Date: Mon, 19 Feb 2007 08:27:44 +0100
From: Harald Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>
Cc: [email-masked]
Subject: Re: application/news-message-id
Sender: [email-masked]

Frank Ellermann wrote:
> Hi, what's an application/news-message-id ?
>
> Frank
>
>
>
>   
http://www.fileformat.info/info/mimetype/application/news-message-id/index.htm 
as well as http://www.iana.org/assignments/media-types/application/ sems 
to claim it's a Spencer-defined son-of-1036-ism.

              Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  application/news-message-id
Date:  Mon, 19 Feb 2007 02:45:36 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 4
Sender: [email-masked]

Hi, what's an application/news-message-id ?

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: Possibility to use non-resolvable domain name as path-identity
Date:  Sun, 18 Feb 2007 23:22:15 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 62
Sender: [email-masked]

Charles Lindsey wrote:
 
> This issue originally arose from a suggestion by Harald that a domain
> name (e.g. foo.com) could be used as a <path-identity> even though the
> only records in the DNS were at one level lower (e.g. news.foo.com,
> mail.foo.com).

I don't recall "one level" and I don't recall "lower".  It could be
also the opposite, a subdomain the.hide.out.example with "real" domain
out.example.

How's your example supposed to work, has foo.com no NS record ?  

> For sure, people are likely to do it whatever we say, but I regard it
> as an undesirable practice. I therefore wish to make it a SHOULD NOT
> (or rather to say it SHOULD be resolvable, to MX/A/AAAA/CNAME, in the
> DNS, which amounts to the same thing).

Why's that the same thing ?  An MX record probably won't help with !!
(aka 'match').  

An alias of 'something' doesn't guarantee that 'something' has an
address.   Talking about CNAME in a NetNews memo strikes me as wrong.

Let the spec. say that "a path-identity SHOULD be the fully qualified
domain name of the corresponding host (here news server)", and be done
with it.  If it has no IP it has a good excuse to pick 'something' else,
and then 'something' with an MX might be the second best choice, but 
maybe reality is weirder than we can imagine.

> Also, I wish to see a mention of RFC 2142, just to draw attention to
> its existence.  It does have the status of a draft-standard

No.  I'm checking facts for an implementation and interoperability
report for its promotion to DS.  The main implementation I'm aware
of is the RFCI abuse zone (postmaster is covered by better RFCs).
I've found references in RFC 2369 (mailing lists), RFC 2635 (FYI 35,
don't spew), RFC 3013 (BCP 46, ISP security), and in the approved
SIEVE vacation RFC.

> we have agreed that we do not intend either to supersede/obsolete/
> commend it.

TINW.  I certainly don't agree.

>| According to [RFC 2142], the forms "usenet@<path-identity>" and
>| "news@<path-identity>" are common addresses for a news server
>| administrator.

Yes, and according to s-o-1036 there are also two common addresses,
only different, for a total of three...

> I do not want to take any position on the relative merits of 
> "usenet@" and "news@".

...making it more interesting for admins and other affected parties,
tiny little mazes, all alike.  The proposed issue could be joined
with #1093.

Frank

Date: Sun, 18 Feb 2007 22:44:16 +0100
From: Harald Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>
Cc: [email-masked]
Subject: ADMIN: Repeating procedure on issue-tracking (Re: ISSUE: content of Approved header)
Sender: [email-masked]

Frank Ellermann wrote:
> Harald Tveit Alvestrand wrote:
>
>   
>> I want a second for this.
>>     
>
> That's an odd way to track issues by not tracking them.
>
>  From my POV the issue could be be noted as "rejected",
> the approved article format standard is clear enough.
> Apparently you also think that USEFOR is good enough.
>   
As I said in my message of Jan 3:
>> The chairs will enter an issue in the tracker if one or more of the 
>> following things is true: - the chairs agree it is an issue that 
>> needs discussion - one or more persons apart from the person raising 
>> the issue agrees that it is an issue, and says so on the mailing list 
>> ("+1" can be used, but be clear on what you're responding to - +1 on 
>> a refutation is not the same as +1 on the original issue!) The chairs 
>> will not enter an issue in the tracker if: - they don't think it is 
>> an issue - the editor responds with "yes, that's an error, fixed in 
>> my copy" or equivalent before the issue can be tracked, and nobody 
>> else disasgrees with that statement. 
In this particular case, I think Charles is raising issues that make no 
sense to discuss, so I don't want to waste time putting them into the 
tracker, soliciting comments, gauging WG consensus, resolving them and 
sending out mail saying "this issue is resolved". I'm far enough behind 
in following up real issues in my WGs that tracking every one of 
Charles' (IMHO) useless non-issues is simply more pain in the posterior 
than I'm willing to tolerate.

                     Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: content of Approved header
Date:  Sun, 18 Feb 2007 22:13:39 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 12
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> I want a second for this.

That's an odd way to track issues by not tracking them.

 From my POV the issue could be be noted as "rejected",
the approved article format standard is clear enough.
Apparently you also think that USEFOR is good enough.

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: Format of batched news in response to sendme
Organization: The Eyrie
Date: Wed, 14 Feb 2007 22:57:29 -0800
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> Second, please.

> The -06 specification of the batch format was incomplete; -07 says that
> it is a private matter.

The specification of ihave/sendme has always left unspecified how the
messages themselves are conveyed.  RFC 1036 says only:

    This message is part of the ihave/sendme protocol, which allows one
    host (say A) to tell another host (B) that a particular message has
    been received on A.  Suppose that host A receives message
    "<[email-masked]>", and wishes to transmit the message to
    host B.

    A sends the control message "ihave <[email-masked]> A" to
    host B (by posting it to newsgroup to.B).  B responds with the
    control message "sendme <[email-masked]> B" (on newsgroup
    to.A), if it has not already received the message.  Upon receiving
    the sendme message, A sends the message to B.

Son-of-1036 says only:

          The  ihave  message  states that the named relayer has filed
          articles with the specified message IDs,  which  may  be  of
          interest to the relayer(s) receiving the ihave message.  The
          sendme message requests that the relayer receiving  it  send
          the  articles  having the specified message IDs to the named
          relayer.

Both make further passing mentions to UUCP, but neither specify how the
messages be sent.  As far as the ihave/sendme specification is concerned,
the messages could be conveyed by FTP or by HTTP PUT.  More to the point,
they could be sent as separate messages without encoding rather than using
a batch format if two peers so chose, or as mbox files, or as a tar.gz
file, or anything else that the other side knew how to process.

I believe their use has been exclusively or nearly exclusively in
conjunction with UUCP batching because UUCP batching is what news servers
have historically implemented, but neither previous formal specification
has explicitly linked those two things.  I think that's for valid reasons;
there really isn't any justification that I can see to constrain the
method of transmission of the articles, and setting up such a feed
requires out-of-protocol arrangements between peers anyway.

That leaves documentation of the batch format as only useful as an aid for
implementors who want to write a new implementation of UUCP batching and
transmission, and I have no idea why one would want to do such a thing at
this point beyond just for the fun of it.  If someone really feels
inspired to document it, perhaps just for historical curiosity, I'd think
that they'd want to include the gunbatch, cunbatch, and c7unbatch
compression and encoding formats as well.  The whole topic would make for
a simple and self-contained informational RFC that those who were curious
could read.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Thu, 15 Feb 2007 07:34:20 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: ISSUE: Format of batched news in response to sendme
Sender: [email-masked]

Second, please.

The -06 specification of the batch format was incomplete; -07 says that it 
is a private matter.

Is ihave/sendme obsolete or not?

--On 14. februar 2007 14:35 +0000 Charles Lindsey <[email-masked]> 
wrote:

>
> In response to a sendme control message, an article is sent, normally
> using the news transport mechanism (typically UUCP in such cases). I.e.,
> it is on the same "wire" as the sendme control message. Therefore it is in
> order to document the (somewhat bizarre) format, as was done in
> Son-of-1036 and in Usepro-06. It is documented nowhere else.
>
> Note that, as compared to Usepro-06, some brief mention of variants of the
> format for compressed batches would be in order.
>
> --
> Charles H. Lindsey ---------At Home, doing my own
> thing------------------------ Tel: +44 161 436 6131 Fax: +44 161 436 6133
> Web: http://www.cs.man.ac.uk/~chl Email: [email-masked]      Snail:
> 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K. PGP: 2C15F1A9      Fingerprint:
> 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
>
>

Date: Thu, 15 Feb 2007 07:25:03 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: ISSUE: content of Approved header
Sender: [email-masked]

I want a second for this.

If USEFOR already requires an email address, "Approved: foobar" is already 
prohibited. I don't see a compelling case for making USEPRO say anything 
about which email address to use.

--On 14. februar 2007 15:15 +0000 Charles Lindsey <[email-masked]> 
wrote:

>
> Issue # requested.
>
> This was discussed earlier, but never resolved.
>
> It is required that a moderator uses an address identifying himself in the
> Approved header. For control messages, nothing is said (so you could
> legitimately write
>
>    Approved: foobar
>
> which is not particularly helpful. USEFOR says
>
>    The Approved header field indicates the mailing addresses (and
>    possibly the full names) of the persons or entities approving the
>    article for posting. ...
>
> And the USEPRO wording for moderators ensures that requirement is met. I
> want to see the same thing said for group control messages, since senders
> of such message clearly have a "duty" to fulfil that USEFOR requirement.
>
> That is not to say the Approved will be identical to the From (one might
> be a personal name and the other a role address). But it should certainly
> be something that sites considering whether to honour the control message
> should be able to look at and expect to find something familiar.
>
> --
> Charles H. Lindsey ---------At Home, doing my own
> thing------------------------ Tel: +44 161 436 6131 Fax: +44 161 436 6133
> Web: http://www.cs.man.ac.uk/~chl Email: [email-masked]      Snail:
> 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K. PGP: 2C15F1A9      Fingerprint:
> 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
>
>

Date: Thu, 15 Feb 2007 07:22:25 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: ISSUE: Possibility to use non-resolvable domain name as path-identity
Sender: [email-masked]

I want a second before entering this one as an issue number.

Note: 2142 is a Proposed standard, not a Draft (of any kind). Mention of 
that standard belongs in issue #1093; it is not appropriate to raise it as 
a new issue.

--On 14. februar 2007 14:56 +0000 Charles Lindsey <[email-masked]> 
wrote:

>
> Issue # requested
>
> There were two alternative texts in Usepro-06, of which Russ arbitrarily
> chose one.
>
> This issue originally arose from a suggestion by Harald that a domain name
> (e.g. foo.com) could be used as a <path-identity> even though the only
> records in the DNS were at one level lower (e.g. news.foo.com,
> mail.foo.com).
>
> For sure, people are likely to do it whatever we say, but I regard it as
> an undesirable practice. I therefore wish to make it a SHOULD NOT (or
> rather to say it SHOULD be resolvable, to MX/A/AAAA/CNAME, in the DNS,
> which amounts to the same thing).
>
> Also, I wish to see a mention of RFC 2142, just to draw attention to its
> existence. It does have the status of a draft-standard, although we have
> agreed that we do not intend either to supersede/obsolete/commend it.
> There were two wordings in Usepro-06, of which I would suggest
>
>    NOTE: According to [RFC 2142], the forms "usenet@<path-identity>" and
>    "news@<path-identity>" are common addresses for a news server
>    administrator.
>
> which uses a little bit of each of each. I do not want to take any
> position on the relative merits of "usenet@" and "news@".
>
> --
> Charles H. Lindsey ---------At Home, doing my own
> thing------------------------ Tel: +44 161 436 6131 Fax: +44 161 436 6133
> Web: http://www.cs.man.ac.uk/~chl Email: [email-masked]      Snail:
> 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K. PGP: 2C15F1A9      Fingerprint:
> 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
>
>

To: [email-masked]
Xref: clerew local.usefor:24409
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Ticket status, USEPRO
Date: Wed, 14 Feb 2007 15:41:53 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> "Charles Lindsey" <[email-masked]> writes:

>In <[email-masked]> Harald Alvestrand <[email-masked]> writes:

>>So we have 2 "text accepted", 3 "No discussion", and 1 "controversial". 
>>Who knows - we might finish this soon?

>OK, so that leaves room for some more ISSUES not so urgent. I have just
>reviewed my original list of protocol problems. and there are still some I
>want to raise. But not tonight.

Now it is morning (or afternoon to be more accurate) so I have raised a
further 5 issues (or lesser importance than my first batch, but still
requiring decisions). I reckon 5 is as much at the WG (and Russ in
particular) can handle at once.

>And I shall also review the three 'do discuusion' cases.

Which I have now done.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24408
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: ISSUE: consistency of Injection-Info
Date: Wed, 14 Feb 2007 15:37:52 GMT
Lines: 29
Sender: [email-masked]

Issue # requested

Injecting agents have considerable freedom in choosing which of the
available Injection-Info parameters to use. However, if they select at
random from the available possibilities, or in the values they insert in
the chosen parameter (and especially in 'posting-account', it is going to
make life difficult for the rest of the net, particularly for those
wishing to use Injection-Info in killfiles in order to catch people who
keep morphing their From headers. So they should choose a consistent
format, and stick to it.

Usepro-06 had the following wording in the Duties of an injecting agent:

   Each injecting agent SHOULD use a consistent form of the Injection-Info
   header field for all articles originating from the same or similar
   origins.

That, or something similar, needs to go after Step 11 of section 3.4

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24407
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: ISSUE: content of Approved header
Date: Wed, 14 Feb 2007 15:15:27 GMT
Lines: 35
Sender: [email-masked]

Issue # requested.

This was discussed earlier, but never resolved.

It is required that a moderator uses an address identifying himself in the
Approved header. For control messages, nothing is said (so you could
legitimately write

   Approved: foobar

which is not particularly helpful. USEFOR says

   The Approved header field indicates the mailing addresses (and
   possibly the full names) of the persons or entities approving the
   article for posting. ...

And the USEPRO wording for moderators ensures that requirement is met. I
want to see the same thing said for group control messages, since senders
of such message clearly have a "duty" to fulfil that USEFOR requirement.

That is not to say the Approved will be identical to the From (one might
be a personal name and the other a role address). But it should certainly
be something that sites considering whether to honour the control message
should be able to look at and expect to find something familiar.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24406
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: ISSUE: Possibility to use non-resolvable domain name as path-identity
Date: Wed, 14 Feb 2007 14:56:00 GMT
Lines: 37
Sender: [email-masked]

Issue # requested

There were two alternative texts in Usepro-06, of which Russ arbitrarily
chose one.

This issue originally arose from a suggestion by Harald that a domain name
(e.g. foo.com) could be used as a <path-identity> even though the only
records in the DNS were at one level lower (e.g. news.foo.com,
mail.foo.com).

For sure, people are likely to do it whatever we say, but I regard it as
an undesirable practice. I therefore wish to make it a SHOULD NOT (or
rather to say it SHOULD be resolvable, to MX/A/AAAA/CNAME, in the DNS,
which amounts to the same thing).

Also, I wish to see a mention of RFC 2142, just to draw attention to its
existence. It does have the status of a draft-standard, although we have
agreed that we do not intend either to supersede/obsolete/commend it.
There were two wordings in Usepro-06, of which I would suggest

   NOTE: According to [RFC 2142], the forms "usenet@<path-identity>" and
   "news@<path-identity>" are common addresses for a news server
   administrator.

which uses a little bit of each of each. I do not want to take any position
on the relative merits of "usenet@" and "news@".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24404
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: ISSUE: Moderators SHOULD retain existing message-id
Date: Wed, 14 Feb 2007 14:26:57 GMT
Lines: 41
Sender: [email-masked]

Issue # requested.

We discussed this earlier, and I had assumed Frank was going to raise it
(maybe he assumes I was). It still needs resolving.

The issue is whether this needs normative wording, or whether it is merely
a USEAGE matter.

The argument for normative wording is that in various situations confusion
can be caused if the Message-ID in the published article is not that
chosen by the poster. Various such situations have been mentioned, but
here are the two main examples:

1. The same message might be both posted and mailed (maybe to some mailing
list). If it is essentially the same messqage in either medium, then it
ought to have the same Message-ID [1].

2. Some posters keep a record of the Message-IDs of the articles they
post, so that their User Agents can flag any replies to them.

There are doubltess other weird and unpredictable circumstances which
could casue similar problems. OTOH, situations where it is _necessary_ for
the moderator to change a Message-ID are much fewer (e.g. he has
substantially altered the article before posting it, or he is aware it has
already been multi-posted to other non-moderated groups). A "SHOULD" still
gives him leeway to change it in such cases.

[1] This is separate from the related issue where that mailing list is
itself gated into Usenet; this situation is already well covered in our
Gayewaying section.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24402
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1415 USEPRO 3.2.1 - Number of path entries per site
Date: Wed, 14 Feb 2007 12:42:40 GMT
Lines: 65
Sender: [email-masked]

As Harald has pointed out, this has received no discussion. I won't repeat
my full argument here (see the issue tracker), but will just summarize.

There are two issues, plus some niggles (including #1414 which refers to
the same bit of text).

1. To permit sites to omit path entries which merely reflect their
internal structure without being of any benefit to the world outside. My
suggested rewording of the first paragraph of 3.2.1 was

   Except possibly when relaying to other hosts on the same site, every
   injecting, relaying, or serving agent that accepts an article MUST
   update the Path header field ....

2. The present Steps 4 and 5 of 3.2.1 are just plain wrong.

Since receiving agents are expected to do a MISMATCH check on articles
relayed to them (inserting "!!" or "!.MISMATCH...." accordingly), and only
look at the leftmost <path-identity> when doing that check, it is
essential that the 'true' identity of the site (the ones its downstreams
will be 'expecting') be the last one to be prepended, after prepending any
extra ones (e.g. a <path-nodot> such as 'demon'). However, Steps 4 and 5
currently prepend these things in the wrong order, which could cause
MISMATCH checks to fail.

Note that such "extra" identities are only put there because that site does
not want to receive articles with any of those identities already present
from its upstreams, and for historical reasons it needed more than one
identity for that purpose.

My suggested fix was to replace both those steps by

   4. The agent MUST then prepend one or more <path-identity>s identifying
   itself (as set out in section 3.2) to the Path header field content,
   separated by either "!!" or "!". However, the last (leftmost) such
   <path-identity> so appended MUST be one that is expected by the
   destination site when it in turn comes to apply Step 3 above.

Note. There is some confusion as to whether a "!" belongs to the
identity/diagnostic to its left, or to its right, and the present wording
switches betweeen these two conventions in the middle of Step 4 (and my
revision does the same switch with its "separated by"). So some further
tidying up of the whole wording might be a good idea.

On top of that, my 'either "!!" or "!"' needs further rewording to reflect
the outcome of issue #1414.

And, for my final niggle, please can we have a definition of the term
"expected" which is used at several places in 3.2.1. Essentially, it needs
to reflect whatever the receiving site is using as the "verified" identity
of the sending site (whether derived from the IP address, or from some
earlier SASL authentication, or whatever else). Usepro-06 covered this by
requiring sites to establish such a verified identity, and by then using
the term "verified" rather than "expected".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24403
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1414 USEPRO 3.2.1: delimiter for multiple Path identities
Date: Wed, 14 Feb 2007 12:52:29 GMT
Lines: 29
Sender: [email-masked]

As Harald has said, this has received no discussion.

It concerns whether several <path-identity>s added by a single site should
be separated by "!" or !!".

"!!" would be more natural, since for sure a site will recognize articles
received from other machines at the same site. Note that it may not be
obvious, from the outside, that such machines were in fact part of the
same site, and the absence of "!!" will more and more become grounds for
suspicion as this convention becomes established.

I do not care whether this is specified as a natural outcome of the rest
of the wording, or as a specific MUST/SHOULD, but it needs to be clear
somehow.

However, issue #1415 also deals with that same part of the wording, so
maybe #1415 should be fixed first, or maybe the two issues should be
combined.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24405
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: ISSUE: Format of batched news in response to sendme
Date: Wed, 14 Feb 2007 14:35:54 GMT
Lines: 19
Sender: [email-masked]

In response to a sendme control message, an article is sent, normally
using the news transport mechanism (typically UUCP in such cases). I.e.,
it is on the same "wire" as the sendme control message. Therefore it is in
order to document the (somewhat bizarre) format, as was done in
Son-of-1036 and in Usepro-06. It is documented nowhere else.

Note that, as compared to Usepro-06, some brief mention of variants of the
format for compressed batches would be in order.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24401
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1412 Cancel newsgroups: matching
Date: Wed, 14 Feb 2007 11:47:52 GMT
Lines: 38
Sender: [email-masked]

>From Frank ELlermann:

>Russ Allbery wrote:
>
>> > To best ensure that it will be relayed to the same news servers
>> > as the original message, a cancel control message SHOULD have the
>> > same Newsgroups header field as the message it is cancelling.
>
>Proposed addition:
>
>| The groups in the Newsgroups header field MUST match at least one
>| of the affected newsgroups.

-1

I think the SHOULD is sufficient (that is what was in Usepro-06), and
is now the same as for the other control messages.

>For Rmgroup and Newgroup we have that anyway, for Checkgroups it's also
>sound, and for a Cancel it's required for transparency. 
>
>All other control messages apparently don't have a Newsgroups header 
>field. If that's the case it has to be mentioned explicitly, USEFOR-11
>claims that this header field is "mandatory".

Not so. All Group Control messages (5.2) now have that SHOULD.
Ihave/Sendme have their own weird provisions.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24400
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Ticket status, USEPRO
Date: Wed, 14 Feb 2007 10:39:33 GMT
Lines: 40
Sender: [email-masked]

In <[email-masked]> Harald Alvestrand <[email-masked]> writes:

>The following tickets are open:

>#1083: USEPRO 5.3: Rules for generating message-ID
>  No change required
>#1093: USEPRO 2.3: Supported email addresses
>  No change required.
>#1412: USEPRO 5.3: Cancel newsgroups: Matching
>  No discussion.
>#1413: USEPRO 5.5: ihave/sendme syntax
>  Text accepted
>#1414: USEPRO 3.2.1: delimiter for multiple Path identities
>  No discussion
>#1415: USEPRO 3.2.1: Number of path entries per site
>  No discussion
>#1416: USEPRO 3.9: Reinjection and Injection-Date
>  Controversial. I'll make a separate note on this.
>#1417: USEPRO 3.4: Injecting-agent modification of message-ID
>  Text accepted

>So we have 2 "text accepted", 3 "No discussion", and 1 "controversial". 
>Who knows - we might finish this soon?

OK, so that leaves room for some more ISSUES not so urgent. I have just
reviewed my original list of protocol problems. and there are still some I
want to raise. But not tonight.

And I shall also review the three 'do discuusion' cases.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 14 Feb 2007 09:02:36 +0100
From: Harald Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1416 Reinjection - an attempted summary, and a suggested resolution
Sender: [email-masked]

So far, my buffer shows:

42 postings on the "#1416" thread, Jan 19 to Feb 7, including Charles' 
initial issue mail
Active participants:
- Charles Lindsey
- Russ Allbery
- Forrest J. Cavalier
- Frank Ellermann
- Harald Tveit Alvestrand
- Richard Clayton

Relevant technical information from the thread:

- Injecting a message (with an unique Message-ID:) several times into a 
single Usenet network is supported, and it's important that it remains 
supported ("multipoint injection").
- Current USENET servers (when acting as relaying agents) check Date: 
for staleness (rejecting articles that have a Date: that is so old that 
their history files won't cover the message-ID).
- Injection of articles into USENET where the client posting agent sends 
to someone acting as an injecting agent, and the article then ending up 
with someone acting as a posting agent towards another injecting agent 
("reinjection from an isolated network"), is quite common.
- No technical means exists today for verifyng whether a network is 
truly isolated.
- No existing mail-to-news gateways modify Date:

There was some report of an experiment on propagation with different 
Date: headers, but I've not been able to find that in my archive. Bad 
choice of keywords to search for, I assume.

Suggested hacks include:
- Changing Date: at injection to be within a short distance from the 
current time
- Allowing posting agent to set Injection-Date:, to ensure it's 
consistent between injected copies
- Drop Injection-Date: from the spec

I've probably dropped some important pieces of information, but I've 
seen the above.

--------------------------------------------------------------------------------------------------------------------------
PROPOSED RESOLUTION:

This is very much on the "Keep It Simple" approach to protocol design.

- Date: identifies the date of composition. Don't change it, ever.
- Injection-Date: identifies the date of injection, by the last agent 
that identifies itself as an injecting agent. Proto-articles don't have 
it. Relaying and serving agents never, ever change it.
- "Reinjection" doesn't exist. There is only multipoint injection. The 
means by which a proto-article (including message-ID, but NOT including 
Injection-Date:) gets from wherever it started out to the posting agent 
is not defined by the protocol; if the protocol happens to be NNTP, 
that's not an issue for this specification. (we can add warnings...)
- When doing multipoint injection, Message-ID: MUST be consistent in all 
copies. Date: SHOULD be consistent, if it's not, Bad Things Can Happen.

On the stale article problem:

- We state that existing implementations use Date: to check for 
staleness, and that will cause discarding of articles that are too old. 
Period, end of story, accepted.
- We recommend (require?) that injecting agents refuse posting of "too 
old" articles by Date: - the exact acceptable Date: range should be a 
local decision.
- We recommend that the staleness check use Injection-Date: if present, 
otherwise Date:
- We state that if stale article rejection is an issue for people, they 
should:
  - Ensure that their injecting agent adds Injection-Date:
  - Put pressure on the discarding agent to check Injection-Date: if present

Case analysis:
If a message has Date: on day 0, and is injected on day 7, into a 
network where anything older than 3 days is considered stale, but the 
injecting agents accept 14-day-old proto-articles, this will happen:

- Servers that check Injection-Date: will relay the message. More 
propagation than today.
- Servers that check Date: will reject the message. Unchanged from today.
- Servers that feed from a server that checks Date: will not get the 
message in the normal feed; they may get it offered from feeds that go 
via Injection-Date: checking servers.

If a message has Date: on day 0, gets injected on day 1 into the 
network, and gets injected again on day 7 into the same network, the 
following will happen:

- Servers that check Date: will see the first copy.
- Servers that check Injection-Date: will relay the message twice, 
possibly causing users to see it twice.

The last looks like a loop, but can be solved by sysadmins beating up on 
the injecting agent to enforce stricter policing of posting of old articles.
Is the latter a Big Deal?

                            Harald

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Ticket status, USEPRO
Organization: The Eyrie
Date: Tue, 13 Feb 2007 21:36:20 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Well Russ wrote a long message in which he suggested, amongst other
> things, allowing the posting agent to set the Injection-Date, with the
> understanding that injecting agents MUST then accept what they are
> given.

> I wrote a long reply, with various comments, but basically accepting
> that as a way forward, but with various details of the approach still
> needing to be fixed.

> It would help if Russ were to reply to that message.

I know, and I will.  As I warned, I don't have a tremendous amount of time
to participate in the working group (which I know was disguised by
managing to find time for several weeks after I said that, but that really
was a fluke).

I owe you a reply and a more specific proposal incorporating your
feedback.  Although please don't let that stop anyone else from weighing
in on that thread; we need more general discussion.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24398
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Ticket status, USEPRO
Date: Tue, 13 Feb 2007 23:00:29 GMT
Lines: 52
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>Harald Alvestrand wrote:
>> The following tickets are open:
>> 
>> #1416: USEPRO 3.9: Reinjection and Injection-Date
>>   Controversial. I'll make a separate note on this.

>I think there is some agreement between those participating.

Well Russ wrote a long message in which he suggested, amongst other
things, allowing the posting agent to set the Injection-Date, with the
understanding that injecting agents MUST then accept what they are given.

I wrote a long reply, with various comments, but basically accepting that
as a way forward, but with various details of the approach still needing
to be fixed.

It would help if Russ were to reply to that message.

>Charles keeps pushing for a larger variety of uses to
>be covered.

I don't think the variety of uses is the issue. It is the underlying
principles, and the various uses merely provide test cases against which
the principles can be assessed (basically, people are going to do all
sorts of things which it is hard to foresee). Having said that, there are
some specific circumstances which we need to discuss and understand, and
that might lead to some specific wording/exceptions/whatever.

>We've also had someone indicate that the 3-4 day typical
>stale cutoff is adequate in practice.

It often is, but sometimes is not. And typical currect practice seems to
be to allow longer.

>  I can live with that too:
>remove the use of Injection-Date header.

But USEFOR is now approved as a P-S. 

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 12 Feb 2007 09:57:34 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: Harald Alvestrand <[email-masked]>
CC: [email-masked]
Subject: Re: Ticket status, USEPRO
Sender: [email-masked]

Harald Alvestrand wrote:

> So we have 2 "text accepted", 3 "No discussion", and 1 "controversial". 
> Who knows - we might finish this soon?

After we settle on the reinjection text, I think there will
need to be some alteration to the Gateways section.

I mentioned this when writing a message on Jan 19. I did not raise an
issue (pending the re-injection text.)

http://www.imc.org/ietf-usefor/mail-archive/msg03883.html

I don't have reason to think the corresponding changes would be controversial.

Date: Mon, 12 Feb 2007 09:48:12 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: Harald Alvestrand <[email-masked]>
CC: [email-masked]
Subject: Re: Ticket status, USEPRO
Sender: [email-masked]

Harald Alvestrand wrote:
> The following tickets are open:
> 
> #1416: USEPRO 3.9: Reinjection and Injection-Date
>   Controversial. I'll make a separate note on this.

I think there is some agreement between those participating.

Charles keeps pushing for a larger variety of uses to
be covered.

Is it possible to settle on a small subset of uses and
define and cover those in the section on re-injection?
For example, the "disjoint network" wording is a problem
because it is not well defined, EXCEPT in the case where
you know the post originated locally, you are not a true
relaying peer, and you are re-posting it to one injection
server.  It is not difficult to write language to cover
this use.  Problems with the text are raised when
we try to cover wider uses, not this constrained case.

We've also had someone indicate that the 3-4 day typical
stale cutoff is adequate in practice.  I can live with that too:
remove the use of Injection-Date header.

Date: Mon, 12 Feb 2007 07:02:04 +0100
From: Harald Alvestrand <[email-masked]>
To: [email-masked]
Subject: Ticket status, USEPRO
Sender: [email-masked]

The following tickets are open:

#1083: USEPRO 5.3: Rules for generating message-ID
  No change required
#1093: USEPRO 2.3: Supported email addresses
  No change required.
#1412: USEPRO 5.3: Cancel newsgroups: Matching
  No discussion.
#1413: USEPRO 5.5: ihave/sendme syntax
  Text accepted
#1414: USEPRO 3.2.1: delimiter for multiple Path identities
  No discussion
#1415: USEPRO 3.2.1: Number of path entries per site
  No discussion
#1416: USEPRO 3.9: Reinjection and Injection-Date
  Controversial. I'll make a separate note on this.
#1417: USEPRO 3.4: Injecting-agent modification of message-ID
  Text accepted

So we have 2 "text accepted", 3 "No discussion", and 1 "controversial". 
Who knows - we might finish this soon?

When following up, please place the ticket number in the subject line, 
and follow up ONE ticket per message, unless the message proposes that 
two tickets should be merged.

                     Harald

Date: Mon, 12 Feb 2007 06:58:34 +0100
From: Harald Alvestrand <[email-masked]>
To: Russ Allbery <[email-masked]>
Cc: [email-masked]
Subject: Re: #1413: USEPRO 5.5: ihave/sendme syntax
Sender: [email-masked]

Given no more comments on this thread, I'm closing this one as 'Text 
accepted'.

Russ Allbery wrote:
> I think we have an agreed resolution for this one.  Namely, change the
> syntax of Ihave-arguments to:
>
>        Ihave-arguments     = 1*WSP *( msg-id 1*WSP ) relayer-name
>
> Replace this language from the second paragraph after:
>
>    If <relayer-name> is not given, it is determined from the origin of
>    the control message.
>
> with:
>
>    Contrary to [RFC1036], the relayer-name MUST be given as the last
>    argument in the Control header field.
>
> I think Charles and I are both happy with this.  It brings our syntax in
> line with Son-of-1036.  Are there any objections to this resolution, or
> anything that I'm missing?
>
>   

Date: Mon, 12 Feb 2007 06:54:50 +0100
From: Harald Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: #1093 USEPRO 3.2: Supported email addresses
Sender: [email-masked]

No new technical information was supplied in the discussion. Some 
information on "which addresses get the more spam" was posted.

I'm closing this as "No change required".

Date: Mon, 12 Feb 2007 06:47:35 +0100
From: Harald Alvestrand <[email-masked]>
To: Russ Allbery <[email-masked]>
Cc: [email-masked]
Subject: No change required: #1083: USEPRO 5.3: Rules for generating message-ID
Sender: [email-masked]

I have not seen any compelling technical arguments that validate saying 
anything in USEPRO about generating Message-ID; the text in RFC 2822 
appears to be sufficient.

I have verified that the current usepro-07 draft says nothing at all 
about how to generate message-ID, just when it has to be present.

I'm marking this one "No change required".

Harald

Russ Allbery wrote:
> I propose closing this ticket with no further changes.
>
> There seems to be a rough consensus in the working group that the $alz
> convention of issuing spam cancels with message IDs formed by adding
> "cancel." to the beginning of the LHS of the original message ID to allow
> for intentional collisions between multiple cancels for the same message
> is the sort of weird stunt that can be left as an intentional protocol
> violation.
>
> The rest of the ticket is talking about namespace issues in deciding what
> message IDs to use for messages.  USEFOR now defers to RFC 2822 for this,
> and I think the treatment in RFC 2822 is sufficient.  This is a tricky
> area, so if we can avoid having to say more about it and use the consensus
> of the RFC 2822 working group, that saves us a lot of work and tricky
> wording discussion.
>
>   

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Protocol Action: 'Netnews Article Format' to Proposed Standard
Organization: The Eyrie
Date: Wed, 07 Feb 2007 11:31:48 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> And no new projects. The URIs are really a matter for the [email-masked]
> list (or maybe for the now dormant NNTP list, if Russ agrees).

I'm happy to have them be discussed there.  As far as I'm concerned, the
ietf-nntp list is available for discussion of any NNTP-related protocol
issues that people feel it would be appropriate for.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Wed, 07 Feb 2007 15:56:08 +0100
From: Frank Ellermann <[email-masked]>
Organization: <URL:http://purl.net/xyzzy>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Sender: [email-masked]

Richard Clayton wrote:

> Don't forget as well, how batteries may fail and hence so many articles
> were apparently written in the small hours of 1st January 1970 !

Some folks intentionally set their Date to "today 00:00 local time" (or
similar) for privacy reasons.  IIRC that was also an option in at least
one UA.  The whole issue is somewhat irrelevant if servers follow the
recommendation in s-o-1036 (9.2) to use a history of at least one week.

Frank

Date: Wed, 07 Feb 2007 17:58:58 +0000
From: Alexey Melnikov <[email-masked]>
To: Charles Lindsey <[email-masked]>, Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: Protocol Action: 'Netnews Article Format' to Proposed Standard
Sender: [email-masked]

Charles Lindsey wrote:

>In <[email-masked]> Frank Ellermann <[email-masked]> writes:
>  
>
>>The IESG wrote:
>>    
>>
>>>The IESG has approved the following document:
>>>      
>>>
>>>- 'Netnews Article Format '
>>>   <draft-ietf-usefor-usefor-12.txt> as a Proposed Standard
>>>      
>>>
>>[...]
>>    
>>
>>>http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-12.txt
>>>      
>>>
>>Great.  How about updating the WG Charter now ?  I'd like to add the
>>I-D about news and nntp URIs, if that could help to settle the two
>>(I think) remaining objections by Charles:
>>    
>>
>
>Please, no charter updates! We have enough on our plate.
>  
>
(chair hat on)
There would be no new documents in the charter until both USEPRO and 
USEAGE are done.

To: [email-masked]
Xref: clerew local.usefor:24358
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Protocol Action: 'Netnews Article Format' to Proposed Standard
Date: Wed, 7 Feb 2007 12:28:15 GMT
Lines: 32
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>The IESG wrote:

>> The IESG has approved the following document:
> 
>> - 'Netnews Article Format '
>>    <draft-ietf-usefor-usefor-12.txt> as a Proposed Standard
>[...]
>> http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-12.txt

>Great.  How about updating the WG Charter now ?  I'd like to add the
>I-D about news and nntp URIs, if that could help to settle the two
>(I think) remaining objections by Charles:

Please, no charter updates! We have enough on our plate.

And no new projects. The URIs are really a matter for the [email-masked] list
(or maybe for the now dormant NNTP list, if Russ agrees). And no, I have
other objections to your draft, but as matters of style rather than
protocol.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24357
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date: Wed, 7 Feb 2007 12:21:22 GMT
Lines: 39
Sender: [email-masked]

In <8BA5E677ED1CB11AD2A7BF7A@[10.71.2.170]> Harald Tveit Alvestrand <[email-masked]> writes:

>I was asking for the behaviour of mail-to-news gateways.

>Since it's been a fairly common occurrence to have "disconnected" mail from 
>laptops being sent a few days after they were composed (certainly far more 
>common than similar news postings), and nobody has identified any 
>mail-to-news gateway that alters the Date: header, we can probably conclude 
>that the issue of getting articles bounced because the Date: header is too 
>old causes so little pain that nobody's bothered to do anything about it.

The evidence, such as it was, that I gathered in my experiment seemed to
indicate that Dates which were stale by up to 6-8 days seemed to propagate
pretty normally (but I was only able to monitor them at a few sites,
because auto-responders to the *.test groups seem to be very few and far
between these days). Stuff older than that did seem to get lost, however.

More of a problem was that some injecting agents (notably NIN) seem to
reject at around 4 days stale (I had to inject elsewhere for my
experiment).

>That is food for thought.

Well we agreed in USEFOR to go ahead with Injection-Date, so we should
continue with that. But it would be useful to have some response to my
reply to Russ's proposal, since I was basically in agreement with his
final scheme (modulo some minor quibbles, and the need for further
discussion of "Magic" exceptions).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 7 Feb 2007 09:18:09 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Sender: [email-masked]

In message <8BA5E677ED1CB11AD2A7BF7A@[10.71.2.170]>, Harald Tveit
Alvestrand <[email-masked]> writes

>Since it's been a fairly common occurrence to have "disconnected" mail from 
>laptops being sent a few days after they were composed (certainly far more 
>common than similar news postings), and nobody has identified any 
>mail-to-news gateway that alters the Date: header, we can probably conclude 
>that the issue of getting articles bounced because the Date: header is too 
>old causes so little pain that nobody's bothered to do anything about it.

I rather suspect that when anyone complains the sysadmins roll their
eyes and ask what it is that is not understood about the word "news" ?

We have one "regular" poster in demon.ip.support.turnpike (the self-help
group for users of the Turnpike email/news client) who is often two to
three months behind in their reading and commenting [and who apparently
works offline and then posts everything in large batches]...  there is
considerable animosity about this behaviour (everyone else has moved on
to other topics, and the large number of posts suddenly appearing from
one person is an annoyance in itself...).

>That is food for thought.

I am quite sanguine about weeks old articles failing to be propagate
well, and so sensible newsservers give immediate feedback and fail to
accept them.  It is in my view a Good Thing that their original date of
writing is recorded and they are rejected on that basis. Hopefully,
before resubmission, the author will reconsider whether their article is
still necessary, or can still be posted without revision.

Don't forget as well, how batteries may fail and hence so many articles
were apparently written in the small hours of 1st January 1970 !

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

Date: Tue, 06 Feb 2007 22:58:19 -0800
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Sender: [email-masked]

--On 6. februar 2007 11:29 +0000 Charles Lindsey <[email-masked]> 
wrote:

>>> Well that's what I had always assumed gateways (I think we are thinking
>>> mainly of mailing list expanders here) would do (read: I have never
>>> spotted one changing anything, though I have not looked that hard).
>
>> Mailing list expanders are, in general, not mail-to-news gateways.
>
> Indeed, but I thought the purpose of your original question was to
> ascertain what current practice was with regard to altering the Date
> header in various mail systems, to see whether it offered any useful
> precedents for how we should proceed.

I was asking for the behaviour of mail-to-news gateways.

Since it's been a fairly common occurrence to have "disconnected" mail from 
laptops being sent a few days after they were composed (certainly far more 
common than similar news postings), and nobody has identified any 
mail-to-news gateway that alters the Date: header, we can probably conclude 
that the issue of getting articles bounced because the Date: header is too 
old causes so little pain that nobody's bothered to do anything about it.

That is food for thought.

               Harald

Date: Tue, 06 Feb 2007 22:55:05 -0800
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Protocol Action: 'Netnews Article Format' to Proposed  Standard  (fwd)
Sender: [email-masked]

Congratulations to all on finally hitting this milestone!

           Harald

------------ Forwarded Message ------------
Date: 5. februar 2007 16:22 -0500
From: The IESG <[email-masked]>
To: IETF-Announce <[email-masked]>
Cc: Internet Architecture Board <[email-masked]>, RFC Editor 
<[email-masked]>, usefor mailing list <[email-masked]>, 
usefor chair <[email-masked]>
Subject: Protocol Action: 'Netnews Article Format' to Proposed 
Standard

The IESG has approved the following document:

- 'Netnews Article Format '
   <draft-ietf-usefor-usefor-12.txt> as a Proposed Standard

This document is the product of the Usenet Article Standard Update
Working Group.

The IESG contact persons are Lisa Dusseault and Ted Hardie.

A URL of this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-12.txt

Summary

This document specifies the syntax of Netnews articles in the context
of the "Internet Message Format" (RFC 2822) and "Multipurpose Internet
Mail Extensions (MIME)" (RFC 2045). This document obsoletes
the mesasge format parts of RFC 1036, providing an updated specification
to reflect current practice and incorporating incremental changes
specified in other documents. The document also adds several new header
fields that replace various non-standard and non-interoperable header
fields in use in Usenet today.

This is the first in a set of documents that obsolete RFC 1036. This
document focuses on the syntax and semantics of Netnews articles.

Another document describes protocol issues of Netnews articles
independent of transport protocols, e.g. control articles.

And yet another document describes policy related issues,
interoperability and usability related recommendations.

Process and goals history of this draft.

The USEFOR WG started its efforts to update RFC 1036 about 9 years ago.
Several email related standards got published and updated in this
timeframe.
Several WG chairs have changed since then, and many WG participants left
or joined the WG (mostly left).

In May 2004, this document was a part of the 98-page
draft-ietf-usefor-article-13.txt. Following the proposal of Pete
Resnick, WG chair at the time, the document got split into 2 documents:
draft-ietf-usefor-usefor-00.txt and draft-ietf-usefor-usepro-00.txt.

Following feedback from the WG members, Alexey Melnikov asked Ken
Murchison to edit draft-ietf-usefor-usefor-xx.txt instead of Charles
Lindsey, taking a more concise draft (draft-kohn-news-article-03.txt) by
Dan Kohn as the base.
Ken Murchison became the primary editor, while Charles Lindsey and Dan
Kohn got
listed as co-editors of the draft-ietf-usefor-usefor-00.txt document.

While the split and new primary editor helped to move things forward,
this was not sufficient to get the document finished.

In April 2005, Harald Alvestand joined as co-chair of the WG. He
introduced issue tracker to the WG, which helped to prevent people from
reraising old issues again and again. He also on one occasion suspended
posting rights of one of the abusive WG members, which helped reduce the
personal attacks in the group.

draft-ietf-usefor-usefor-10.txt is the result of a WG Last Call in April
2006 (which raised several issues, none of them requiring major changes)
and a call for consensus on the resolution of Last Call comments in
September 2006.

This draft is being submitted for Proposed Standard.

The USEFOR WG has reviewed the draft, last-call (and post last-call)
reviews included:
- Frank Ellermann
- Russ Allbery
- Richard Clayton
- Ralph Babel
- Forrest J. Cavalier III
- Seth Breidbart
- Dan Schlitt

Of the reviewers, two argued that the group should be disbanded and the
draft ditched.

Protocol Quality

 Lisa Dusseault reviewed this document for the IESG.

---------- End Forwarded Message ----------

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Protocol Action: 'Netnews Article Format' to Proposed Standard
Date:  Tue, 06 Feb 2007 18:45:36 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 44
Sender: [email-masked]

The IESG wrote:

> The IESG has approved the following document:
 
> - 'Netnews Article Format '
>    <draft-ietf-usefor-usefor-12.txt> as a Proposed Standard
[...]
> http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-12.txt

Great.  How about updating the WG Charter now ?  I'd like to add the
I-D about news and nntp URIs, if that could help to settle the two
(I think) remaining objections by Charles:

http://permalink.gmane.org/gmane.org.w3c.uri/970

If I got it right he'd prefer to specify a new "range" syntax for
nntp-URLs, while I'd prefer to stick to the RFC 1738 syntax, because
one "message" of RFC 1738, draft-gilman, and draft-hoffman (continued
in the actual I-D) was always that news-URLs are better than nntp-URLs
for various reasons.  Which isn't the point where I'd wish to add new
features not supported by existings UAs (as far as they support this
scheme at all), even if it should be easy to implement it in new UAs.

His second objection is about the "new" (3977) <wildmat> construct,
he wants news-URLs to allow the complete wildmat syntax with "!" and
",", while I want only "*" and the percent-encoded "?" as a "minimal"
superset of Gilman's old news-URL draft.  Actually the same concerns
as for nntp-ranges, Charles wants to support all new NNTP features if
they're low-hanging fruits, I want maximum backwards compatibility
with Gilman's I-D and common practice.

To some degree it's a matter of taste, and the feedback I got so far
is somewhat limited, most folks are happy with deprecating snews, two
contributors submitted nits (fixed) otherwise supporting the I-D as is,
and one contributor wanted the "*" as it now is (in an earlier draft I
tried to get away with the original 1738 idea of either group or "*").

Any opinions here ?  The old 2004 WG Charter mumbles something about:

| The Group shall also aid and/or oversee the production of other
| Usenet related Internet Drafts and Standards.

Frank

To: [email-masked]
Xref: clerew local.usefor:24348
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date: Tue, 6 Feb 2007 11:29:07 GMT
Lines: 30
Sender: [email-masked]

In <5E2FA472884ABE251F9B6A63@[10.71.2.170]> Harald Tveit Alvestrand <[email-masked]> writes:

>--On 31. januar 2007 17:59 +0000 Charles Lindsey <[email-masked]> 
>wrote:

>>> I think it's hard to get a read on what all gateways do, since there are
>>> so many different implementations and they're not really discussed in a
>>> common place.  All of mine keep the Date header, though.
>>
>> Well that's what I had always assumed gateways (I think we are thinking
>> mainly of mailing list expanders here) would do (read: I have never
>> spotted one changing anything, though I have not looked that hard).

>Mailing list expanders are, in general, not mail-to-news gateways.

Indeed, but I thought the purpose of your original question was to
ascertain what current practice was with regard to altering the Date
header in various mail systems, to see whether it offered any useful
precedents for how we should proceed.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Content-class: urn:content-classes:message
Subject: RE: Protocol Action: 'Netnews Article Format' to Proposed Standard 
Date: Tue, 6 Feb 2007 07:07:00 -0500
From: "Hollenbeck, Scott" <[email-masked]>
To: "The IESG" <[email-masked]>
Cc: "usefor mailing list" <[email-masked]>, "Internet Architecture Board" <[email-masked]>, "usefor chair" <[email-masked]>
Sender: [email-masked]

Congratulations on finally getting this document finished!

-Scott- 
(former USEFOR area advisor)

> -----Original Message-----
> From: The IESG [mailto:[email-masked]] 
> Sent: Monday, February 05, 2007 4:22 PM
> To: IETF-Announce
> Cc: usefor mailing list; Internet Architecture Board; usefor 
> chair; RFC Editor
> Subject: Protocol Action: 'Netnews Article Format' to 
> Proposed Standard 
> 
> The IESG has approved the following document:
> 
> - 'Netnews Article Format '
>    <draft-ietf-usefor-usefor-12.txt> as a Proposed Standard
> 
> This document is the product of the Usenet Article Standard Update 
> Working Group. 

From: The IESG <[email-masked]>
To: IETF-Announce <[email-masked]>
Cc: Internet Architecture Board <[email-masked]>, RFC Editor <[email-masked]>, usefor mailing list <[email-masked]>, usefor chair <[email-masked]>
Subject: Protocol Action: 'Netnews Article Format' to Proposed  Standard 
Date: Mon, 05 Feb 2007 16:22:08 -0500
Sender: [email-masked]

The IESG has approved the following document:

- 'Netnews Article Format '
   <draft-ietf-usefor-usefor-12.txt> as a Proposed Standard

This document is the product of the Usenet Article Standard Update 
Working Group. 

The IESG contact persons are Lisa Dusseault and Ted Hardie.

A URL of this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-12.txt

Summary

This document specifies the syntax of Netnews articles in the context
of the "Internet Message Format" (RFC 2822) and "Multipurpose Internet
Mail Extensions (MIME)" (RFC 2045). This document obsoletes
the mesasge format parts of RFC 1036, providing an updated specification
to reflect current practice and incorporating incremental changes
specified in other documents. The document also adds several new header
fields that replace various non-standard and non-interoperable header
fields in use in Usenet today.

This is the first in a set of documents that obsolete RFC 1036. This
document focuses on the syntax and semantics of Netnews articles.

Another document describes protocol issues of Netnews articles
independent of transport protocols, e.g. control articles.

And yet another document describes policy related issues,
interoperability and usability related recommendations.

Process and goals history of this draft.

The USEFOR WG started its efforts to update RFC 1036 about 9 years ago.
Several email related standards got published and updated in this
timeframe.
Several WG chairs have changed since then, and many WG participants left
or joined the WG (mostly left).

In May 2004, this document was a part of the 98-page
draft-ietf-usefor-article-13.txt. Following the proposal of Pete
Resnick, WG chair at the time, the document got split into 2 documents:
draft-ietf-usefor-usefor-00.txt and draft-ietf-usefor-usepro-00.txt.

Following feedback from the WG members, Alexey Melnikov asked Ken
Murchison to edit draft-ietf-usefor-usefor-xx.txt instead of Charles
Lindsey, taking a more concise draft (draft-kohn-news-article-03.txt) by
Dan Kohn as the base.
Ken Murchison became the primary editor, while Charles Lindsey and Dan
Kohn got
listed as co-editors of the draft-ietf-usefor-usefor-00.txt document.

While the split and new primary editor helped to move things forward,
this was not sufficient to get the document finished.

In April 2005, Harald Alvestand joined as co-chair of the WG. He
introduced issue tracker to the WG, which helped to prevent people from 
reraising old issues again and again. He also on one occasion suspended 
posting rights of one of the abusive WG members, which helped reduce the 
personal attacks in the group.

draft-ietf-usefor-usefor-10.txt is the result of a WG Last Call in April
2006 (which raised several issues, none of them requiring major changes)
and a call for consensus on the resolution of Last Call comments in
September 2006.

This draft is being submitted for Proposed Standard.

The USEFOR WG has reviewed the draft, last-call (and post last-call)
reviews included:
- Frank Ellermann
- Russ Allbery
- Richard Clayton
- Ralph Babel
- Forrest J. Cavalier III
- Seth Breidbart
- Dan Schlitt

Of the reviewers, two argued that the group should be disbanded and the
draft ditched. 
 
Protocol Quality
 
 Lisa Dusseault reviewed this document for the IESG.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date:  Fri, 02 Feb 2007 19:24:47 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 15
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
 
> what's current practice for email gateways on incoming messages? Do they:
 
> - keep the Date: header and accept rejection for old articles?
> - modify the Date: header if the article is rejected on submission?
> - modify the Date: header in all cases?

GateBau '94 (and apparently also '97) don't mention to update a Date:
http://www.elektron-bbs.de/files/dfue/dokument/gatebau.94

But of course GateBau '94 isn't exactly any "current practice"... :-)

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date:  Fri, 02 Feb 2007 18:53:06 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

Charles Lindsey wrote:
 
> I believe this lists uses Mailman.

AFAIK Paul uses majordomo, see also
http://gmane.org/info.php?group=gmane.ietf.usenet.format

It needs a manual intervention of the list owner to disable
mails from the list.  But what mailing lists do with a Date
isn't related to Harald's question about mail2news gateways.

>     Date: Wed, 31 Jan 2007 17:59:52 GMT
 
> Will people please report if it arrives exactly like that?

Yes, my UA displays 18:59:52 local time, that's correct.
Your "raw" article as seen and archived on GMaNe is here:
http://article.gmane.org/gmane.ietf.usenet.format/31586/raw

But GMaNe is very far from any typical mail2news gateways,
it's a unique concept, combining nntp2list + list2nntp and
similarly http2nntp + nntp2http. 

Frank

Date: Fri, 02 Feb 2007 07:22:07 -0800
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Sender: [email-masked]

--On 31. januar 2007 17:59 +0000 Charles Lindsey <[email-masked]> 
wrote:

>> I think it's hard to get a read on what all gateways do, since there are
>> so many different implementations and they're not really discussed in a
>> common place.  All of mine keep the Date header, though.
>
> Well that's what I had always assumed gateways (I think we are thinking
> mainly of mailing list expanders here) would do (read: I have never
> spotted one changing anything, though I have not looked that hard).

Mailing list expanders are, in general, not mail-to-news gateways.

                 Harald

To: [email-masked]
Xref: clerew local.usefor:24325
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date: Thu, 1 Feb 2007 21:34:43 GMT
Lines: 35
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:

>>> I'm not sure what Mailman does.

>> Your message to which I am replying arrived with

>>     Date: Tue, 30 Jan 2007 14:27:35 -0800

>> Is that identical to what you submitted? I believe this lists uses
>> Mailman.

>Uh, I don't know what Mailman's mail to news gateway does, I mean.  Which
>I don't think this list checks.

Sure, but Harald's original question related to gatewaying involving mail
only AIUI, just to see what precedents mail has already established.

>I do know that it always replaces the message ID and generates a new one
>for the newsgroup, though.  (And then doesn't change References.)

Arrrrrgggggh!. They need to read your gatewaying texts in Usepro :-) .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5