Subject: draft-ietf-usefor-usepro-02

From: John Stanley <[email-masked]>
Date: Mon, 6 Dec 2004 11:01:17 -0800 (PST)

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>See http://www.ietf-usefor.imc.org/drafts/draft-ietf-usefor-usepro-02.txt 
>and http://www.ietf-usefor.imc.org/drafts/draft-ietf-usefor-usepro-02.unpaged

"www.ietf-usefor.imc.org could not be found. Please check the name and try 
again."

>    A "poster" is the person or software that composes a possibly
>    compliant article for submission to a posting agent. The poster is
>    synonymous with [RFC 2822]'s author.

Then we ought to call it "author" and use the term "poster" for the person
who posts the messages. I thought we had a discussion about this already
and determined that using "poster" in that sense made no difference to 
anything in the draft AND removed the confusion of having a deliberately
different term than the overriding RFC.

>   1. It MUST remove any Injection-Info- or Complaints-To-header already
>      present (though it might be useful to copy them to suitable X-
>      headers). It SHOULD likewise remove any NNTP-Posting-Host, X-Trace
>      or other undocumented tracing header.

If it is "undocumented", how does one know it is a tracing header?

>   2. It SHOULD verify that the article is from a trusted source.
>      However, it MAY allow articles in which headers contain unverified
>      email addresses, that is, addresses which are not known to be
>      valid for the trusted source, and notably so if they end in
>      ".invalid".

Since the injecting agent cannot know, in the general case, that an 
address is not valid for the poster, then it SHOULD NOT pretend that it
can, and we SHOULD NOT tell authors of injecting agents they should
pretend they can. 

I missed the section in RFC2119 which explains how "notably so" modifies
any of the RFC2119 terms. We should either explain what "notably so" 
means, or say "Injection agents MUST accept addresses that end in 
.invalid." (For those who say otherwise, then please explain exactly what
purpose .invalid service, then?)

>7.2.2.  Procedure to be followed by Injecting Agents

>   An injecting agent receives proto-articles from posting and followup
>   agents. 

Ok. Then we see in later sections talk about "reinjection". But we have
been told:

>        NOTE: An article that is offered for reinjection has, by
>        definition, already been injected once, and is not therefore to
>        be considered as a proto-article.  

So, either an injection agent doesn't deal just with "proto-articles", or
there needs to be a "reinjection agent".

>   1. It MUST forward it to the moderator of the first (leftmost)
>      moderated group listed in the Newsgroups-header via email (see 7.8
>      for how that moderator may forward it to further moderators).
>      There are two possibilities for doing this:

Here's this artificial limitation again. Why must it be via EMAIL? Is
there some reason those two words have to be there? I understand that 
whatever IS done will have to "interoperate", but if that "interoperation"
means that MY server hosting MY private newsgroups hands the moderators
articles to approve via a web interface, how is that any skin off of YOUR 
nose?

>   A serving agent MUST maintain a list of the newsgroups it stores in
>   its news database showing the moderation status of each one (see
>   6.2.1), and SHOULD include in that list all groups likely to be
>   crossposted to from those groups (e.g. all other groups in the same
>   hierarchy(ies)).

Knowing as I do the interesting interpretation of SHOULD that most folks 
here have, I have to ask whether this sentence is saying nothing less than 
that a serving agent must list all groups that it has ever heard of. I 
mean, a "strong recommendation that can be avoided only after careful 
study and consideration of all possible negative outcomes" combined with 
the guessing game of "group likely to be crossposted to" results in "all 
groups ever heard mentioned at least once".

>   2. It MUST examine the Injection-Date-header (or, if that is absent,
>      the Date-header) and reject the article as stale (a-5.7) if that
>      predates the earliest articles of which it normally keeps record,
>      or if it is more than 24 hours into the future (the margin MAY be
>      less than that 24 hours).

Isn't expiration policy based on arrival date, not injection date? In any 
case, is there a reason for RFC2119 language regarding a site's expiration
policy? E.g., if a site wants to serve articles for "three days" even if 
they arrive two days after being posted, is that our business?

>   Serving agents MUST NOT create new newsgroups simply because an
>   unrecognized newsgroup-name occurs in a Newsgroups-header (see a-
>   7.2.1 for the correct method of newsgroup creation).

However, we've just told it that it SHOULD include that group in the list 
of groups that it keeps. After all, if it appears in a Newsgroups header, 
it is quite likely that it will be crossposted to.

>   Contrary to [RFC 2822], which states that the mailbox(es) in the From
>   header is that of the poster(s), a poster who does not, for whatever
>   reason, wish to use his own mailbox MAY use any mailbox ending in the
>   top level domain ".invalid" [RFC 2606].

This paragraph is meaningless without an accompanying MUST for the 
injection agent. 

>                                                        Wherever in
>   the following it is stated that, "by default", a header is to be
>   taken from some header in the precursor, it means that its initial
>   content is to be copied from the content of that precursor header.

Then why don't we just say "copied" in the relevant sections and avoid
this redefinition and nonsense of "taken from". *NOTHING IS TAKEN FROM
ANYTHING. It is copied.

And once again, when we specify ONE ACTION for an agent, that is THE 
DEFAULT. We don't need to keep saying "by default", we already know
that the one action we specify IS THE DEFAULT.

>   However, posters MAY then override that default before posting if
>   they so wish.

Of course they can enter their own material. 

>   1. The Newsgroups-header (a-5.5) SHOULD by default be taken from the
>      precursor's Followup-To-header if present, and otherwise from the
>      precursor's Newsgroups-header. 

No. 

1. The content of the Newsgroups-header SHOULD be copied from the content 
of the precursor's Followup-To-header, if present. Otherwise, it is copied
from the content of the precursor's Newsgroups-header.

>   2. The Subject-header SHOULD by default be taken from that of the
>      precursor. The case sensitive string "Re: " MAY be prepended to
>      its Subject-Content unless it already begins with that string.

s/by default be taken from that of/be copied from/ 

Same for 3. 

4. /taken from/copied/

>      Followup agents MAY trim References-headers which have grown to
>      excessive length, but the first and last message identifiers from
>      the precursor MUST NOT be removed.

Is it then acceptable to remove the last message id in the 
References-content of the followup? The last message id in the followup's
References header is not the last message id in the precursor's References
header.

>   A reading agent downloads articles from a serving agent, as directed
>   by the reader, and displays them (or processes them in some other
>   manner), subject to any limitations of the reading agent, such as
>   availability of charsets, and having first decoded any Content-
>   Transfer-Encodings, encoded-words, etc.  It SHOULD also have the
>   capability to show the raw article exactly as received.

A reading agent is free to show articles to the user in any manner the 
user so desires. This is not limited to character sets. If we need a 
definition for "reading agent", then:

"A reading agent accepts articles from a serving agent and displays them 
to the reader."

>   It MAY present lists of articles available for display, and MAY
>   structure those lists so as to show the relationships between the
>   articles, as determined by the References-, Subject-, Date- and
>   other-headers (see [USEAGE] for some usual methods of doing this).

Of course it may. It may also display icons or green flashing dots. 

>7.8.  Duties of a Moderator

>   A Moderator receives news articles by email, decides whether to
>   accept them 

s/by email//
s/accept/approve/

He has already accepted the message -- it is in his mailbox or other input 
queue.

>7.9.1.  Duties of an Outgoing Gateway

>   From the perspective of Netnews, an outgoing gateway is just a
>   special type of reading agent.

Irrelevant and meaningless. Elide.

>                               The operation of the outgoing gateway
>   is only subject to additional constraints in the presence of one or
>   more corresponding incoming gateways back from that medium to
>   Netnews, since this opens the possibility of loops.

The operation of a gateway ought always to be based on the possibility
of the back gateway, not on the actual existance of same. That implies
contstraints based not on the presence, but on the ability. Elide 
sentence.

>   1. The message identifier of the news article should be preserved if
>      at all possible, preferably as or within the corresponding unique
>      identifier of the other medium, but if not at least as a comment
>      in the message. This helps greatly with preventing loops.

I fail to see how a comment in the body of a message helps to prevent 
loops.

>7.9.2.  Duties of an Incoming Gateway

>   The incoming gateway has the serious responsibility of ensuring that
>   all of the requirements of this standard are met by the articles that
>   it forms.

ALL? Or do you mean that it must meet the standards expected by the 
injecting agent that it will be using?

>              In addition to its special duties as a gateway, it bears
>   all of the duties and responsibilities of an injecting agent as well,
>   and additionally has the same responsibility of a relaying agent to
>   reject articles that it has already gatewayed.

I guess you did mean "all". I disagree. An incoming gateway does NOT need 
to be an injecting and relaying agent too. It can function quite well 
below the injection agent. 

>   An incoming gateway MUST NOT gate the same message twice. 

What is the reason for this RFC2119 mandate? The injecting agent will stop 
a duplicate.

>   Incoming gateways MUST NOT pass control messages (articles containing
>   a Control- or Supersedes-header) without removing or renaming that
>   header. 

Thus preventing a user on the other side of an incoming gateway from ever 
cancelling or superceding his own articles. 

>                          If a gateway receives a message that it can
>   determine is a valid equivalent of a cancel message in the medium it
>   is gatewaying, it SHOULD discard that message without gatewaying it,
>   generate a corresponding cancel message of its own, and inject that
>   cancel message.

Cool. My gateway's method of determining a "valid equivalent" is by seeing
a Control header. But I must remove it, even though I am permitted to 
put it in. Which is it?

>        different portions).  In these cases, where there is no one
>        "official" gateway, some other method of generating message
>        identifiers has to be used to avoid collisions. It would
>        obviously be preferable for there to be only one gateway which
>        crossposts, but this may not be possible to coordinate.

Given the previous discussion about possiblilities and not actual 
presence, then what we are actually saying is that gateways MUST create
message ids for news messages and MUST NOT use existing ids found in any
incoming message.

---

From: Bruce Lilly <[email-masked]>
Date: Mon, 6 Dec 2004 14:57:23 -0500

> different term than the overriding RFC.

We indeed had a discussion, but it was pointed out that the
poster and the author may very well be different (e.g. a FAQ
written by one or more authors and posted by an individual
or automatic process).  The "poster" corresponds to
"sender", not to "author".

---

From: "Sebastian Brocks" <[email-masked]>
Date: Tue, 7 Dec 2004 08:38:30 +0100 (MET)

> "sender", not to "author".

Except for the "Followup-To" Header, certainly in that case, the "author"
excepts any email replies, not the "sender".

---

From: Bruce Lilly <[email-masked]>
Date: Tue, 7 Dec 2004 07:39:22 -0500

> excepts any email replies, not the "sender".

No, "certainly" if a Reply-To field is present, a response is
directed to the addresses specified there, which might
include neither authors nor the poster, the poster but
no authors, some authors but not the poster, poster
plus authors and possibly others, ...

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 7 Dec 2004 17:03:47 GMT

>"www.ietf-usefor.imc.org could not be found. Please check the name and try 

Sorry:
See http://www.imc.org/ietf-usefor/drafts/draft-ietf-usefor-usepro-02.txt 
and http://www.imc.org/ietf-usefor/drafts/draft-ietf-usefor-usepro-02.unpaged

>Then we ought to call it "author" and use the term "poster" ...

See my separate response to Bruce Lilly.

>If it is "undocumented", how does one know it is a tracing header?

With difficulty. But people have always managed to discover that
NNTP-Posting-Host was a tracing header.

>can, and we SHOULD NOT tell authors of injecting agents they should

We don't. It says they "MAY" allow certain articles. If they can't tell
the address is bad, then they pretty well have to let them through
(otherwise they would never inject any articles at all). But if they can
detect the bad ones (they might even check with the DNS that the domain
does not exist) then they have the right to refuse them (according to
local site policy).

>purpose .invalid service, then?)

"Notably so" is only modifying a "MAY". Essentially, it is encouraging you
to take advantage of that "MAY" in this circumstance. But there is no
MUST about it, since a site may very well say "we require all our users to
use valid and working email addresses".

>there needs to be a "reinjection agent".

The first sentence of 7.2 indicates the normal situation in which an
injecting agent is given a proto-article. A couple of paragraphs further
on it states that, "in exceptional circumstances" an injection agent may
also be given a full article (not a proto-article). So both circumstances
are covered, and I see little point in complicating the wording by being
more explicit.

Do you really want me to change the first sentence to read
   "An Injecting Agent is responsible for taking a (proto-)article from a
   posting (or other) agent ..."   ?

>Here's this artificial limitation again. Why must it be via EMAIL?

Because, for the worldwide Usenet, that is the way the protocol has been
defined, and there is no other machinery that is guaranteed to work.

> Is

You are talking about a "cooperating subnet" which has chosen (and
presumably published to its membership) some alternative method. The term
"cooperating subnet" is still defined in USEPRO, though it seems to have
disappeared from USEFOR (and I will have to watch that it remains
somewhere when we finally sort out where all the various definitions are
going to go).

>groups ever heard mentioned at least once".

Not, it doesn't result in that. A group that is seen to have been
crossposted to "once" is not thereby a group that is "likely to be
crossposted to" again. The use of the word "likely" indicates that the
admin of the serving agent has to exercise some judgement here, and if he
concludes "after careful study and consideration" that he can ignore the
possibility of crossposts between the chinese newsgroups (which he
carries) and the groups in the national hierarchy for Mozambique (of which
he has probably never heard), then I think he has fulfilled all his
obligations under that "SHOULD".

But I am sure that there nevertheless exists some chinaman somewhere in
the world who has a cousin in Mozambique.

>they arrive two days after being posted, is that our business?

The wording is designed to cover whatever expiration policy the particular
relaying agent observes (which is indeed customarily x days after
arrival). So if the Injection-Date is earlier than the expiry date
(however computed) that it implements (probably of a per group basis), then
it MUST drop it.

>it is quite likely that it will be crossposted to.

No, we have not told it that. It has decided (after careful study and
consideration) not to list groups in the Mozambiquean hierarchy. If an
article cross-posted to that hierarchy now appears, it MUST NOT create the
group just for that reason (it MAY choose to list it henceforth, but the
list is not restricted to groups that it actually keeps).

>This paragraph is meaningless without an accompanying MUST for the 

Yes, you may have a point there. Would you suggest that I weaken the
wording here, or strengthen the wording under injecting agent ("SHOULD"
would probably suffice)? And other opinions would be welcome. AIUI, we
don't want to prevent people doing that kind of munge, but we don't want to
force injecting agents to accept it either.

>this redefinition and nonsense of "taken from". *NOTHING IS TAKEN FROM

Now that I have removed the text regarding MIME-style parameters that used
to be in there, I considered using "copied" instead of "taken". But since
it is the header content rather than the whole header that gets copied, I
concluded on balance that defining the special term "taken" would shorten
the wording, on balance.

>that the one action we specify IS THE DEFAULT.

There are TWO actions for this agent. Either it uses the user's override
(if any) or else it uses the "default".

>from the content of the precursor's Newsgroups-header.

That is precisely the longer wording I have avoided by defining the term
"taken". And "by default" is necessary because sometimes the outgoing
Newsgroups-header is not the Followup-To (etc) one.

>References header is not the last message id in the precursor's References

OK. How about:

      If the References header of the precursor has grown to excessive
      length, it MAY first be trimmed, but its first and last message
      identifiers MUST NOT be removed.

>user so desires. This is not limited to character sets.

It does not say it is limited to character sets. It says "such as
availability of charsets". 

>Of course it may. It may also display icons or green flashing dots.

Very pretty, especially at Christmas time. But not a reason for mentioning
them :-) .

>s/by email//

No, see above.

>He has already accepted the message -- it is in his mailbox or other input 

OK, and several other occurrences of "accept" in that section.

>Irrelevant and meaningless. Elide.

The next sentence would be meaningless without it. Generally, the whole of
the section on gatewaying is in a more discursive style than the rest of
the document because it is dealing with a variety of situations which much
current software handles badly, and is setting out basic principles to be
observed rather than precise protocols.

>contstraints based not on the presence, but on the ability. Elide 

How about:

   .... The operation of the outgoing gateway
   is subject to additional constraints due to the possibility of one or
   more corresponding incoming gateways back from that medium to
   Netnews, since this opens the possibility of loops.

>I fail to see how a comment in the body of a message helps to prevent 

There might be no other place to put it in the other medium. But the
gateway that put it there should be able to find and recognize it if it
ever comes back to the same gateway, and that is how a loop would be
prevented.

>injecting agent that it will be using?

It is responsible for ensuring that proper checks are done. If it can
trust a local injecting agent to do the checks properly, then that is
fine.

>below the injection agent. 

How it organizes itself internally is none of our business, just so long
as its duties and responsibilities are taken care of.

>What is the reason for this RFC2119 mandate? The injecting agent will stop 

Not necessarily. Even if it uses the services of an injecting agent, it
cannot expect that agent to be aware of all the sneaky ways in which an
already gatewayed article may come back to it from the other medium.

>cancelling or superceding his own articles. 

Yes, this may be a bit severe. But it is what Russ originally wrote and
nobody has queried it before now. I am open to further views (and I have
certainly had occasion to put control messages through mail2news before
now).

>a Control header. But I must remove it, even though I am permitted to 

I would suggest checking for and noting the presence of the cancel message
_before_ removing it.

>message ids for news messages and MUST NOT use existing ids found in any

No, I think what it (Russ) is saying is that people who maintain gateways
need to be aware of the way things actually happen within the particular
group (or set of related groups) concerned. Because there just is no "one
size fits all" solution to the gatewaying problem. Every case is
different.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 7 Dec 2004 17:06:46 GMT

>"sender", not to "author".

Yes, that is more or less what we decided. And as a consequence you will
now find the words "author or sender" in 7.6.

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 08 Dec 2004 05:24:36 +0100

Charles Lindsey wrote:

> if they can detect the bad ones (they might even check with
> the DNS that the domain does not exist) then they have the
> right to refuse them (according to local site policy).

That's what news.t-online.de does (or at least did), and IIRC
they even checked the complete address if it was one of their
own user@t-online

> there is no MUST about it, since a site may very well say
> "we require all our users to use valid and working email
>  addresses".

Yes, that was the idea, and as a consequence TLD .invalid did
not work on this server.  And what you've written in the draft
addresses it perfectly.

 [Message-ID as "comment"] 
> There might be no other place to put it in the other medium.

Time to close or burn this gateway and whatever is behind it.

> that is how a loop would be prevented.

Do you have an example ?  Is this a news-irc-news scenario ?

>>> where there is no one "official" gateway, some other method
>>> of generating message identifiers has to be used to avoid
>>> collisions.

Gatebau94 tried to define that for Fido gateways among others.

> people who maintain gateways need to be aware of the way
> things actually happen within the particular group (or set
> of related groups) concerned.

In other words "don't use GiGo", now that is very old news ;-)
Add some more MUST-NOTs to this chapter, the gateway operator
MUST be terrified.

> there just is no "one size fits all" solution to the
> gatewaying problem. Every case is different.

"Don't make up Message-IDs without your lawyer" MUST be clear.

                           Bye, Frank