Subject: Issues outstanding

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 15 Dec 2004 19:14:01 GMT

We are coming to the point where there is little more that can be done on
the documents we are supposed to be producing without deciding how various
outstanding issues are to be resolved.

1. Complaints-To

I published a list of 4 options (and invited other options) in
http://www.imc.org/ietf-usefor/mail-archive/msg00151.html. Only three
people have expressed any preference amongst them. I think #4 is dead, and
#2 is the one most people could live with (but on such a small sample,
that is hardly meaningful).

#2 was to do it in Injection-Info rather than in a Complaints-To header,
and to provide only a mail-complaints-to parameter (which would leave open
the option to provide a separate url-complaints parameter as a future
extension).

2. Path header delimiters

See http://www.imc.org/ietf-usefor/mail-archive/msg00123.html.

I offered 4 schemes A, B, C and D with different ways of indicating:
    #1 the injecting site
    #2 identity of previous site verified
    #3 identity of previous site not verified
    #4 identity of previous site not checked

A. (draft-13) Uses '%' for #1, '/' for #2, '?' for #3 amd '!' for #4

B. (Henry)    Uses '@' for #1, ',' for #2, ' ' for #3 amd '!' for #4

C. (Diablo)   !foo.example.POSTED! for #1
              !bar.example.MISMATCH! for #3
              !baz.example! for #2 and #4 which are not distinguished

D. (CHL)      !foo.example!POSTED! for #1
              !baz.example!M! (or !baz.example!!) for #2
              !bar.example!MISMATCH! for #3
              !baz.example! for #4

But that's just a summary. Please follow the URL for all the pros and
cons.

3. Mail-Copies-To and Posted-And Mailed

Available options appear to be:

1.  Include them as in draft-13
2a. Defer them to a future document (standards-track)
2b. Defer them to a future document (experimental)
3.  Drop them entirely

Earlier discussions were inconclusive. I gather our Chair prefers #2 (a or
b), but he has made no definitive pronouncement.

4.  Terminology for followups.

1. A followup is a response, and MUST have a References header. A part of
   a multi-part FAQ (or anything similar) is not a followup, but it MAY
   nevertheless have a References header.

2. A followup is a response, or a part of a multi-part FAQ (or anything
   similar). A followup MUST have a References header, and anything else
   MUST NOT have one.

It has been established that there is no technical difference between
these formulations. It is just a matter of wording, so a simple majority
for one of the other should settle the matter.

There are alternative definitions in USEPRO, but no corresponding wordings
for the References header in USEFOR yet, so maybe we should wait until
there are.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 21 Dec 2004 19:27:29 +0000

Charles Lindsey wrote:

>[...]
The WG has enough things to do already, so we should do 2) or even 3).

>[...] 
And based on recent discussions I am adding to this list:

5. Review Injection-Info syntax (this might be related to Complaints-To)

6. Remove filename parameter from the Archive header.

7. FWS issue in headers.

Any other major issues for the USEFOR document that I've missed?

Alexey

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 21 Dec 2004 22:47:12 +0100

Alexey Melnikov wrote:

> other major issues for the USEFOR document that I've missed?

Define a Message-ID compatible with NNTP, get rid of NO-WS-CTL.

Maybe replace "id-right" by "mdomain", because that's the idea:

"domain" is already defined in 2822 and not what we want, and
we also can't use the 2822 "id-right" as is.  Simple proposal:

| msg-id-core     =  "<" unique "@" mdomain ">"
| mdomain         = dot-atom-text / ( "[" dot-atom-text "]" )

dot-atom-text is defined in 2822.  Many strange characters, but
no serious problems, i.e. no "<", "@", ">", "[", "]", "\", '"',
WSP, or NO-WS-CTL.  It's just KISS to use dot-atom-text for
both cases, domain or address literal.

| unique          = 1*( unique-qtext )
| unique-text     = %d33 / %d35-61 / ; no WSP, no DQUOTE
|                   %d63 / %d65-91 / ; no ">", no "@"
|                   %d93-127         ; no "\", no CTL

That's automatically in canonical form, as wanted by NNTP, and
it's already more than s-o-1036 allowed:  !()<,;:[] are "new".

                       Bye, Frank

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 22 Dec 2004 16:49:14 +0000

>Define a Message-ID compatible with NNTP, get rid of NO-WS-CTL.
Of course.

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 22 Dec 2004 14:44:00 GMT

>The WG has enough things to do already, so we should do 2) or even 3).

Well the definition of these, should we retain them, is not in dispute (it
just codifies a moderately implemented existing practice). The texts
exist. It would cost 15 minutes of my time to remove them from USEPRO, or
15 minutes of Ken's time to add them to USEFOR, whichever we decide.

What were the arguments against? Just a little unease expressed by one,
maybe two, persons IIRC.

>5. Review Injection-Info syntax (this might be related to Complaints-To)

Well that _would_ consume a lot of WG time :-( . Last I heard, even Russ
was prepared (reluctantly) to put up with the MIME style syntax.

But if this is to be considered, may we have some proposals please? I
presume we still want some form of keyword:attribute pairs; it needs to be
extensible (including x-keywords for occasional use) and it needs to allow
for non-ASCII characters in the attributes. OTOH, splitting over-long
attributes across several lines is not really needed.

>6. Remove filename parameter from the Archive header.

Had probably better wait until we have decided on Injection-Info. If a
MIME filename= parameter is considered good enough for the
Content-Disposition header (where MUAs regularly need to parse and act
upon it), then I can't see why it is not suitable for Archive (where it
only needs to be interpreted by a few special purpose archiving agents).

>7. FWS issue in headers.

Please specify what you have in mind here.

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 22 Dec 2004 17:33:31 +0000

I am answering the easy questions first:

>15 minutes of Ken's time to add them to USEFOR, whichever we decide.
I agree.
I just don't want to spend another few weeks debating the issue.

[...]

>only needs to be interpreted by a few special purpose archiving agents).
I have not seen any argument in support of the "filename" parameter.
If there is no support - we don't need to document it.

>Please specify what you have in mind here.
I was just thinking about recent discussion with Frank about mandatory 
space after field names, etc.

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 22 Dec 2004 19:04:31 GMT

>| mdomain         = dot-atom-text / ( "[" dot-atom-text "]" )

That is indeed a subset of RFC 2822, but if you are going to restrict what
is allowed in a no-fold-literal, then you may as well go the whole hog and
adopt the RFC 2821 restriction (i.e. to an ipv4 or ipv6 address).

>|                   %d93-127         ; no "\", no CTL

But that is not a subset of RFC 2822, and could not therefore be accepted.
For example, you have allowed <[foo]@example.com>, which is not allowed by
RFC 2822.

If our Chair is willing to permit any further restrictions on the
message-id format, then something along the lines you suggest might be in
order, but please bear in mind that we shall want to sell back whatever we
decide to the next revision of RFC 2822, and the more we change the
harder that will be.

The only strong argument we have at the moment is the incompatibility with
the current NNTP draft (and I an waiting to hear Russ's views on that).
There might be some mileage to be gained from that, at least so as to
remove the NO-WS-CTL.

---

From: Dirk Nimmich <[email-masked]>
Date: Thu, 23 Dec 2004 00:01:55 +0100

Alexey Melnikov wrote:
> Charles Lindsey wrote:
>> In <[email-masked]> Alexey Melnikov 
>> <[email-masked]> writes:
[...]
>> Had probably better wait until we have decided on Injection-Info.
>> If a MIME filename= parameter is considered good enough for the
>> Content-Disposition header (where MUAs regularly need to parse
>> and act upon it), then I can't see why it is not suitable for
>> Archive (where it only needs to be interpreted by a few special
>> purpose archiving agents).
>>
> I have not seen any argument in support of the "filename"
> parameter.  If there is no support - we don't need to document it.

I haven't had the time to follow the discussion on this topic, but
the original intent was to replace the Archive-Name pseudo header
(or to have a formal way to represent it). If the semantics for
Content-Disposition's filename parameter were the same, that would
be alright; but since Archive-Names routinely contain (logical) path
information which is to be deleted for security reasons in
Content-Disposition's filename parameter I doubt it is.

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 23 Dec 2004 08:14:08 +0100

> That is indeed a subset of RFC 2822, but if you are going to
> restrict what is allowed in a no-fold-literal, then you may
> as well go the whole hog and adopt the RFC 2821 restriction
> (i.e. to an ipv4 or ipv6 address).

Yes, but the 2821 definitions are not exactly what we want for
a Message-ID.  Some points:

- at least one dot, fine for SMTP, TLDs wanting to be a host
  like TV can be ignored.  Less convincing for us, if we want
  to allow UUCP names.
- ldh-string, I don't see why <[email-masked]> is a
  bad idea ("_" is not in ldh-string), dito other atext char.s
  not covered by ldh-string
- simplicity, a memo about news articles is not in the business
  of defining address literals, that's the job of other texts,
  and so far the results are a proper subset of [dot-atom-text]
- if a really stupid implementation get's some things wrong, at
  least dot-atom-text should work whereever it's used.  It's
  essentially the same idea as your msg-id-core, no [CFWS] etc.

> that is not a subset of RFC 2822

Then it's wrong.  And allowing ":" would be very dubious, this
could result in <news:x@example> or <mailto:x@example>, where
a browser (or human) would be forced to guess, is it a URL or
a complete Message-ID ?

> something along the lines you suggest might be in order

How about this (= your idea of a "must-quote" again):

| unique       = dot-atom-text / ( DQUOTE unique-quote DQUOTE )
| unique-quote = 1*([dot-atom-text] unique-lit [dot-atom-text])
| unique-lit   = ["."] unique-text ["."]
| unique-text  = "(" / ")" / "<" / ; all specials, minus ">",
|                "[" / "]" / ":" / ; minus DQUOTE, minus "\",
|                ";" / "@" / ","   ; and minus "." (dot)

If you insist on it add the quoted-pairs "\\" and ("\" DQUOTE).
The unique-quote and unique-lit productions ares not parser-
friendly, please improve it somehow.

> please bear in mind that we shall want to sell back whatever
> we decide to the next revision of RFC 2822, and the more we
> change the harder that will be.

ACK.  There is already a strange restriction, the limit 250 for
a complete Message-ID in NNTP.  AFAIK a domain can have up to
253 characters (actually 255 minus overhead used for DNS), and
2822 mumbles something about 64 char.s for the local part.  If
I add this I arrive at 320 (incl. <@>) instead of 250.  Nothing
we can change, but it's still odd.

                        Bye, Frank

---

From: Henry Spencer <[email-masked]>
Date: Thu, 23 Dec 2004 02:46:31 -0500 (EST)

On Thu, 23 Dec 2004, Frank Ellermann wrote:
> I add this I arrive at 320 (incl. <@>) instead of 250...

The 250 limit for news message IDs comes from son-of-1036, which was
essentially documenting the implementation limits of common news software. 

                                                          Henry Spencer
                                                       [email-masked]

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 23 Dec 2004 09:22:30 +0100

Frank Ellermann wrote:

> friendly

They are also wrong, I forgot 2*(".") as another "must-quote",
and anything starting or ending with dots is a "must-quote".

| unique-quote = 1*( unique-part unique-lit unique-part )) /
|                ( 1*(".") unique-part) / (unique-part 1*("."))
| unique-part  = *([dot-atom-text] / [unique-lit])
| unique-lit   = ( *(["."]) unique-text *(["."])) / 2*(".")
| unique-text  = "(" / ")" / "<" / ; all specials, minus ">",
|                "[" / "]" / "@" / ; minus DQUOTE, minus "\",
|                ":" / ";" / "," / ; and minus "." (dot)
|                "\\" / ("\" DQUOTE)

Now I've added the two dubious "must-quote" quoted pairs.  It's
still not very parser-friendly, what I really wanted is simply:

1 - it starts with a dot, or 2 - it ends with a dot. or
3 - it contains 2*("."),  or 4 - it contains \\ or \" or
5 - it contains one or more ()<[]@;:, with an arbitrary number
    of dots before or after the quoted-pair (4) or special (5).

                            Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 23 Dec 2004 09:51:51 +0100

Henry Spencer wrote:
 
> The 250 limit for news message IDs comes from son-of-1036,
> which was essentially documenting the implementation limits
> of common news software.

Okay, if there must be a limit then sticking to an established
number below 256 is fine at least for the PASCAL fans ;-)  From
time to time the question of "excessively" long Message-IDs is
discussed in de.admin.net-abuse.news, because some tools and
UAs (especially OE) don't like this.  Even some news servers
are unhappy if the unfolded References are "excessively" long.

For "excessively" read "something below 250 or (1+3)*250 + 18".

                       Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 23 Dec 2004 10:58:30 +0100

Frank Ellermann wrote (another error):

  [Charles said:]

> a memo about news articles is not in the business

Nice argument, but completely wrong, ":" is not in atext, but
RfC 3513 2.2 and other texts with address literals need it.
We could get away with...

| mdomain          = dot-atom-text / ("[" address-literal "]")
| address-mliteral = 1*( atext / "." / ":" )    ; see RfC 2821

...if the idea is to enumerate the legal char.s without the
semantics.  Otherwise copying address-literals from 2821 is
better, as Charles said.

A simplified solution for the "must-quote" double-dots problem:

| unique-quote = 1*( unique-part unique-lit unique-part )) /
|                ("." unique-part) / (unique-part ".")
| unique-part  = *([dot-atom-text] / [unique-lit])
| unique-lit   = *(".") unique-text *(".")
| unique-text  = "(" / ")" / "<" / ; all specials, minus ">",
|                "[" / "]" / "@" / ; minus DQUOTE, minus "\",
|                ":" / ";" / "," / ; and minus "." (dot), plus
|                "\\" / ("\" DQUOTE) / ".."

I've replaced 2*(".") in unique-lit by ".." in unique-text,
and any leading or trailing 1*(".") by "." in unique-quote,
a leading or trailing 2*(".") is handled by unique-text "..".

                      Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 23 Dec 2004 13:27:15 +0100

No more nonsense before 12-24, I promise, but the proposed
msg-id-core syntax was just ugly, here's my "xmas-edition":

| msg-id-core     =  "<" unique "@" mdomain ">"
| mdomain         = dot-atom-text / ("[" address-literal "]")
| address-literal = 1*( atext / "." / ":" )    ; see RfC 2821

| unique          = dot-atom-text / ( DQUOTE unique-quote DQUOTE )
| unique-quote    = ( "." [unique-part] ) /
|                   ( [unique-part] "." ) /
|                   ( [unique-part] unique-literal [unique-part] )
| unique-part     = 1*( atext / "." / unique-literal )
| unique-literal  = "(" / ")" / "," / ; all specials, minus ">",
|                   "[" / "]" / "@" / ; minus DQUOTE, minus "\",
|                   ":" / ";" / "<" / ; minus single ".", plus:
|                   ".." / "\\" / ( "\" DQUOTE )

                    Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 23 Dec 2004 19:15:55 GMT

>I just don't want to spend another few weeks debating the issue.

A quick head count, perhaps?

>If there is no support - we don't need to document it.

Well Dirk has just posted on this. Please do bear in mind that it is hard
to add new stuff to existing headers in the future if you do not make
provision at the start. That is the whole beauty of MIME-style parameters,
which have been very successful in allowing expansion of the Content-Type
header (though that beauty was indeed marred by the RFC 2231 mess).

>space after field names, etc.

I don't think there is any dispute about what is / is not to be allowed.
Just whether we try to express all the rules in the syntax. I have been
trying to explain why it is better left as it is.

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 23 Dec 2004 19:25:29 GMT

>| address-mliteral = 1*( atext / "." / ":" )    ; see RfC 2821

Yes, that is the only bit of RFC 2821 I had in mind to adopt. Indeed, Dan
Kohn's original draft (and the usefor-draft-01) had that, but it was taken
out because the WG had never agreed to that, and it was contrary to our
remit to stick with RFC 2822. But if our Chair is willing to allow it ...

>|                "\\" / ("\" DQUOTE) / ".."

But that looks exactly the same as my current <id-left>. Was there
supposed to be some difference?

---

From: Henry Spencer <[email-masked]>
Date: Thu, 23 Dec 2004 14:29:24 -0500 (EST)

On Thu, 23 Dec 2004, Frank Ellermann wrote:
> UAs (especially OE) don't like this...

Yeah, the main reaction I got on that number was "you're sure you don't
want to make it a lot smaller?". :-)

If I'd simply been trying to pick a *reasonable* size limit, I'd probably
have used something around a third of that.  But I'm sure that would have
sparked debate, while the software limit was hard to argue with. 

                                                          Henry Spencer
                                                       [email-masked]

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 23 Dec 2004 19:08:31 GMT

>>Define a Message-ID compatible with NNTP, get rid of NO-WS-CTL.

I think we need some indication from you as to how much leeway we have to
make chenges to the msg-id format.

The <msg-id> format in RFC 2822 is a mess, and I would dearly like to
throw it away and start over. But then there are lots of things in RFC
2822 that I would dearly like to throw away, but my understanding is that
our remit is to follow RFC 2822 as closely as possible.

But, for compatibility with the NNTP draft, it seems we have to do
something about NO-WS-CTL, and that might give us an excuse to tidy some
other things, such as restricting the id-right to be a dot-atom-text or an
IPv[46] literal. You will have seen the suggestions Frank has been making.
How many of them are you prepared to permit? It would seem that getting
rid of NO-WS-CTL is the only one with a cast-iron technical justification.

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 23 Dec 2004 22:11:57 +0100

> Yes, that is the only bit of RFC 2821 I had in mind to adopt.
> Indeed, Dan Kohn's original draft (and the usefor-draft-01)
> had that, but it was taken out because the WG had never agreed

Sorry, anything before 07 is far outside my radar, do you
recall the specific arguments against it ?  Was it the simple
form shown above, an enumeration of legal characters with a
pointer to one relevant RfC, or was it a complete copy of the
syntax ?

> if our Chair is willing to allow it ...

Alexej said that a Message-ID compatible with NNTP is okay,
this covers no NO-WS-CTL and some auto-canonical "unique"
instead of the 2822 id-left.  It also kills the 2822 id-right
for the same reasons.  Renaming your "id-right" to "mdomain",
and your "id-left" to "unique" is only an editorial trick to
highlight the purpose of these beasts as in 1036 and s-o-1036,
and because it's different from 2822 "id-left" / "id-right".

 [unique]
> But that looks exactly the same as my current <id-left>. Was
> there supposed to be some difference?

For the later and improved xmas-edition see
<news://news.gmane.org/[email-masked]>
<http://article.gmane.org/gmane.ietf.usenet.format/28304/raw>

One small difference is anything starting or ending with a dot.
That's not covered by dot-atom-text.  And anything with ".."
(two or more dots) for the same reasons.

Your mqspecial contains "." (single dot), and that's not good
enough for a "must-quote", the "." is also in dot-atom-text.

My unique-literal has ".." instead of your mqspecial ".", the
rest is identical.  I don't use no-fold-literal, that's a 2822
term, redefinitions are too confusing.  Instead of your mqtext
I've unique-part = 1*( atext / "." / unique-literal ), that's
IMHO clearer for readers familiar with 2822 atext and specials.

Parsing some ".." constructs will be ambiguous, OTOH there's no
reason to parse "unique-quote" as long as it either starts or
ends with a dot, or contains one unique-literal, in addition to
any number of atext, dots, and further unique-literals.

                           Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 23 Dec 2004 22:55:17 +0100

Charles Lindsey wrote:

  [3. Mail-Copies-To and Posted-And Mailed}

> A quick head count, perhaps?

Count me against Posted-And-Mailed.  You say that some UAs know
it, that's fine, but my UA does not, and in the extremely rare
cases when I get a "courtesy copy" I want this info in the body
of the message.  Otherwise I would love to auto-killfile any
unsolicited Posted-And-Mailed without info in the body, but my
UA also has no killfile (for news).  This header is a PITA and
conflicts with any minimal netiquette I know of.

  [6. Remove filename parameter from the Archive header]
> Please do bear in mind that it is hard to add new stuff to
> existing headers in the future

The complete "Archive:" is dead on arrival, even without the
filename.  Some existing software supports X-NoArchive somehow,
e.g. Google now interprets it as "expires after 7 days".  X-No
discussions in groups like de.soc.recht.datennetze (I don't
know an equivalent, something like "cyber law") are always
unsatisfying, many users don't want their articles abused by
a commercial Web forum as "support", but X-No doesn't really
help.  Maybe a recommendation in USEAGE about a configurable
"Expires:" default would be better than X-No or Archive: NO ?

> That is the whole beauty of MIME-style parameters,

After 2231 nothing about MIME parameters is still beautiful :-(

And the filename stuff just makes no sense for different file
systems, let alone I18N.  Martin's Archived-At: is the future.

> that beauty was indeed marred by the RFC 2231 mess

Yes.  If the FAQ-hunters really want new tricks they could try
to (ab)use Summary: or Keywords:, almost nobody else uses these
headers, adding another dead header is dubious.   Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 27 Dec 2004 12:40:19 GMT

>conflicts with any minimal netiquette I know of.

All a UA can do with that header is to display it (and a decent UA should
be configurable to display and header the user asks for, though sadly that
is not always the case). But there is no way we could mandate putting
stuff in the body. But I think Mail-Copies-To is the header we really need
to decide upon; Posted-And-Mailed just comes with it as an added extra.

>e.g. Google now interprets it as "expires after 7 days".

Again, a decent UA will display it if a user asks. But normal UAs don't
need to see it - it is just specialized archiving agents that need to see
and act upon it, and Google is the obvious example of such an agent.

>  Maybe a recommendation in USEAGE about a configurable

No, because there are legitimate uses of "Expires:" (keeping FAQs around
until the next issue arrives) that are quite consistent with "Archive:
yes".

>systems, let alone I18N.  Martin's Archived-At: is the future.

Martin's Archived-At: fulfils a totally different function. It is for
where the author has already arranged for a permanent archive, not for
where is is going to be achived by a third party. Though, come to think of
it, it might be OK for FAQs in news.answers where the author knows in
advance that it will automatically get archived by rtfm.mit.edu and its
mirrors.

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 27 Dec 2004 12:22:39 GMT

>pointer to one relevant RfC, or was it a complete copy of the

Actually, looking at RFC 2821, I see that it tried to give a syntax for
<IPv4-address-literal>, <IPv6-address-literal> and
<General-address-literal>, involving <Standardized-tag>s such as "IPv6" or
whatever else IANA might specify. It used the syntactic elememt
<dcontent> from RFC 2822, and hence it allowed most things allowable in
the current <id-right>. Dan Kohn, now I look more closely, had a different
allowance of HEXDIG, ",". ":" and <quoted-pair>s, different from anything
else seen anywhere else, so forget it.

The whole thing looks such a mess that I cannot now see any reason for
departing further than needed from RFC 2822 as regards the id-right.

Certainly, adopting <address-literal> from RFC 2821 does not help us,
because it too contains quoted-pairs, which would have to be canonicalized
in our version, as is done already in our version of <id-right>.

>instead of the 2822 id-left.

Alexey has not really made any pronouncement yet. I am waiting for him to
do so, and also for Russ to comment on the NNTP definition which disallows
NO-WS-CTL. And yes, I now see that NO-WS-CTL also appears in <id-right>.

I am coming to the conclusion that we should just omit NO-WS-CTL from our
versions if <id-left> and <id-right>, and leave it at that.

>(two or more dots) for the same reasons.

If an <id-left> starts or ends with a ".", then it is not allowed under
RFC 2822, and hence we cannot allow it either.

>enough for a "must-quote", the "." is also in dot-atom-text.

Aaaaghhh! Yes! That is indeed a bug in my syntax.

>rest is identical.

OK, I shall look at your latest syntax.

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 28 Dec 2004 02:32:55 +0100

Charles Lindsey wrote:

 [mdomain instead of id-right]
> The whole thing looks such a mess that I cannot now see any
> reason for departing further than needed from RFC 2822 as
> regards the id-right.

If you want to get rid of NO-WS-CTL and quoted pairs, then...

...is good enough.  Address literals don't contain [, \, or ].

The stuff about "dcontent after a generalized tags registered
with IANA" in RfC 2821 is science fiction, but the pointer to
RfC 2373 (2.2 Text Representation of Addresses) IP6v is a fact.

> adopting <address-literal> from RFC 2821 does not help us,

Yes, I didn't see the "generalized" nonsense.  And I'm not sure
about the IPv6 tag in 2821, do we really want it ?  It's not
very critical in a Message-ID, but it starts to get confusing
in a path-identity.  Why not use the RfC 2373 format without a
tag ?  Anything with ":" is IPv6, anything without ":" is IPv4.

> Alexey has not really made any pronouncement yet.

Quoting <news://news.gmane.org/[email-masked]> :

|>>Define a Message-ID compatible with NNTP, get rid of NO-WS-CTL.
|
|> Of course.

That's a clear "go", he can say "stop" if we're getting carried
away and start to reinvent the Internet... :-)

> I now see that NO-WS-CTL also appears in <id-right>.

It's everywhere in 2822, even in Subject and other unstructured
headers.  The idea was probably to avoid "over-specifications",
and the minimum "they" (incl. you ;-) could get away with was
to exclude CR, LF, and NUL in message/rfc2822.  That's the old
"be liberal in what you accept" rule, but it's too liberal for
our purposes, we want something working with existing servers
and newsreaders.

> I am coming to the conclusion that we should just omit
> NO-WS-CTL from our versions if <id-left> and <id-right>, and
> leave it at that.

Then you'd still allow <unique@[IPv9:\]\\\[]> and that's wrong,
IPv9:]\[ is no address literal.  BTW, your mdtext enumerates
%d33-61 etc., but ">" is %d34, and the comment says 'no ">"'.

My style to base enumerations on atext + additions allows to
catch such simple errors more easily.

  [unique vs. id-left]
>> anything starting or ending with a dot. That's not covered
>> by dot-atom-text.  And anything with ".." (two or more dots)
>> for the same reasons.

> If an <id-left> starts or ends with a ".", then it is not
> allowed under RFC 2822, and hence we cannot allow it either.

Yes, I was talking about the differences in id-left and unique,
i.e. your no-fold-quote and my unique-quote:

RfC 2822 and unique-quote allow <".xy."@example>, but that's
not the intention in your no-fold-quote.  Dito <"x..y"@example>.

> OK, I shall look at your latest syntax.

Our main difference or problem are now address literals.  JFTR:

                    Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 28 Dec 2004 11:01:16 GMT

>Content-Disposition's filename parameter I doubt it is.

That was not so much an intent, as a possible application that became
apparent soon afterwards.

I mentioned in reply to Frank yesterday that Martin Dürst's proposed
Archived-At header (if it ever happens) might also be suitable as an
Archive-Name pseudo header replacement, but now I am not so sure, since
that would normally provide a URL including a host name (e.g.
Archived-At:
  <ftp://rtfm.mit.edu/pub/usenet/news.answers/usenet/welcome/part1>),
whereas what is used in the Archive-Name pseudo header (and what I had in
mind for the Archive filename parameter) is just the
"usenet/welcome/part1" bit, on the grounds that there are lots of mirrors
of rtfm.mit.edu, all using the same filename (not necessarily with the
same Path preceding it).

Also, I do not quite follow your point about deleting path information for
security reasons. There is no requirement in RFC 2183 for such deletion,
though there is a warning of dangers under Security Considerations. But
then there is also a similar warning under Security Considerations in our
draft for the Archive header, so there is no real difference there.

Anyway, the question to be decided is whether or not to retain the
filename parameter in our Archive header, both because of its possible
future use, and also to allow future extensions to add further parameters.

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 28 Dec 2004 12:24:42 GMT

>| address-literal = 1*( atext / "." / ":" )    ; see RfC 2821

But you can't say "see RfC 2821" when the syntax you give bears no
resemblance to anything actually in RFC 2821.

The syntax actually given in RFC 2821 may well be, as you say "science
fiction". The syntax given in RFC 2822 for <domain-literal> and
<no-fold-literal> is also "science fiction" (and a superset of the RFC
2821 one). So which piece of science fiction should we follow? I think it
_has_ to be the RFC 2822 one, since our remit is to be compatible with RFC
2822 wherever possible.

Now our Chair _might_ allow us to go for something much simpler such as
you suggested, and it would indeed be tidier, but there is no _technical_
requirement to do so, so it would be purely a matter of tidiness.

Ruling from the Chair, please?

If you stick with something close to RFC 2822, then it would have to be:

   mdomain         = dot-atom-text / ("[" no-fold-literal "]")
   no-fold-literal = "[" *( mdtext / "\[" / "\]" / "\\" ) "]"
   mdtext          = %d33-61 /   ; printable US-ASCII
                     %d63-90 /   ; not including
		     %d94-126    ; ">", "[" , "]", or "\"

Observe the omission of NO-WS-CTL.

>|                   ".." / "\\" / ( "\" DQUOTE )

Yes, that syntax would work, though I am not so convinced by the names you
want to give to the various pieces (particularly the "unique" stuff, since
the id-left has no requirement to be globally unique on its own - only
when taken in conjunction with the id-right).

Anyway, for the benefit of others who have not been following this
exchange too closely, here is a set of examples that are permitted by RFC
2822, but not to be permitted in USEFOR:

    <"abcd"@example.com>	(use <[email-masked]>)
    <"ab\cd"@example.com>	(use <[email-masked]>)
    <"a.b.c"@example.com>	(use <[email-masked]>)
    <abcd@[ab\cd]>		(use <abcd@[abcd]>)

and here are some example still permitted by both:
    <"a\"bc\"d"@example.com>
    <"a..c"@example.com>
    <".a.b.c."@example.com>
    <abcd@[a\[bc\]d]>

and observe that
    <[email-masked]>
and <[email-masked]>
are _different_ <msg-id>s (opinions differ as to whether they are different
in RFC 2822).    

I think all those examples should appear in our draft.

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 28 Dec 2004 12:39:22 GMT

>...is good enough.  Address literals don't contain [, \, or ].

See my response to your other message with your "Christmas Syntax".

>RfC 2373 (2.2 Text Representation of Addresses) IP6v is a fact.

And RFC 2373 is different Yet Again :-( .

>away and start to reinvent the Internet... :-)

I think "Of course" merely meant it could go on our agenda of Issues, or
at most that we could get rid of the NO-WS-CTL problem. But there is still
time for him to permit more.

>IPv9:]\[ is no address literal.

   1. It's allowed in RFC 2822
   2. It's allowed in RFC 2821 (if registered with IANA)
   3. It's allowed in NNTP
   4. It's ugly

Ruling awaited from the Chair.

>  BTW, your mdtext enumerates

Eh? ">" is %d62.  %d34 is <DQUOTE>.

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 29 Dec 2004 23:56:25 +0100

> And RFC 2373 is different Yet Again :-( .

Yes, and I must admit that I know almost nothing about IPv6,
but the format of IPv6 address literals isn't too difficult,
nothing like say UTF-8 minus surrogates. ;-)

Trying to find the new (2004) RfC about IPv6 example addresses
I stumbled over RfC 3513, it obsoleted 2373, but the chapter
<http://www.apps.ietf.org/rfc/rfc3513.html#sec-2.2> is still
the same.

>    2. It's allowed in RFC 2821 (if registered with IANA)

4 - It's not only ugly, it's plain nonsense, and definitely no
    IPv6 or even IPv4.
3 - NNTP doesn't "allow" it, they simply don't care, because
    it's none of their business to define address literals, as
    long as the result pa[rs]ses as NNTP Message-ID
1 - dito, 2822 only wants to reach the closing ">" without any
    syntax error, and they did it in a way that's incompatible
    with NNTP.
2 - 2821 also doesn't care, the IANA registry for address tags
    doesn't exist or rather I don't find it on the IANA pages.

    Is something like [IPv6:2001:DB8::] really okay in SMTP ?
    IPv6 URLs simply use [3ffe:2a00:100:7031::1] without tag,
    see RfC 2732 (some months older than RfC 2821).
 
> Ruling awaited from the Chair.

It's hard for Alexej to decide the address literal issue while
we are (or at least I am) still unsure about the _real_ format,
with or without tag.  You said that you found something in an
old draft with hex. digits, and that was probably the format in
RfCs 2373 (now 3513) and 2732.  Maybe RfC 2821 got it wrong (?)

>> your mdtext enumerates %d33-61 etc., but ">" is %d34, and
>> the comment says 'no ">"'.

> Eh? ">" is %d62.  %d34 is <DQUOTE>.

Sorry.  One more reason to use enumerations based on atext - I
simply don't know all ASCII decimal codes by heart. ;-)  You
must of course use the %d-style for odd cases like "poster".

Okay, mdtext includes DQUOTE, because you want <unique@["?~*"]>
to be a "valid" Message-ID for the system with IPv? "?~*" in
the future IANA registered IPv?, or isn't that just nonsense ?

                           Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 30 Dec 2004 01:15:14 +0100

Charles Lindsey wrote:

> you can't say "see RfC 2821" when the syntax you give bears
> no resemblance to anything actually in RFC 2821.

ACK, please replace 2821 by 3513 for IPv6, and any _good_ RfC
defining IPv4 literals.  Unfortunately 2821 isn't good enough.

> our remit is to be compatible with RFC 2822 wherever
> possible.

We're also trying to be compatible with 1036, s-o-1036, and the
finished NNTP draft (or whatever its status is).  The proposed...

| address-literal = 1*( atext / "." / ":" )

...is "compatible" with 2822, if you take it as "proper subset",
and that's in fact what you've done so far:  msg-id-core instead
of msg-id, absolutely no ">", even if it's quoted, and no more
NO-WS-CTL.

If your strategy is as little derivations from 2822 as possible,
then you in fact allow stuff like <unique@[212.82.226.0/24]> or
<unique@[http://purl.net/xyzzy/]> as Message-ID.  Do you really
want this ?

> Now our Chair _might_ allow us to go for something much
> simpler such as you suggested, and it would indeed be tidier,
> but there is no _technical_ requirement to do so

The technical requirement to generate proper Message-IDs is to
use an FQDN as RHS.  And only if it's absolutely necessary an
address literal.  But not a CIDR, URL, ISBN. SSN, phone number,
avian carrier license number, or what else.  Not because that
is technically impossible, but because implementors will get it
wrong, and that could cause technical trouble somewhere else,
where you don't expect it and don't need it.

> mdomain         = dot-atom-text / ("[" no-fold-literal "]")

Please don't use 2822 terms like "no-fold-literal" with a new
definition.  Many UAs support news and mail, and an implementor
could be tempted to use the first definition he finds.  You can
use "address-literal", it's a hint about the intended semantics
no matter what the syntax says.  Same idea as for the new terms
s/id-right/mdomain/ and s/id-left/unique/

>                   %d94-126    ; ">", "[" , "]", or "\"

Okay, that's the minimal derivation from RfC 2822.  OTOH it's
something like a maximal derivation from the semantics of an
address literal, let alone 1036 and its son.

 [unique-* stuff]
> that syntax would work, though I am not so convinced by the
> names you want to give to the various pieces (particularly
> the "unique" stuff, since the id-left has no requirement to
> be globally unique on its own

True, but it's the name used in RfCs 1036 and 1738.  And if you
have a some = thing production then it's natural to use "some"
as prefix for related terms "some-begin", "some-item", etc., as
in "path", "path-list", "path-identity", and "path-delimiter".

Maybe unique-text would be better than unique-literal for your
general 2822-style, although "..", "\\", '\"' are no char.s (?)

                    Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 30 Dec 2004 01:34:09 +0100

Charles Lindsey wrote:

> the question to be decided is whether or not to retain the
> filename parameter in our Archive header, both because of its
> possible future use, and also to allow future extensions to
> add further parameters.

The latter is an unrelated issue, here's an example (RfC 3834):

|  auto-submitted-field     = "Auto-Submitted:" [CFWS]
|                             auto-submitted [CFWS] CRLF
|  auto-submitted           = ( "no" / "auto-generated" /
|                             "auto-replied" / extension )
|                             opt-parameter-list
|  extension                = token
|  opt-parameter-list       = *( [CFWS] ";" [CFWS] parameter )
|
| The symbols "CFWS" and "CRLF" are defined in [N2.RFC2822].
| The symbols "token", and "parameter" are as defined in
| [N7.RFC2045] (as amended by [N4.RFC2231]).

RfC 3834 is nice, even if you consider its Subject: [Auto] tag
as "intentionally annoying the IETF-2822-cabal" ;-)  Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 30 Dec 2004 14:27:11 GMT

><unique@[http://purl.net/xyzzy/]> as Message-ID.  Do you really

Not particularly, but it is allowed in RFC 2822 and is technically
harmless in News.

>address literal.

No, there is no technical requirement to use an FQDN or IP as RHS (though
it is a practice much to be recommended, and USEAGE will indeed recommend
it).  The only technical requirement is that an FQDN or IP will at least
be a subset of what is allowed as RHS.

>   But not a CIDR, URL, ISBN. SSN, phone number,
>avian carrier license number, or what else.

All those things are permitted by RFC 2822. You have to provide a good
reason why we should rule them out, and moreover a reason that would not
have applied to email when RFC 2822 was written (eliminating NO-WS-CTL was
indeed such a reason because the new NNTP standard will not allow it).

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 30 Dec 2004 14:16:33 GMT

>RfCs 2373 (now 3513) and 2732.  Maybe RfC 2821 got it wrong (?)

No, the _real_ format, if there is such a thing, is totally irrelevant,
provided that it is at least permitted. RFC 2822 allowed considerably more
than proper IP addresses in the id-right - ours not to reason why. The
question is whether or not we can justify doing anything different from
RFC 2822.

>the future IANA registered IPv?, or isn't that just nonsense ?

RFC 2822 allows that. Nobody suggests that any future IP will allow it. So
what?

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 30 Dec 2004 15:38:50 GMT

>| [N7.RFC2045] (as amended by [N4.RFC2231]).

Aha! So Keith Moore has authored a standard which defines a header with
MIME-style parameters to allow for future extensibility, even though he
vehemently opposes any such headers suggested by other people on the
ietf-822 mailing list :-) .

But that header does set a precedent for the use of MIME-style parameters
in other headers, and what we propose for the Archive header (and the
Injectio-Info header) is of basically the same form.

And some of the wording from RFC 3834 might be useful to copy into USEFOR.

Do I take it that you are content to leave the Archive header as we have
so far defined it?

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 31 Dec 2004 19:54:43 +0100

Charles Lindsey wrote:

> the _real_ format, if there is such a thing, is totally
> irrelevant, provided that it is at least permitted.

For our discussion about the news URL I read some stuff in
<http://gbiv.com/protocols/uri/rev-2002/rfc2396bis.html#host>
2396bis uses this syntax:

| IP-literal = "[" ( IPv6address / IPvFuture  ) "]"
| IPvFuture  = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" )

For IPv6 there's a pointer to RfC 3513 and the complete fun.
There's no pointer for IPv4 (so probably the only existing
form is 2821, and 2396bis couldn't use it for several reasons).

| unreserved  = ALPHA / DIGIT / "-" / "." / "_" / "~"
| sub-delims  = "!" / "$" / "&" / "'" / "(" / ")" 
|             / "*" / "+" / "," / ";" / "="

Whatever that is, it's more than my...

...but still less than the dcontent in 2821 and 2822.  And you
can be sure that the 2396bis IP-literal won't cause havoc for
the news URL.  No DQUOTE, no ">", no "\", no "]", no "[", it's
perfect for our purposes.

Let's use another old UNIX rule, after I tried KISS and failed:
"build on the work of others".  Copy this 2396bis list and be
done with it.

> RFC 2822 allowed considerably more than proper IP addresses
> in the id-right - ours not to reason why.

The reasons are IMHO obvious, reach the closing ">", and don't
care about the semantics.  But our input is not only RfC 2822.
It's also 1036, s-o-1036, common sense, and compatibility with
what you do for the news URL.

BTW, you already gave the reason for id-right, you couldn't use
the right part of addr-spec (= domain) in RfC 2822 because that
allows CFWS.  Therefore you invented id-left and id-right, only
a hack to get addr-spec minus any CFWS.

You can't say that the right part of an addr-spec supports SSNs,
URLs, ISBNs, or other nonsense in square brackets.  It's either
a dot-atom-text or an address literal, nothing else.

                      Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 31 Dec 2004 20:23:06 +0100

Charles Lindsey wrote:

> Do I take it that you are content to leave the Archive header
> as we have so far defined it?

No, I still don't like the filename parameter.  You said that
the purposes is (quote) "its  possible future use, and also to
allow future extensions to add further parameters".

RfC 3834 found a solution for the latter.  You could copy its
opt-parameter-list + explanation to Archive and Injection-Info
if you like it.  Not the token, and maybe you could manage to
just forget the phrase "(as amended by [N4.RFC2231])", please ?

RfC 2231 is firmly on place three (after RfC 3865 and ISO 3166)
of my shit list.  MIME was a piece of art, what they did to it
in RfC 2231 is a crime.

Some minutes later, maybe also the token ?  I could then write
Archive: creativecommons.org_licenses_by-nc-sa_2.0_de

                         Done, bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: Archive
Date: Fri, 31 Dec 2004 20:23:06 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 23
Archive: creativecommons.org_licenses_by-nc-sa_2.0_de
Sender: [email-masked]

Charles Lindsey wrote:

> Do I take it that you are content to leave the Archive header
> as we have so far defined it?

No, I still don't like the filename parameter.  You said that
the purposes is (quote) "its  possible future use, and also to
allow future extensions to add further parameters".

RfC 3834 found a solution for the latter.  You could copy its
opt-parameter-list + explanation to Archive and Injection-Info
if you like it.  Not the token, and maybe you could manage to
just forget the phrase "(as amended by [N4.RFC2231])", please ?

RfC 2231 is firmly on place three (after RfC 3865 and ISO 3166)
of my shit list.  MIME was a piece of art, what they did to it
in RfC 2231 is a crime.

Some minutes later, maybe also the token ?  I could then write
Archive: creativecommons.org_licenses_by-nc-sa_2.0_de

                         Done, bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: msg-id
Date: Fri, 31 Dec 2004 19:54:43 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 52
Sender: [email-masked]

                      Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: FWS problem
Date: Fri, 31 Dec 2004 19:00:07 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 103
Sender: [email-masked]

Charles Lindsey wrote:

> When they start to develop RFC 2922bis, please join in and
> get them to remove some of their stupidities. But not now,
> for USEFOR.

It's not too bad, we'll fix their msg-id for the real world.

The remaining problems for 2822bis are only NO-WS-CTL and a
mandatory FWS after the colon.  I'm not too happy with our
SP plus "MUST be non-empty" for the body of the first line.

The old 2822 obs-stuff could be moved to an appendix together
with NO-WS-CTL.  The new obs would be a missing FWS after the
colon, ready, submit this 2822bis this year or wait for 2005 ?

> "Supersedes:" is just the example I chose to give.

Yes, my general idea to fix a leading or trailing [CFWS] did
not work as expected.  In the cases of Lines: and Supersedes:
it was intentionally ( [CFWS] => [FWS] ) => *WSP, because I
didn't want the comments there.

> the problem arises eberywhere you propose to use *WSP

Not where it was meant to replace [FWS] (all 7 cases of [FWS]
plus Lines: and Supersedes:).  It did not work for References:
(msg-id-list) and Injection-Info (but at least there was still
a real problem also with your syntax).

 [2822 FWS in Date:]
> that was an explcit exception put into RFC 2822 after
> discussion. But everywhere else it was understood that
> comments would be allowed wherever WSP was allowed. You can't
> have them in some headers but not in others, because no-one
> can be expected to remember which cases are which.

ACK, that's a point.  And References: _is_ a real 2822-header,
you only need a new syntax for it because you modified msg-id
and added msg-id-core.  And maybe because no separator at all
between msg-ids is dubious, s-o-1036 had 1*WSP.

> only because of loud protests by the server implementors that
> we agreed to make exceptions for the Newsgroups and Path
> headers, etc, but only on account of the performance hits.

Supersedes: is relevant for news servers, like Control:, where
you also have only three [FWS] or in the fixed variant two *WSP
and still one [FWS] in control-message.  No [CFWS] in Control:

For Lines: you could say that it's now deprecated and the old
syntax was Lines = "Lines:" SP *WSP 1*DIGIT *WSP CRLF

That's no lie, and if somebody tries new tricks with deprecated
headers, then old software might fail for Lines: (huge)9999(!)

> If there is something you want to forbid in a standard (such
> as empty header lines, etc), then it is an exceedingly Bad
> Idea to have two mechanisms for forbidding it

Then let's use your CWSP and WSPC trick.  It's a worse idea to
say something as a MUST in the text and then something else in
the syntax.  The [FWS] problem is already solved for 7+2 cases,
or 7+1 cases if you insist on Lines: (over)999(lines)

Only 5 (or 6) similar [CFWS] cases, all solved by your idea:

| WSPC = *WSP [ comment [CFWS] ]

Replace all (5 or 6) "leading" (near colon) [CFWS] by WSPC.

| CWSP = [ [CFWS] comment ] *WSP

Dito 5 or 6 "trailing" [CFWS] => CWSP (near CRLF), ready.
For our most critical example of the References: I get

| references  = "References:" SP msg-id-list CRLF
| msg-id-list = CWSP msg-id-core *( CFWS msg-id-core ) WSPC

That's a variant enforcing at least either a comment or a WSP
between msg-ids, it allows (x)<y@z>(a)<b@c>(u)<v@w>(e) and
<y@z> <b@c> <v@w>, but not adjacent <y@z><b@c><v@w>.   The RfC
2822 syntax is very similar:

| references  = "References:" 1*msg-id CRLF
| msg-id      = [CFWS] "<" id-left "@" id-right ">" [CFWS]

You don't use it, because msg-id one of these overloaded terms:

| msg-id      = [FWS] msg-id-core [FWS]

IMHO you should get rid of your special USEFOR msg-id, there's
exactly one line where you use it:

| message-id  = "Message-ID:" SP msg-id CRLF

Why not simply delete the special msg-id, and use msg-id-core ?

  message-id  = "Message-ID:" SP *WSP msg-id-core *WSP CRLF

                              Bye, Frank

To: LIST: [email-masked];
Xref: clerew local.usefor:20536
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: FWS problem
Date: Thu, 30 Dec 2004 16:12:59 GMT
Lines: 67
Sender: [email-masked]

> [NO-WS-CTL in utext]
>> Indeed, but politically impossible to change it unilaterally.

>Sooner or later you get me to believe in this IETF-2822-cabal,
>and then I'll start unilaterally a war against it.

Yes, please do. When they start to develop RFC 2922bis, please join in and
get them to remove some of their stupidities. But not now, for USEFOR.

>> you have forbidden, for example:
>> Supersedes:  (a comment)
>>     <[email-masked]>
>>     (another comment)

>Yes, "Supersedes:" is an odd case, it's no 2822 header, and you
>have already restricted it to one msg-id, s-o-1036 had a list.
>Here are some alternatives:

No, "Supersedes:" is just the example I chose to give. In fact, the
problem arises eberywhere you propose to use *WSP, even in the References
header (even though comments are "SHOULD NOT generate yet" there).

>They do have FWS in their syntax where they needed it, e.g. a
>Date: header has no CFWS before or within the date, only at the
>end.

Yes, that was an explcit exception put into RFC 2822 after discussion.

But everywhere else it was understood that comments would be allowed
wherever WSP was allowed. You can't have them in some headers but not in
others, because no-one can be expected to remember which cases are which.
There is no reason for us to break with that principle (indeed, taking it
on board was one of the first things this WG did). It was only because of
loud protests by the server implementors that we agreed to make exceptions
for the Newsgroups and Path headers, etc, but only on account of the
performance hits. It was _never_ the intention to exclude them for other
News headers, even for ones that we invented.

>Sure, I wanted our syntax to _reflect_ our MUST where possible,
>not to _replace_ it.  And it's not worth the effort for [CFWS],
>if that means that we need a CWSP and a WSPC.  But for [FWS]
>it's possible.

If there is something you want to forbid in a standard (such as empty
header lines, etc), then it is an exceedingly Bad Idea to have two
mechanisms for forbidding it, one for some cases and another for others.
It is a recipe for confusion and for the introduction of unintended
errors (of which your failure to allow header lines containing just a
comment was a typical example).

To: LIST: [email-masked];
Xref: clerew local.usefor:20535
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Archive (was: Issues outstanding)
Date: Thu, 30 Dec 2004 15:38:50 GMT
Lines: 46
Sender: [email-masked]

Date: Thu, 30 Dec 2004 17:33:24 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

Seth Breidbart <sethb@xxxxxxxxx>:

>Anybody who can log in to my machine can post through it.

All you've verified is that they have the password to an account. You've
not verified WHO they are or is they are actually the person to whom
any specific email address belongs. So no, they cannot post, if your 
earlier statement were true.

>Somehow, doing nothing has been transformed into acting in your
>universe.  

Rejecting an article is an action just as much as accepting it. 

>I wonder what happens if I get you to say your name backwards.

Sorry, I thought you were discussing this seriously, and not just trying
to waste my time.  I'll not bother reading further, since I doubt that
you've been honest in any part of it.

Date: Thu, 30 Dec 2004 17:11:36 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

John Stanley <[email-masked]> wrote:
> Seth Breidbart <sethb@xxxxxxxxx>:

>>If I can't verify you as the owner of that email address, then you
>>can't use my server to post with that email address.
> Heh heh. You must run a very quiet server. Nobody can post through it.

Anybody who can log in to my machine can post through it.

Google uses that rule, too; yet I see a lot of postings through them.

>>I'm not acting, I'm declining to act.
> Wrong. Excluding the decision on the From header, your action would be to 
                                                         ^^^^^^
See that word?

> accept the article. By not accepting it, you are ACTING, and you are 
> basing your action on information you do not have. 

Somehow, doing nothing has been transformed into acting in your
universe.  I wonder what happens if I get you to say your name
backwards.

>>If you can't verify the address to me, then you can never use it on my
>>server.
> Which server would that be?

Google groups acts that way.  (My personal server nobody cares about,
because nobody uses it even if they could validate their email address
by returning a token.)

>>I'm not assuming that it's possible, I'm stating that my policy
>>(that's my policy, I'm not requiring that anyone else adopt it)
>>requires that verification.
> A verification that is impossible.

A verification that google performs, therefore it's possible.

A verification that any ISP that has users logging in can perform.

A verification that any NSP that requires users to return an emailed
token can perform.

> By claiming that you require that verification, you imply that it is
> possible

It is possible.  Some sites do it.

>>> Wow. And Wow, you think that MOST of the articles that are posted to
>>> USENET shouldn't be posted because the address cannot be verified?
>>No, I never claimed that.
> Yessir, you did. I'll quote from just a few lines earlier:
>>>> And since they DO NOT KNOW, they should DO NOTHING SPECIAL.
>>>That is, they shouldn't post the article?
> Or was your "shouldn't" a typo?

No, that was referring to sites that choose to follow the policy I'm
writing about.  Other sites need not follow that policy: their
servers, their rules.

>>I claimed that it's an acceptable policy
>>for a server to refuse to post articles if the From header can't be
>>verified to the satisfaction of that server.
>
>>It's equally acceptable for a server to allow any From address ending
>>in ".invalid" to be used.
>
> Since verifying is impossible in many if not most cases,

So what?

> then what you are actually saying is that some admins ought not to
> be running servers.

I'm not suggesting that at all.  I'm saying that the admins who
require verification will not accept postings in the cases where
verification is actually impossible.  If, as you claim, that's a lot
of cases, then there are a lot of cases they won't accept.  Their
servers, their rules.  I know of a server whose policy is "You can use
this server if you're a personal friend of the owner."  That's even
more restrictive; would you therefore claim that the owner shouldn't
be running a server?

> And if WE tell posters they MAY use .invalid, then of course it is
> an acceptable "policy" to accept it. It is not an acceptable
> "policy" to not accept it, because that "policy" is really "don't
> conform to the standards".

What part of interoperability breaks if a server does not accept From
addresses ending in ".invalid"?

>>> I think that counts as a majority over the relatively few that
>>> Panix can.
>>Actually Panix can verify infinitely many addresses, too.  
>
> Hardly. But then, aleph1 is larger than aleph0 (if I am using the terms 
> correctly.)

You aren't.  The number of finite length strings over a finite
alphabet is aleph null.  Ignoring length limits (which we have to in
order to get any infinite number), Panix can verify
"sethb+<anything>@panix.com" as belonging to me, for aleph null values
of <anything>.  The total number of possible email addresses (still
ignoring length limits) is the same aleph null.

> There may be an infinity of Panix account they can verify, but there
> are an infinity of infinities of accounts they cannot. Still a
> majority.

Aleph null times aleph null is aleph null, not a larger infinity.

>>> That's what I said. Where did you get "the _site_ is permitted"?
>>Are you claiming that the owner of a machine is required to run the
>>machine the way _you_ want instead of the way _he_ wants?
>
> It has nothing to do with what I want. It has everything to do with
> what the standards say. They say that the poster MAY use
> .invalid. That implies that the server has to accept it, otherwise
> there would be a contradiction.

There's no contradiction.

The law says that I MAY go into a store and offer to pay them with
Canadian money.  The store MAY choose not to accept it; there's
neither a violation of the law nor a contradiction when that happens.

> And you didn't answer the question: I was talking about a section of
> the draft that says "the poster MAY", and you told me that this
> means the the server MAY. Where did you get that nonsense?

You're the one claiming that "the poster MAY" implies "the server
MUST".

>>You may ask me to post something.  I may decline to accede to your
>>request.  Those are both "MAY".
>
> That is not what we are discussing. "MAY ASK" is very different from "MAY
> USE". MAY ASK implies there is a question that you get to answer. MAY USE
> means there is no question, the poster MAY USE that address.

The poster MAY USE that address when he says POST.  The server MAY
reply "NO".

>>> A clearer statement of abdication is rarely seen here. The site will
>>> do what it wants no matter what we say, so why bother saying
>>> anything?
>>To tell it what it ought to do so that interaction with other sites
>>works properly.
>
> A site will do what it wants no matter what we say. Why bother saying 
> anything?

See above.

>>> Where the fuck do we say that posters don't have to pay their bills?
>>When you said that a poster MAY use ".invalid" and that the server is
>>therefore REQUIRED to post the message.
>
> Answer the question. Where does this draft say ANYTHING about paying 
> bills? It doesn't.

You're the one who claimed that the server is REQUIRED to accept and
post a message if the poster used ".invalid".  I'm the one saying that
the server is not REQUIRED to do something against its policies.  As
an example of such a policy, I gave "users must pay their bills".

So you are apparently claiming that some server policies ("pay your
bills") are permissible, while others ("use your verified email
address") are not.

>>> Why do you keep making this shit up?
>>It's a logical consequence of the rule that you're arguing for.
>
> And now you've fabricated some rule that I am supposedly arguing
> for. Why do you keep doing this?

Are you, or are you not, arguing that a server is not permitted to
have a policy of accepting messages only if it has verified that the
poster is entitled to use the email address in the From header?

>>>>If sites *have to* accept it, then they have to accept the articles
>>>>where it is used, right? 
>>> No, of course not. Don't be stupid.
>>So now you agree that a site need not accept an article that has a
>> From header ending in .invalid?
>
> Stop being stupid. We are talking about a criterion for acceptance
> based on the use of .invalid here, and nothing else. I've said that
> a site cannot reject an article because the poster has used a
> .invalid address because WE told the user he MAY use it.

And I'm saying that a site may have a more restrictive policy than
that.

>>Where do you think you get the right to decide on policies that sites
>>must follow? 
>
> This isn't site policy anymore. This draft tells posters they MAY do
> something. I didn't write that part of the draft, so it isn't me you want
> to argue with about this.

You're the one arguing that "the poster MAY" implies "the site MUST".
I haven't seen anyone else agreeing with you.

>>Maybe their rule is "must use address you have verified to be valid in
>>the From header."
>
> Still trying to put words in my mouth, huh? Maybe their rule is what
> I wrote.

In that case, there's no problem.  I'm not arguing against a site
having the rule you wrote.  You're arguing against a site having the
rule I wrote.

>>Except for people who want to post, where "verified" is the required
>>case.
>
> When verified is impossible, it cannot be the "required case".

It isn't always impossible.  When it is impossible, it doesn't happen,
therefore (since it's required) the posting doesn't happen.

> By saying it can be the "required case", you imply it IS always
> possible,

No, I imply it is _sometimes_ possible.

>>But they may, if they so choose, reject an article for not having a
>>From header that they do not know to be valid.
>
> It is easy to check for a syntactically valid from header; thus is
> it correct to imply that they can. It is impossible to verify, in
> most cases, a contextually valid From header, thus it is incorrect
> to imply they can.

The draft doesn't imply that they always or even usually can.

> Our draft makes this implication;

No, it doesn't.

Seth

Date: Thu, 30 Dec 2004 13:07:48 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

Seth Breidbart <sethb@xxxxxxxxx>:

>If I can't verify you as the owner of that email address, then you
>can't use my server to post with that email address.

Heh heh. You must run a very quiet server. Nobody can post through it.

>I'm not acting, I'm declining to act.

Wrong. Excluding the decision on the From header, your action would be to 
accept the article. By not accepting it, you are ACTING, and you are 
basing your action on information you do not have. 

In other words, if your decision to not accept the article is based solely 
on the lack of verification of the From address, then you are ACTING 
(rejecting the article is an action) on information that you do not have 
(the address is inavlid.)

>I'm acting based on the KNOWN FACT that you have failed to verify the
>address you're attempting to use.

In a sentence earlier above, you claimed you were not acting. And this 
"known fact" is a "bug" in the world that you cannot get around. 

>If you can't verify the address to me, then you can never use it on my
>server.

Which server would that be?

>I'm not assuming that it's possible, I'm stating that my policy
>(that's my policy, I'm not requiring that anyone else adopt it)
>requires that verification.

A verification that is impossible. By claiming that you require that 
verification, you imply that it is possible -- the same error that appears 
in the draft.

>> Wow. And Wow, you think that MOST of the articles that are posted to
>> USENET shouldn't be posted because the address cannot be verified?

>No, I never claimed that.

Yessir, you did. I'll quote from just a few lines earlier:

>>> And since they DO NOT KNOW, they should DO NOTHING SPECIAL.
>>That is, they shouldn't post the article?

Or was your "shouldn't" a typo?

>I claimed that it's an acceptable policy
>for a server to refuse to post articles if the From header can't be
>verified to the satisfaction of that server.

>It's equally acceptable for a server to allow any From address ending
>in ".invalid" to be used.

Since verifying is impossible in many if not most cases, then what you are 
actually saying is that some admins ought not to be running servers. 

And if WE tell posters they MAY use .invalid, then of course it is an
acceptable "policy" to accept it. It is not an acceptable "policy" to not
accept it, because that "policy" is really "don't conform to the
standards".

>> I think that counts as a majority over the relatively few that
>> Panix can.

>Actually Panix can verify infinitely many addresses, too.  

Hardly. But then, aleph1 is larger than aleph0 (if I am using the terms 
correctly.) There may be an infinity of Panix account they can verify, but 
there are an infinity of infinities of accounts they cannot. Still a 
majority.

>So now let's consider reality, instead.

Ok. Let's. It appears your use of Panix as an example is incorrect, since 
they have no published policy regarding what addresses may be used in the 
>From header of email.

>> That's what I said. Where did you get "the _site_ is permitted"?

>Are you claiming that the owner of a machine is required to run the
>machine the way _you_ want instead of the way _he_ wants?

It has nothing to do with what I want. It has everything to do with what 
the standards say. They say that the poster MAY use .invalid. That implies
that the server has to accept it, otherwise there would be a contradiction.

And you didn't answer the question: I was talking about a section of the 
draft that says "the poster MAY", and you told me that this means the the 
server MAY. Where did you get that nonsense?

>You may ask me to post something.  I may decline to accede to your
>request.  Those are both "MAY".

That is not what we are discussing. "MAY ASK" is very different from "MAY
USE". MAY ASK implies there is a question that you get to answer. MAY USE
means there is no question, the poster MAY USE that address.

>> A clearer statement of abdication is rarely seen here. The site will
>> do what it wants no matter what we say, so why bother saying
>> anything?

>To tell it what it ought to do so that interaction with other sites
>works properly.

A site will do what it wants no matter what we say. Why bother saying 
anything?

>> Where the fuck do we say that posters don't have to pay their bills?

>When you said that a poster MAY use ".invalid" and that the server is
>therefore REQUIRED to post the message.

Answer the question. Where does this draft say ANYTHING about paying 
bills? It doesn't. You made it up. Just like you made up the part about my 
saying that a server is REQUIRED to post a message. It would save both of 
us a lot of time if you would stop making things up and stick with what
was actually said.

>> Why do you keep making this shit up?

>It's a logical consequence of the rule that you're arguing for.

And now you've fabricated some rule that I am supposedly arguing for. Why 
do you keep doing this?

>>>If sites *have to* accept it, then they have to accept the articles
>>>where it is used, right? 
>
>> No, of course not. Don't be stupid.

>So now you agree that a site need not accept an article that has a
> From header ending in .invalid?

Stop being stupid. We are talking about a criterion for acceptance based 
on the use of .invalid here, and nothing else. I've said that a site 
cannot reject an article because the poster has used a .invalid address
because WE told the user he MAY use it. That says NOTHING AT ALL about any 
other part of the article, and you know it. You know it because I've 
already told you this. It was in the next paragraph that you even quoted.

If you really are so obtuse as to be honestly arguing about some fictional
world where I said that it MUST accept an article no matter what else the
article contained, then say so now and then be quiet. Nobody said that. I 
certainly did not. 

>Where do you think you get the right to decide on policies that sites
>must follow? 

This isn't site policy anymore. This draft tells posters they MAY do
something. I didn't write that part of the draft, so it isn't me you want
to argue with about this.

>"Articles must be syntactically valid according to these
>rules" is fine, because that affects interoperability.

Well, apparently, so does the use of .invalid. I'm not the one promoting
it. Go ask those who are why it is being pushed on users.

>> IF there were such terms presented to him for his acceptance, you might
>> have an argument. (There were no such terms in anything I've seen anyplace 
>> I've posted, so I expect such public presentation of such specific terms 
>> is rare.)

>They don't get that specific, but they all say that the ISP can do as
>it wishes.

You claimed that he agreed to specific terms that told him he must use a 
certain address when posting, and now you admit that none of them actually 
say that, they just say that the ISP can "do as it wishes". Well, that 
statement is wrong, too. The ISP is not free to do as it wishes; the ECPA 
is just one example that proves you wrong.

>> By telling the poster that he MAY do something, we've taken it out of
>> the realm of "site policy".

>I don't agree; but if you insist that's the case, we have to put it
>back.

No, we don't. We can leave that part as is, and correct the other parts 
which contradict it.

>> I'm not the one misinterpreting it. Their rule is "must use valid
>> email address in From header".

>Maybe their rule is "must use address you have verified to be valid in
>the From header."

Still trying to put words in my mouth, huh? Maybe their rule is what I 
wrote.

>If I try to login to your system without giving a password, your
>system doesn't know whether or not I'm the owner of the account.

Another nonsequitor.

>Without knowing that I'm not you, your system has no grounds for
>rejecting my login request.

Continued ad-nauseum.

>Except for people who want to post, where "verified" is the required
>case.

When verified is impossible, it cannot be the "required case". By saying 
it can be the "required case", you imply it IS always possible, and that 
is the error that needs to be rectified in the draft.

>But they may, if they so choose, reject an article for not having a
>From header that they do not know to be valid.

It is easy to check for a syntactically valid from header; thus is it
correct to imply that they can. It is impossible to verify, in most cases,
a contextually valid From header, thus it is incorrect to imply they can.
Our draft makes this implication; it needs to be fixed. To leave this 
stuff in will just result in the spread of kludges and hacks that try to 
accomplish the impossible, and users suffer for it in the long run, as 
well as developers. How much of this draft reads the way it does because
someone came up with some kludge that is now protected as "existing 
practice"? Too much. Why create more?

To: LIST: [email-masked];
Xref: clerew local.usefor:20533
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Thu, 30 Dec 2004 14:16:33 GMT
Lines: 36
Sender: [email-masked]

To: LIST: [email-masked];
Xref: clerew local.usefor:20534
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Thu, 30 Dec 2004 14:27:11 GMT
Lines: 43
Sender: [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: FWS problem
Date: Thu, 30 Dec 2004 04:49:05 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 18
Sender: [email-masked]

Stupid Boy wrote:

>| references  = "References:" SP [CFWS] msg-id-list [CFWS] CRLF
>| msg-id-list = msg-id-core *( WSP [CFWS WSP] msg-id-core )

Nonsense, that should be:

| references  = "References:" SP msg-id-list CRLF
| msg-id-list = [CFWS WSP] msg-id-core *( WSP [CFWS WSP] msg-id-core ) [WSP CFWS]

-or- (if a comment is good enough as separator)

| msg-id-list = [CFWS] msg-id-core *(CFWS msg-id-core ) [CFWS]

The latter allows (x)<y@z>(a)<b@c>(u)<v@w>(e) where the former
insists on (x) <y@z> (a) <b@c> (u) <v@w> (e).  These comments
almost everywhere are a pain, just like NO-WS-CTL.  Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: FWS problem
Date: Thu, 30 Dec 2004 04:19:00 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 135
Sender: [email-masked]

Charles Lindsey wrote:

 [NO-WS-CTL in utext]
> Indeed, but politically impossible to change it unilaterally.

Sooner or later you get me to believe in this IETF-2822-cabal,
and then I'll start unilaterally a war against it.  You found
a good reason to change the 2822 "unstructured" into a USEFOR
"unstructured".  It's already different, so why stick to the
utter dubious NO-WS-CTL ?  You could still add a "MAY accept"
and "SHOULD NOT generate" in the text, the polite form of "be
liberal".  [[ I'm only ranting, please leave utext as it is ]]

 [general [FWS] => *WSP]
> all that syntax works, but that is not to say that we should
> change it.

Why not ?  It reflects the MUST about non-empty characters, if
you don't like it, then something with this MUST is not okay.

> you have forbidden, for example:
> Supersedes:  (a comment)
>     <[email-masked]>
>     (another comment)

Yes, "Supersedes:" is an odd case, it's no 2822 header, and you
have already restricted it to one msg-id, s-o-1036 had a list.
Here are some alternatives:

 [your version]
| supersedes =  "Supersedes:" SP [CFWS] msg-id-core [CFWS] CRLF

 [simple version based on msg-id]
| supersedes = "Supersedes:" SP msg-id CRLF

 [s-o-1036 version with a msg-id-list also used in references]
| supersedes = "Supersedes:" SP msg-id-list CRLF

But IIRC we have a rough consensus for one and only one msg-id
in supersedes, and that kills any msg-id-list here.  Why do you
want comments in this header ?  After all it's something a news
server has to understand, and you eliminated CFWS in almost all
other headers relevant for servers.  Is that something about
relaying agent vs. injecting agent, or what's the idea ?

> you would need some syntax of the form:
>     WSPC = *WSP [ comment [CFWS] ]
>     CWSP = [ [CFWS] comment ] *WSP

Yes, that's the idea, but it's a syntactical PITA.  Apparently
it's better to leave all [CFWS] before and after the header
content alone, both your text and 2822 have an explicit MUST
about non-empty header lines.  With one minor and very obscure
difference, 2822 allows "name:" SP CRLF WSP stuff CRLF, and
you don't allow it, no matter how stuff begins (incl. comment).

 [msg-id-list in references]
> you just spotted another bug in USEFOR.

Maybe this works:

| references  = "References:" SP [CFWS] msg-id-list [CFWS] CRLF
| msg-id-list = msg-id-core *( WSP [CFWS WSP] msg-id-core )

 [Lines = "Lines:" SP *WSP 1*DIGIT *WSP CRLF]
> It is a fundamental principle of RFC 2822 that comments and
> folding are allowed in _all_ headers.

They do have FWS in their syntax where they needed it, e.g. a
Date: header has no CFWS before or within the date, only at the
end.  That cries for a compromise:

| Lines = "Lines:" SP *WSP 1*DIGIT [WSP CFWS] CRLF

Actually it's not that important if Lines: isn't a part of the
overview.  But if it is, then I'd want a trivial way to spot
very long articles before I download this stuff.  You allow a
"Lines:" SP CRLF WSP "(many" CRLF WSP "lines)9999(!)" CRLF

It's an old header only used in news, it has no comments today
(taking s-o-1036 as state of the art, the news.txt.z fraction
 please reads 1036 ;-), you deprecate it, so why add comments
to its grave ?

> there are no grounds to make an exception for Lines.

Now admit it, the big IETF-2822-cabal is in fact you... <gd&r>

> I am still not convinced that your syntax has forbidden
> folded lines containing only WSP. All you need is some
> situation where two CFWS can appear adjacent

That's then a problem of 2822, any you've convinced me that we
keep [CFWS] as is where necessary (not in Lines and Supersedes
IMHO).  They tried to avoid this CFWS problem in 2822 somehow:

| CFWS = *([FWS] comment) (([FWS] comment) / FWS)

A "comment" is never empty, it starts with "(".  So there's a
"(" after each folding, i.e. *WSP CRLF 1*WSP, and whatever
happens within a comment, it ends with a non-empty ")".  Even
if the syntax allows CFWS CFWS somewhere it works for comments.

It fails if you replace CFWS CFWS by FWS FWS, and then fold
both resulting in an illegal *WSP CRLF 1*WSP *WSP CRLF 1*WSP.

The discussed simple cases in "our" headers had no CFWS CFWS
and no FWS FWS, or I missed it.  If I missed it it's within the
header content, not at the begin or the end.

> there are plenty of places in RFC 2822 where that can happen

None of our business, or are you planning to edit a 2822bis ?

> Moreover, you have got now to deal with all those things we
> take from RFC 2822, such as <mailbox-list> in the From

No, you already said that you won't touch any 2822 constructs,
only if it's necessary, but not for cases like mailbox-list.

> you still need wording to forbid folded lines with WSP only

Sure, I wanted our syntax to _reflect_ our MUST where possible,
not to _replace_ it.  And it's not worth the effort for [CFWS],
if that means that we need a CWSP and a WSPC.  But for [FWS]
it's possible.

> your modified syntax has still not solved the problem it set
> out to solve, so what have you gained?

A correct syntax for all [FWS] cases.  But if you say that we
should also try CWSP and WSPC looking for cases of CFWS CFWS,
then let's do it.
                     Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Archive (was: Issues outstanding)
Date: Thu, 30 Dec 2004 01:34:09 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 24
Sender: [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: msg-id
Date: Thu, 30 Dec 2004 01:15:14 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 72
Sender: [email-masked]

 [unique-* stuff]

                    Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: msg-id
Date: Wed, 29 Dec 2004 23:56:25 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 58
Sender: [email-masked]

 

                           Bye, Frank

Date: Tue, 28 Dec 2004 16:14:38 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

John Stanley <[email-masked]> wrote:
> Seth Breidbart <sethb@xxxxxxxxx>:

>>No, I'm not.  I'm stating that in some cases it is possible to verify
>>some addresses. 
>
> Nobody is debating that. The problem comes in the next step, when 
> "possible to verify" gets turned upside down and becomes "not permitted
> to use".

If I can't verify you as the owner of that email address, then you
can't use my server to post with that email address.

> The same philosophy ought to hold: if you don't know, don't 
> pretend you do, and don't act based on the lack of info.

I'm not acting, I'm declining to act.

> "I can't verify this address, so I will act based on that lack of
> info..." is WRONG.

I'm acting based on the KNOWN FACT that you have failed to verify the
address you're attempting to use.

>>Or, perhaps, "now + 30 seconds" is when the posting will be made,
> Or perhaps "now + eternity" when the verification cannot be
>accomplished.

If you can't verify the address to me, then you can never use it on my
server.

> You keep assuming that verification is possible, and yet you just said 
> that you were talking about a few cases where it is.

I'm not assuming that it's possible, I'm stating that my policy
(that's my policy, I'm not requiring that anyone else adopt it)
requires that verification.

>>> And since they DO NOT KNOW, they should DO NOTHING SPECIAL.
>>That is, they shouldn't post the article?
>
> You think posting an article is the "special" case?

Yes, I think a lot more strings are not posted than are posted.

> Wow. And Wow, you think that MOST of the articles that are posted to
> USENET shouldn't be posted because the address cannot be verified?

No, I never claimed that.  I claimed that it's an acceptable policy
for a server to refuse to post articles if the From header can't be
verified to the satisfaction of that server.

It's equally acceptable for a server to allow any From address ending
in ".invalid" to be used.

> That's the result of "not posting unless verified" being the normal
> case.

I'm saying there is _no_ "normal case" but rather that each server can
have the policy its owner desires.

>>In most cases where the account actually belongs to the poster, it can.
>
> Wrong. Perhaps at Panix, where they pretend they can, they've convinced
> you they can. Not in the real world. The server I use cannot even verify
> the address the agents put in for me, much less anything else I might want
> to use. (If the server tried to verify the address it wouldn't accept
> anything I post, since the agents put in a bogus address BY ISP
> CONFIGURATION.) There are an infinite number of addresses it cannot
> verify.  I think that counts as a majority over the relatively few that
> Panix can.

Actually Panix can verify infinitely many addresses, too.  So now
let's consider reality, instead.

>>>>MAY means the _site_ is permitted to do something. 
>>> Not when we say the _poster_ MAY do something.
>>The poster MAY do something.  
>
> That's what I said. Where did you get "the _site_ is permitted"?

Are you claiming that the owner of a machine is required to run the
machine the way _you_ want instead of the way _he_ wants?

>>The site is not required to accept it.
>
> Yes, and that is the contradiction.  If the server doesn't accept
> it, then the poster MAY NOT do it. MAY isn't the same as MAY NOT.

You may ask me to post something.  I may decline to accede to your
request.  Those are both "MAY".

>>The site will tell him what it wants, no matter what we say.
>
> A clearer statement of abdication is rarely seen here. The site will
> do what it wants no matter what we say, so why bother saying
> anything?

To tell it what it ought to do so that interaction with other sites
works properly.

>>Or do you want to say that a site which says "I won't let you post
>>that article because you didn't pay your bill" is non-compliant?
>
> Where the fuck do we say that posters don't have to pay their bills?

When you said that a poster MAY use ".invalid" and that the server is
therefore REQUIRED to post the message.

> Why do you keep making this shit up?

It's a logical consequence of the rule that you're arguing for.

>>> And I think anyone who has seen crossposting wars knows that the
>>> likelyhood of a group appearing a second time is extremely high,
>>Even to
>>alt.mycatcreatedthisnewsgroup.asdjkl32587iodjklgu8904756o2i5hgoi9834ht?
>
> Yes. Followup agents are instructed to copy (oops, I mean "take") the
> content of the previous newsgroups header as the "default" destination
> for their new article. Unless the user notices the group and removes it
> (which in practice does not happen very much, at least until the 
> crossposting is pointed out) it will be in the next article.

Or there's a Followup-To header.

>>If sites *have to* accept it, then they have to accept the articles
>>where it is used, right? 
>
> No, of course not. Don't be stupid.

So now you agree that a site need not accept an article that has a
 From header ending in .invalid?

> The From header is just one part of the message. The discussion is
> about accepting or rejecting the message based on the content of the
> From header and nothing else. If the message has some other problem,
> then it is not being rejected because the poster used .invalid in
> the From header, now is it?

Where do you think you get the right to decide on policies that sites
must follow?  "Articles must be syntactically valid according to these
rules" is fine, because that affects interoperability.

>>Telling the user that he must follow the terms of service that he
>>agreed to when he signed up for the account is not harming him, 
>
> IF there were such terms presented to him for his acceptance, you might
> have an argument. (There were no such terms in anything I've seen anyplace 
> I've posted, so I expect such public presentation of such specific terms 
> is rare.)

They don't get that specific, but they all say that the ISP can do as
it wishes.

>>I'm not saying that it isn't a good thing.  I'm saying that you (or
>>we) can't control sites' policies, and we shouldn't pretend we can.
>
> By telling the poster that he MAY do something, we've taken it out of
> the realm of "site policy".

I don't agree; but if you insist that's the case, we have to put it
back.

> Otherwise, the ENTIRE standard is "site policy" and we have no
> business pretending we can control anything.

We "control" whether or not something is considered well-formed and
standards-compliant.

> I'm not the one misinterpreting it. Their rule is "must use valid
> email address in From header".

Maybe their rule is "must use address you have verified to be valid in
the From header."

>>The fact that they CAN validate some email addresses does not imply
>>that they know whenever an email address in invalid.
>
> Now you understand. So if they don't know it is invalid, they ought not
> to act as if they know it is. 

If I try to login to your system without giving a password, your
system doesn't know whether or not I'm the owner of the account.  So
why does it act as if it knows I'm not?

>>As I've said consistently, what they know is that the person
>>attempting to post has not validated that email address to them.
>
> And that is not the same as using an invalid address, and without
> knowing that, there are no grounds for rejecting the article.

Without knowing that I'm not you, your system has no grounds for
rejecting my login request.

> So, by rejecting the article, they are pretending to be able to know
> "invalid" when all they know is "unverified". "Unverified" is the
> normal case.

Except for people who want to post, where "verified" is the required
case.

> they should not reject an article for having
> an invalid From header when they do not know it is invalid.

But they may, if they so choose, reject an article for not having a
 From header that they do not know to be valid.

Seth

Date: Tue, 28 Dec 2004 12:39:30 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

Seth Breidbart <sethb@xxxxxxxxx>:

>No, I'm not.  I'm stating that in some cases it is possible to verify
>some addresses. 

Nobody is debating that. The problem comes in the next step, when 
"possible to verify" gets turned upside down and becomes "not permitted
to use". The same philosophy ought to hold: if you don't know, don't 
pretend you do, and don't act based on the lack of info. "I can't verify
this address, so I will act based on that lack of info..." is WRONG.

>Or, perhaps, "now + 30 seconds" is when the posting will be made,

Or perhaps "now + eternity" when the verification cannot be accomplished.
You keep assuming that verification is possible, and yet you just said 
that you were talking about a few cases where it is.

And just as an example, it will take much more than 30 seconds for the 
server to send email to my home system, for my home system to dial in and 
pick up my email, for me to drive home after work so I can read my home 
email, for me to reply, for the reply to get spooled back out, and then 
back to the server. If I'm travelling, it may be a week or more before I
get to my home email. Thirty seconds my ass. Stop pretending you know how 
everyone else's systems work.

And as another example, it will take much more than 30 seconds for the
server to send email to my home system and get any response back when the
address I am using is a spam-trap that goes into /dev/null. And yet, it is
a "valid address" (deliverable) in a valid domain that I am authorized to
use.

>> Right. Turn a ten second process for posting an article into a three
>> day adventure.
>  ^^^
>YM "minute"  HTH

No, ass, I meant what I wrote. If I meant "minute", I would have written
minute. But nice try at putting words in my mouth, again.

>> And since they DO NOT KNOW, they should DO NOTHING SPECIAL.

>That is, they shouldn't post the article?

You think posting an article is the "special" case? Wow. And Wow, you 
think that MOST of the articles that are posted to USENET shouldn't be
posted because the address cannot be verified? That's the result of
"not posting unless verified" being the normal case.

>In most cases where the account actually belongs to the poster, it can.

Wrong. Perhaps at Panix, where they pretend they can, they've convinced
you they can. Not in the real world. The server I use cannot even verify
the address the agents put in for me, much less anything else I might want
to use. (If the server tried to verify the address it wouldn't accept
anything I post, since the agents put in a bogus address BY ISP
CONFIGURATION.) There are an infinite number of addresses it cannot
verify.  I think that counts as a majority over the relatively few that
Panix can.

>>>MAY means the _site_ is permitted to do something. 
>
>> Not when we say the _poster_ MAY do something.

>The poster MAY do something.  

That's what I said. Where did you get "the _site_ is permitted"?

>The site is not required to accept it.

Yes, and that is the contradiction.  If the server doesn't accept it, then
the poster MAY NOT do it. MAY isn't the same as MAY NOT.

>The site will tell him what it wants, no matter what we say.

A clearer statement of abdication is rarely seen here. The site will do
what it wants no matter what we say, so why bother saying anything? If
your opinion is the consensus, and I don't see anyone else opposing it so
it must be, then to whom do we send the email to get the working group 
disbanded?

>Or do you want to say that a site which says "I won't let you post
>that article because you didn't pay your bill" is non-compliant?

Where the fuck do we say that posters don't have to pay their bills? Why 
do you keep making this shit up?  Where do you see anything in the draft
about paying the bill? Is it the same place you see crap about spammers
getting a "free ride"?

>> And I think anyone who has seen crossposting wars knows that the
>> likelyhood of a group appearing a second time is extremely high,

>Even to
>alt.mycatcreatedthisnewsgroup.asdjkl32587iodjklgu8904756o2i5hgoi9834ht?

Yes. Followup agents are instructed to copy (oops, I mean "take") the
content of the previous newsgroups header as the "default" destination
for their new article. Unless the user notices the group and removes it
(which in practice does not happen very much, at least until the 
crossposting is pointed out) it will be in the next article. That's 
"likely" in english. I don't know what it is in your language.

>If sites *have to* accept it, then they have to accept the articles
>where it is used, right? 

If you say so. I didn't say that, but if you think it should be, who am I
to argue?

No, of course not. Don't be stupid. The From header is just one part of 
the message. The discussion is about accepting or rejecting the message 
based on the content of the From header and nothing else. If the message
has some other problem, then it is not being rejected because the poster
used .invalid in the From header, now is it? 

Do you just not have any way of supporting your position, so you'll try to
claim I'm supporting spammers? That's cheap and dishonest.

>Telling the user that he must follow the terms of service that he
>agreed to when he signed up for the account is not harming him, 

IF there were such terms presented to him for his acceptance, you might
have an argument. (There were no such terms in anything I've seen anyplace 
I've posted, so I expect such public presentation of such specific terms 
is rare.) But there is still harm, even if the poster was presented with
such terms and did agree to them. Of course, if you don't pay for 
transport and processing of spam, you don't recognize that harm.

>I'm not saying that it isn't a good thing.  I'm saying that you (or
>we) can't control sites' policies, and we shouldn't pretend we can.

By telling the poster that he MAY do something, we've taken it out of
the realm of "site policy". Otherwise, the ENTIRE standard is "site 
policy" and we have no business pretending we can control anything. Let's
fire off that email to the IETF telling them we've dropped the idea of a 
news standard, ok?

>>>Because your interpretation of the rule they stated isn't the same as
>>>their interpretation of it.
>
>> It is simple english. It is pretty hard to misinterpret.

>Yet you managed to.  

I'm not the one misinterpreting it. Their rule is "must use valid email 
address in From header". When I use a valid address and they reject it,
that's a misinterpretation of their own rule. But they've been mislead 
into thinking they can determine this BECAUSE WE IMPLY TO THEM THEY CAN.

>Their rule is "Postings made through this server
>must carry the address assigned to you by this company in the From
>header."

That's your interpretation of the rule, but that's not "must use valid 
email address". That's just another example of your pretense that "valid" 
and "verified" are the same thing. Or perhaps the rule at Panix is what 
you list. It's not the rule anyplace I post from, and I've never seen it.
And unless I considered Panix to be a spam-trap address, I'd not accept
those terms -- if they were presented to me prior to my signing up. 

But that isn't even a rule that Panix publishes. I've just looked at all
the rules they list, and the rule you state is not one of them. There 
isn't even anything close. So, were I to sign up to Panix, I'd expect to
be able to use any of the nearly infinite number of valid addresses I have
control over. If they don't let you do that, then you need to talk to them
about it. You're paying them for service, they ought to provide it.

>The fact that they CAN validate some email addresses does not imply
>that they know whenever an email address in invalid.

Now you understand. So if they don't know it is invalid, they ought not
to act as if they know it is. 

>As I've said
>consistently, what they know is that the person attempting to post has
>not validated that email address to them.

And that is not the same as using an invalid address, and without knowing
that, there are no grounds for rejecting the article. So, by rejecting
the article, they are pretending to be able to know "invalid" when all 
they know is "unverified". "Unverified" is the normal case. It should not
merit special action -- and since you confused "special" and "posting" 
before, I'll make it clearer: they should not reject an article for having
an invalid From header when they do not know it is invalid.

To: LIST: [email-masked];
Xref: clerew local.usefor:20525
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: FWS problem
Date: Tue, 28 Dec 2004 14:13:33 GMT
Lines: 115
Sender: [email-masked]

>> USEFOR already says that we take only the syntax from
>> section 3 of RFC 2822, not that from section 4.

>Okay, then the FWS problem is reduced to 2*7 occurences
>in 7 productions of usefor-02, and about 2*7 [CFWS].  For
>[FWS] I get:

>1: unstructured    = *WSP 1*utext *([FWS] 1*utext) *WSP

>Very dubious, it's essentially any ASCII minus NUL, CR, or LF,
>but allowing CRLF.  And again all those funny NO-WS-CTL, sigh.

Indeed, but politically impossible to change it unilaterally.

>2: msg-id          = *WSP msg-id-core *WSP
>3: newsgroup-list  = *WSP newsgroup-name
>                     *( [FWS] "," [FWS] newsgroup-name ) *WSP
>4: path-list       = *WSP
>                     *( path-identity [FWS] path-delimiter [FWS] )
>                     tail-entry *WSP
>5: poster-text     = *WSP %d112.111.115.116.101.114 *WSP
>6: control         = "Control:" SP control-message CRLF
>   control-message = *WSP verb *( [FWS] argument ) *WSP

Move <control> down to your [CFWS] cases. I think that is a bug in USEFOR
which I have aksed Ken to fix.

>7: dist-list       = *WSP dist-name *( "," [FWS] distaname ) *WSP

>For newsgroup-list you allow [FWS] before and after the comma,
>for dist-list it's only after the comma, is this what you want ?

No, I think you just spotted another bug in USEFOR (draft-13 allowed the
[FWS] on either side of the ",").

Yes, all that syntax works, but that is not to say that we should change
it.

>And now the seven [CFWS] in "our" headers, defining "our" by
>"already not the same as in RfC 2822 for some other reasons"):

But your next lot of syntax does not work, because you have forbidden, for
example:

Supersedes:  (a comment)
    <[email-masked]>
    (another comment)

So you would need some syntax of the form:

    WSPC = *WSP [ comment [CFWS] ]
    CWSP = [ [CFWS] comment ] *WSP

>1: references      = "References:" SP msg-id-list CRLF
>   msg-id-list     = *WSP msg-id-core *([CFWS] msg-id-core) *WSP

>Is that really what we want, no separator or only a comment as
>separator is okay ?  s-o-1036 has apparently 1*WSP as separator.

No, you just spotted another bug in USEFOR.

>2: supersedes      = "Supersedes:" SP *WSP msg-id-core *WSP CRLF
>3: injection-info  = "Injection-Info:" SP *WSP path-identity
>                     *( [CFWS] ";" [CFWS] inj-info-para )
>                     *WSP CRLF

>Dubious.  I've added [CFWS] before and after the semicolon, your
>syntax allowed "Injection-Info:" SP CFWS path-identity CFWS CRLF
>among other things.

Yes, that is right. Optional CFWS is allowed on either side of any MIME
<parameter>, and that needs to be made clear in USEFOR.

>4: lines           = "Lines" ":" SP *WSP 1*DIGIT *WSP CRLF

>Who needs a comment in a deprecated header, let alone folding ?

It is a fundamental principle of RFC 2822 that comments and folding are
allowed in _all_ headers. We stuck out our necks rather in the case of
Newsgroups, Message-ID, etc where there are performance issues, but there
are no grounds to make an exception for Lines.

But I am still not convinced that your syntax has forbidden folded lines
containing only WSP. All you need is some situation where two CFWS can
appear adjacent, and you will get it. And there are plenty of places in
RFC 2822 where that can happen. Moreover, you have got now to deal with
all those things we take from RFC 2822, such as <mailbox-list> in the From
header. That has [CFWS] at either end, and you can't start rewriting all
the syntax inside RFC 2822. So you still need wording to forbid folded
lines with WSP only, and header content 1st lines with WSP only, so all
your modified syntax has still not solved the problem it set out to solve,
so what have you gained?

>P.S. and OT:  Sometimes something on your side adds ***SPAM-3***
>              to the subject, and I forget to remove it.

Yes, that is added as a warning by my Spamassassin filter when the score
is 3 (anything with a score of 4 or more goes straight to devnull).

To: LIST: [email-masked];
Xref: clerew local.usefor:20523
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Tue, 28 Dec 2004 12:24:42 GMT
Lines: 81
Sender: [email-masked]

and observe that
    <[email-masked]>
and <[email-masked]>

To: LIST: [email-masked];
Xref: clerew local.usefor:20524
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Tue, 28 Dec 2004 12:39:22 GMT
Lines: 63
Sender: [email-masked]

To: LIST: [email-masked];
Xref: clerew local.usefor:20522
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Issues outstanding
Date: Tue, 28 Dec 2004 11:01:16 GMT
Lines: 55
Sender: [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: msg-id
Date: Tue, 28 Dec 2004 02:32:55 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 92
Sender: [email-masked]

Charles Lindsey wrote:

  [unique vs. id-left]

                    Bye, Frank

Date: Mon, 27 Dec 2004 16:52:18 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

John Stanley <[email-masked]> wrote:
> Seth Breidbart <sethb@xxxxxxxxx>:

>>> Were that as far as it goes, fine. It isn't. The next step is the 
>>> pretense. "You may not use this address".
>
>>It's not a pretense, and it's saying "I won't let you use this address
>>_now_."
>
> pre-tense n.
>    1. a claim, esp. an unsupported one, as to some
>       distinction or accomplishment; pretension
>    2. a false claim or profession
>    3. a false show of something
>
> Yes, it certainly is a pretense,

Where is the pretense?  It is saying something that's absolutely
correct.

> and "_now_" is the only time that matters. NOW is when the posting
> is being made, and NOW is when the posting is most likely most
> relevant to anything.

Or, perhaps, "now + 30 seconds" is when the posting will be made,
after the poster verifies the ability to use that address.

>>> This statement demonstrates the farce. It might not be verified
>>> because it is impossible to verify.
>
>>It's certainly a reasonable policy to say "If you can't verify it to
>>me, then I won't let you use it."
>
> You are continuing the farce. You are pretending that it is possible,
> in the general case, to verify an arbitrary address.

No, I'm not.  I'm stating that in some cases it is possible to verify
some addresses.  I'm stating that it is a _reasonable_ policy (not the
only possible policy, merely one of many reasonable ones) to allow
posting only with verified addresses.

> That only leaves "can't verify", and thus it is a poor policy to say
> "you can't use an address you can't verify to me".

I don't understand that statement at all.

> It's like saying "you can withdraw your money from this bank on the
> days when the sun comes up in the west."

No, it's like saying "You can withdraw your money from this bank after
you've proven that it's your money."

> Part of the condition being required will NEVER happen, so the whole
> statement is a farce.

Why do you claim I can never verify that I have the right to use a
particular email address?

I've verified the ability to use the one I'm sending this from to
Panix.

I could easily verify it to google (I might even have done so in the
past).

>>They could send you a token and ask you to prove you received it.
>
> Right. Turn a ten second process for posting an article into a three
> day adventure.
  ^^^
YM "minute"  HTH

>>No, what they know is which ones they know to be valid, and which ones
>>they don't know to be valid. 
>
> And since they DO NOT KNOW, they should DO NOTHING SPECIAL.

That is, they shouldn't post the article?

>>Not knowing that something is valid is
>>not the same as knowing that it isn't valid.
>
> Not when the result is that it is treated as if it is not valid.

I don't think I want to deposit money in the bank you run.  You'll let
anybody have it unless there's proof they aren't me.

>>Do you know that I am wearing shoes right now? 
>
> I neither know nor care, nor do I intend on doing anything based on my
> lack of information. That's how news agents should act. They don't know,
> they ought to do nothing based on information they don't have.

There's no reason that every news agent in the world must act in the
way that you think they ought to.  Their policies are up to their
owners and administrators.

>>It _can_ make the determination "You have not demonstrated to me that
>>you own the account you wish to post as."
>
> Big F'ing deal. So what? In most cases, it simply cannot be done.

In some cases it can, in some it can't.  In most cases where the
account actually belongs to the poster, it can.

> Get over it. Stop pretending that the news agent is the arbiter of
> who you are and what addresses are valid for you.

I'm not pretending that it is.  I'm saying that it's the arbiter of
which addresses IT will let me use.

>>> MAY is a permissive word meaning they are permitted to do something. If 
>>> they are not permitted to do something, then we are wrong.
>
>>MAY means the _site_ is permitted to do something. 
>
> Not when we say the _poster_ MAY do something.

The poster MAY do something.  The site is not required to accept it.

> That is clearly and unambiguously a statement about what the POSTER
> may do. If we tell the poster he MAY do something, we have no
> business saying that the site can tell him otherwise.

The site will tell him what it wants, no matter what we say.

Or do you want to say that a site which says "I won't let you post
that article because you didn't pay your bill" is non-compliant?

>>> The draft makes no such fine distinction in tenses. "Likely to be 
>>> crossposted to" is open-ended. Even so, given the instructions we provide 
>>> for followup agents, and observed conditions on the net, a group that HAS
>>> BEEN crossposted to is very likely to be crossposted to again. Not just 
>>> "likely", VERY likely.
>>So you claim; but I can show examples otherwise.
>
> Unfortunately for your argument, this would be proving a negative, and 
> that you cannot do.

I've proven many negatives.

> The fact that it has not happened in a certain case does not prove
> that it is not likely to happen, only that it did not happen in that
> specific case.

Nor did I say otherwise.

> And I think anyone who has seen crossposting wars knows that the
> likelyhood of a group appearing a second time is extremely high,

Even to
alt.mycatcreatedthisnewsgroup.asdjkl32587iodjklgu8904756o2i5hgoi9834ht?

>>> Since I have already, the answer is yes, I am very likely to graduate. It 
>>> is a certainty.
>
>>Not in the language I speak.
>
> Well, this draft is in english, and that is the language _I_ speak, so 
> you'll just have to accept that, while it may not be true in whatever 
> language YOU speak, it is true in english. Something that has already 
> happened is "very likely" to happen. You cannot unring the bell.

I'm sorry that your language doesn't have tenses, but English does.

>>> We ought to tell them MUST, since we also tell users they MAY use them.
>
>>Sites don't MUST accept _anything_.  Are you saying that spammers
>>using .invalid should get a free pass?
>
> Please read what is actually written and stop making stupid things up. I 
> don't recall saying anything about spamming or "free pass", and if you can 
> find anything that I've said about either, please feel free to produce it, 
> or the apology for implying it was said.
>
> What I DID say, if you read the words, is that we tell POSTERS that they 
> MAY use .invalid. If we say they MAY use it, then we cannot honestly tell 
> sites that they don't have to accept it.

If sites *have to* accept it, then they have to accept the articles
where it is used, right?  But those articles might be spam or
otherwise objectionable (to the site's policy).

> It's a contradiction. 

No, it isn't.

> What sites accept as far as UCE or UBE or whatever has NOTHING to do
> with the address that appears in the From header, and you ought to
> know better.

Sites can have their own policies for what they'll accept in that
header.

>>> What is the global harm saying they must accept the standard way of 
>>> denoting an invalid address?
>
>>Preventing them from doing what they want which doesn't harm anyone
>>else.
>
> Excuse me? Telling the user, who WE have told MAY use an invalid address
> crafted in an obviously invalid way, that he must use a VALID and
> spammable address in his USENET postings is NOT harming him?

Telling the user that he must follow the terms of service that he
agreed to when he signed up for the account is not harming him, even
if you want a standard that's more lax.

> I'm happy for you that you don't get buckets of spam and don't have
> to pay for the transport of same, but in the real world quite a bit
> of time and money goes towards that problem, and not giving spammers
> a "free ride" by handing them spammable addresses is a Good Thing,
> and doing otherwise DOES cause them harm. That's the entire
> reasoning behind the .invalid address scheme.  It's a bit late to
> claim that it isn't a Good Thing.

I'm not saying that it isn't a good thing.  I'm saying that you (or
we) can't control sites' policies, and we shouldn't pretend we can.

>>"Here's the standard way of doing things.  Some sites don't allow you
>>to do that despite it being the standard way of doing it at sites that
>>do allow it."
>
> We just told them they may do it. Either we tell them they may do it or we 
> don't. 

As far as _we_ are concerned, they may.  Their site might feel
otherwise.

Are you saying that sites may not put a size limit on messages, or
otherwise constrain them more than we do?

>>> So, I use an address from one of my other domains. They say "no". And yet
>>> I'm using a valid and working address. I'm following the stated rule, and 
>>> yet, I am not permitted to post. Why is that?
>
>>Because your interpretation of the rule they stated isn't the same as
>>their interpretation of it.
>
> It is simple english. It is pretty hard to misinterpret.

Yet you managed to.  Their rule is "Postings made through this server
must carry the address assigned to you by this company in the From
header."

> They are misinterpreting it because the implication they have gotten
> from our draft is that they CAN, indeed, validate email addresses
> and thus CAN tell me that my address is invalid.

The fact that they CAN validate some email addresses does not imply
that they know whenever an email address in invalid.  As I've said
consistently, what they know is that the person attempting to post has
not validated that email address to them.

Seth

Date: Mon, 27 Dec 2004 13:07:37 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

Seth Breidbart <sethb@xxxxxxxxx>:

>> Were that as far as it goes, fine. It isn't. The next step is the 
>> pretense. "You may not use this address".

>It's not a pretense, and it's saying "I won't let you use this address
>_now_."

 pre-tense n.
    1. a claim, esp. an unsupported one, as to some
       distinction or accomplishment; pretension
    2. a false claim or profession
    3. a false show of something

Yes, it certainly is a pretense, and "_now_" is the only time that 
matters. NOW is when the posting is being made, and NOW is when the 
posting is most likely most relevant to anything.

>> This statement demonstrates the farce. It might not be verified
>> because it is impossible to verify.

>It's certainly a reasonable policy to say "If you can't verify it to
>me, then I won't let you use it."

You are continuing the farce. You are pretending that it is possible,
in the general case, to verify an arbitrary address. That only leaves
"can't verify", and thus it is a poor policy to say "you can't use an
address you can't verify to me". It's like saying "you can withdraw your
money from this bank on the days when the sun comes up in the west." Part
of the condition being required will NEVER happen, so the whole statement
is a farce.

>They could send you a token and ask you to prove you received it.

Right. Turn a ten second process for posting an article into a three day
adventure. 

>No, what they know is which ones they know to be valid, and which ones
>they don't know to be valid. 

And since they DO NOT KNOW, they should DO NOTHING SPECIAL.

>Not knowing that something is valid is
>not the same as knowing that it isn't valid.

Not when the result is that it is treated as if it is not valid.

>Do you know that I am wearing shoes right now? 

I neither know nor care, nor do I intend on doing anything based on my
lack of information. That's how news agents should act. They don't know,
they ought to do nothing based on information they don't have.

>It _can_ make the determination "You have not demonstrated to me that
>you own the account you wish to post as."

Big F'ing deal. So what? In most cases, it simply cannot be done. Get over
it. Stop pretending that the news agent is the arbiter of who you are and
what addresses are valid for you.

>> MAY is a permissive word meaning they are permitted to do something. If 
>> they are not permitted to do something, then we are wrong.

>MAY means the _site_ is permitted to do something. 

Not when we say the _poster_ MAY do something. That is clearly and 
unambiguously a statement about what the POSTER may do. If we tell the 
poster he MAY do something, we have no business saying that the site can 
tell him otherwise. Otherwise, there is no purpose for the standard and we 
should just stop right now.

>> The draft makes no such fine distinction in tenses. "Likely to be 
>> crossposted to" is open-ended. Even so, given the instructions we provide 
>> for followup agents, and observed conditions on the net, a group that HAS
>> BEEN crossposted to is very likely to be crossposted to again. Not just 
>> "likely", VERY likely.

>So you claim; but I can show examples otherwise.

Unfortunately for your argument, this would be proving a negative, and 
that you cannot do. The fact that it has not happened in a certain case 
does not prove that it is not likely to happen, only that it did not 
happen in that specific case. And I think anyone who has seen crossposting 
wars knows that the likelyhood of a group appearing a second time is 
extremely high, even if you consider the fact that it has already happened
as anything other than proof that it has already happened and already 
meets the "likely" standard. (Note that it is not "likely again", just
"likely", in our draft.)

>> Since I have already, the answer is yes, I am very likely to graduate. It 
>> is a certainty.

>Not in the language I speak.

Well, this draft is in english, and that is the language _I_ speak, so 
you'll just have to accept that, while it may not be true in whatever 
language YOU speak, it is true in english. Something that has already 
happened is "very likely" to happen. You cannot unring the bell.

>> We ought to tell them MUST, since we also tell users they MAY use them.

>Sites don't MUST accept _anything_.  Are you saying that spammers
>using .invalid should get a free pass?

Please read what is actually written and stop making stupid things up. I 
don't recall saying anything about spamming or "free pass", and if you can 
find anything that I've said about either, please feel free to produce it, 
or the apology for implying it was said.

What I DID say, if you read the words, is that we tell POSTERS that they 
MAY use .invalid. If we say they MAY use it, then we cannot honestly tell 
sites that they don't have to accept it. It's a contradiction. 

What sites accept as far as UCE or UBE or whatever has NOTHING to do with 
the address that appears in the From header, and you ought to know better.

>> What is the global harm saying they must accept the standard way of 
>> denoting an invalid address?

>Preventing them from doing what they want which doesn't harm anyone
>else.

Excuse me? Telling the user, who WE have told MAY use an invalid address
crafted in an obviously invalid way, that he must use a VALID and
spammable address in his USENET postings is NOT harming him? I'm happy for
you that you don't get buckets of spam and don't have to pay for the
transport of same, but in the real world quite a bit of time and money
goes towards that problem, and not giving spammers a "free ride" by
handing them spammable addresses is a Good Thing, and doing otherwise DOES
cause them harm. That's the entire reasoning behind the .invalid address 
scheme.  It's a bit late to claim that it isn't a Good Thing.

>"Here's the standard way of doing things.  Some sites don't allow you
>to do that despite it being the standard way of doing it at sites that
>do allow it."

We just told them they may do it. Either we tell them they may do it or we 
don't. 

>Or would you claim that sites are required to allow crossposts to 9
>newsgroups because that's the "standard way" to have a posting show up
>in those 9 newsgroups?

Does our draft explicitely say that people MAY crosspost to 9 groups? I 
don't think so. Were we to say that, then yes, I would claim that OUR 
telling people they MAY do something is binding on the servers which 
deal with that. That's why this is a "standard" and not just "hints".

>> So, I use an address from one of my other domains. They say "no". And yet
>> I'm using a valid and working address. I'm following the stated rule, and 
>> yet, I am not permitted to post. Why is that?

>Because your interpretation of the rule they stated isn't the same as
>their interpretation of it.

It is simple english. It is pretty hard to misinterpret. They are 
misinterpreting it because the implication they have gotten from 
our draft is that they CAN, indeed, validate email addresses and thus CAN
tell me that my address is invalid. Even with all of the "forget for a 
moment" clauses in the description of the situation. 

I want that error removed. This draft is the cause. It can be corrected.

To: LIST: [email-masked];
Xref: clerew local.usefor:20518
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Archive (was: Issues outstanding)
Date: Mon, 27 Dec 2004 12:40:19 GMT
Lines: 66
Sender: [email-masked]

To: LIST: [email-masked];
Xref: clerew local.usefor:20517
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Mon, 27 Dec 2004 12:22:39 GMT
Lines: 79
Sender: [email-masked]

Date: Sat, 25 Dec 2004 15:23:28 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

John Stanley <[email-masked]> wrote:
> Seth Breidbart <sethb@xxxxxxxxx>:

>>No, the implication is that the agent can _sometimes_ tell that an
>>email address is (known to be) valid, and sometimes cannot tell.
>
> Were that as far as it goes, fine. It isn't. The next step is the 
> pretense. "You may not use this address".

It's not a pretense, and it's saying "I won't let you use this address
_now_."

>>An unverified email address is one that has not been verified.  It
>>might not be verified because it's bogus, or it might not be verified
>>because nobody got around to verifying it.
>
> This statement demonstrates the farce. It might not be verified
> because it is impossible to verify.

It's certainly a reasonable policy to say "If you can't verify it to
me, then I won't let you use it."  Consider a bank; I walk in and ask
for some money from Seth Breidbart's account.  If I can verify to
their satisfaction that I am the Seth Breidbart who owns the account,
they let me have the money, else they don't.

Isn't that the policy google groups uses?

> The ISP through which I post has no idea what other domains I own,
> much less have access to email through, and in the domains I own, I
> can create addresses at whim. They have no way of verifying any of
> them.

They could send you a token and ask you to prove you received it.

> But we'll tell them they can pretend to know which 
> ones are invalid.

No, what they know is which ones they know to be valid, and which ones
they don't know to be valid.  Not knowing that something is valid is
not the same as knowing that it isn't valid.

Do you know that I am wearing shoes right now?  Do you know that I'm
not?  Would you stake your reputation on either one being true?

>>No, it is saying "I do not know this to be valid so _I_ won't let you
>>use it."
>
> Thus the pretense. If it cannot make the determination, it should do 
> nothing different. 

Nothing different from what?

It _can_ make the determination "You have not demonstrated to me that
you own the account you wish to post as."

>>> B. Not if we are going to tell users that they may use .invalid.
>>"MAY" means it's a matter of local policy.
> MAY is a permissive word meaning they are permitted to do something. If 
> they are not permitted to do something, then we are wrong.

MAY means the _site_ is permitted to do something.  It implies that
the site is permitted not to do it.  The users are subject to the
policy of the site.

>>The probability that it _will have been_ crossposted to is unity.  The
>>probability that it _will be_ crossposted to is less than unity.
>
> The draft makes no such fine distinction in tenses. "Likely to be 
> crossposted to" is open-ended. Even so, given the instructions we provide 
> for followup agents, and observed conditions on the net, a group that HAS
> BEEN crossposted to is very likely to be crossposted to again. Not just 
> "likely", VERY likely.

So you claim; but I can show examples otherwise.

>>No, it doesn't.  Crossposting to a group whose name was clearly
>>generated by a cat walking on the keyboard is not likely to be
>>repeated.
>
> Read USENET for awhile and see if actual practice doesn't contradict you.

Cats walking across keyboards is the best explanation for a lot of
Usenet that I've seen.

(It's too hard to train monkeys to type.)

>>Are you likely to graduate from high school?
> Since I have already, the answer is yes, I am very likely to graduate. It 
> is a certainty.

Not in the language I speak.

>>> SHOULD does not suffice. Regular people will see the RECOMMENDATION 
>>> synonym for SHOULD and configure their servers to reject .invalid 
>>> addresses, even though we explicitely tell people they MAY use them.
>>No, we tell sites they MAY accept them.
>
> We ought to tell them MUST, since we also tell users they MAY use them.

Sites don't MUST accept _anything_.  Are you saying that spammers
using .invalid should get a free pass?

>>What is the global harm if a site requires posts to have verifiable
>>local addresses?
>
> What is the global harm saying they must accept the standard way of 
> denoting an invalid address?

Preventing them from doing what they want which doesn't harm anyone
else.

>>"Here's the authorized way of doing this.  Sites will adopt their own
>>policies, which may be weaker or stronger."
>
> "Here's the standard way of doing things, but it might not be standard
> where you are because we didn't actually make it part of the requirements 
> for the standard." Foof.

"Here's the standard way of doing things.  Some sites don't allow you
to do that despite it being the standard way of doing it at sites that
do allow it."

Or would you claim that sites are required to allow crossposts to 9
newsgroups because that's the "standard way" to have a posting show up
in those 9 newsgroups?

> Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx>:

>>Yes, that was the idea, and as a consequence TLD .invalid did
>>not work on this server.  And what you've written in the draft
>>addresses it perfectly.
>
> Let's see. My ISP goes anal and tells me "you must use a valid and working
> email address in your postings." (Forget for the moment that THEY are
> inserting a non-working address in my articles for me at the present time.
> Forget for the moment that they are also outsourcing news to another
> company.) Ok. We'll even assume that they have some magic method of
> determining whether or not an "@peak.org" address is valid and working.  
. . .
> So, I use an address from one of my other domains. They say "no". And yet
> I'm using a valid and working address. I'm following the stated rule, and 
> yet, I am not permitted to post. Why is that?

Because your interpretation of the rule they stated isn't the same as
their interpretation of it.

Seth

Date: Fri, 24 Dec 2004 11:58:47 +0000
From: Alexey Melnikov <[email-masked]>
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: Is this a STraw Poll?
Sender: [email-masked]

Charles Lindsey wrote:

>In <[email-masked]> Alexey Melnikov <[email-masked]> writes:
>  
>
>>Charles Lindsey wrote:
>>    
>>
>>>In <[email-masked]> Russ Allbery <[email-masked]> writes:
>>>      
>>>
>>>>Charles Lindsey <[email-masked]> writes:
>>>>        
>>>>
>>>>>Alexey Melnikov <[email-masked]> writes:
>>>>>          
>>>>>
>>>>>>This WG has enough work to do already, so I am Ok with defining
>>>>>>complaints-url in a separate document. However the syntax for
>>>>>>Injection-Info (or whatever we end up using) should allow for multiple
>>>>>>URLs.
>>>>>>       
>>>>>>            
>>>>>>
>>>>>  mail-complaints-param = "mail-complaints-to" "=" address-list
>>>>>          
>>>>>
>>My question is: do we *want* to allow for multiple complaints-to email 
>>addresses?
>>    
>>
>
>The syntax I gave already allows for multiple complaints-to email
>addresses, because it contains an address-list (the Complaints-To header
>likewise). OK, we have to make it clear that it means "send your complaint
>to one of these addresses", and not "send it to all of them", and that is
>indeed not quite the usual interpretation of <address-list>. So first, let
>us be clear which of those interpretations you have in mind, and then we
>can ask whether <address-list> is the proper syntax to do it.
>  
>
I think that only "send to one of those addresses" interpretation makes 
sense.
However, I can't even see a practical need for multiple addresses. Can 
somebody give an example?

>If a future document introduces a complaints-url (whatever) parameter, as
>you have suggested, then it will be up to that document to make provision
>for multiple URLs.
>  
>
Of course.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Archive (was: Issues outstanding)
Date: Thu, 23 Dec 2004 22:55:17 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 42
Sender: [email-masked]

Charles Lindsey wrote:

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: msg-id
Date: Thu, 23 Dec 2004 22:11:57 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 53
Sender: [email-masked]

 [unique]

                           Bye, Frank

To: LIST: [email-masked];
Xref: clerew local.usefor:20506
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Is this a STraw Poll?
Date: Thu, 23 Dec 2004 18:57:35 GMT
Lines: 46
Sender: [email-masked]

>>In <[email-masked]> Russ Allbery <[email-masked]> writes:
>>  
>>
>>>Charles Lindsey <[email-masked]> writes:
>>>    
>>>
>>>>Alexey Melnikov <[email-masked]> writes:
>>>>      
>>>>
>>>>>This WG has enough work to do already, so I am Ok with defining
>>>>>complaints-url in a separate document. However the syntax for
>>>>>Injection-Info (or whatever we end up using) should allow for multiple
>>>>>URLs.
>>>>>        
>>>>
>>>>   mail-complaints-param = "mail-complaints-to" "=" address-list
>>
>My question is: do we *want* to allow for multiple complaints-to email 
>addresses?

The syntax I gave already allows for multiple complaints-to email
addresses, because it contains an address-list (the Complaints-To header
likewise). OK, we have to make it clear that it means "send your complaint
to one of these addresses", and not "send it to all of them", and that is
indeed not quite the usual interpretation of <address-list>. So first, let
us be clear which of those interpretations you have in mind, and then we
can ask whether <address-list> is the proper syntax to do it.

If a future document introduces a complaints-url (whatever) parameter, as
you have suggested, then it will be up to that document to make provision
for multiple URLs.

To: LIST: [email-masked];
Xref: clerew local.usefor:20509
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id (was: Issues outstanding)
Date: Thu, 23 Dec 2004 19:25:29 GMT
Lines: 50
Sender: [email-masked]

To: LIST: [email-masked];
Xref: clerew local.usefor:20508
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Issues outstanding
Date: Thu, 23 Dec 2004 19:15:55 GMT
Lines: 47
Sender: [email-masked]

To: LIST: [email-masked];
Xref: clerew local.usefor:20507
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Issues outstanding
Date: Thu, 23 Dec 2004 19:08:31 GMT
Lines: 40
Sender: [email-masked]

Date: Thu, 23 Dec 2004 14:29:24 -0500 (EST)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Issues outstanding
Sender: [email-masked]

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: FWS problem
Date: Thu, 23 Dec 2004 14:32:06 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 65
Sender: [email-masked]

Charles Lindsey wrote:

> USEFOR already says that we take only the syntax from
> section 3 of RFC 2822, not that from section 4.

Okay, then the FWS problem is reduced to 2*7 occurences
in 7 productions of usefor-02, and about 2*7 [CFWS].  For
[FWS] I get:

1: unstructured    = *WSP 1*utext *([FWS] 1*utext) *WSP

Very dubious, it's essentially any ASCII minus NUL, CR, or LF,
but allowing CRLF.  And again all those funny NO-WS-CTL, sigh.

Is "Subject:" SP ESC CRLF TAB NAK CRLF SP SUB CRLF really what
we want ?  A SUB is an EOF on some systems, and my text editor
and some other tools still handle it as EOF unless I want them
to ignore it.

2: msg-id          = *WSP msg-id-core *WSP
3: newsgroup-list  = *WSP newsgroup-name
                     *( [FWS] "," [FWS] newsgroup-name ) *WSP
4: path-list       = *WSP
                     *( path-identity [FWS] path-delimiter [FWS] )
                     tail-entry *WSP
5: poster-text     = *WSP %d112.111.115.116.101.114 *WSP
6: control         = "Control:" SP control-message CRLF
   control-message = *WSP verb *( [FWS] argument ) *WSP
7: dist-list       = *WSP dist-name *( "," [FWS] distaname ) *WSP

For newsgroup-list you allow [FWS] before and after the comma,
for dist-list it's only after the comma, is this what you want ?

And now the seven [CFWS] in "our" headers, defining "our" by
"already not the same as in RfC 2822 for some other reasons"):

1: references      = "References:" SP msg-id-list CRLF
   msg-id-list     = *WSP msg-id-core *([CFWS] msg-id-core) *WSP

Is that really what we want, no separator or only a comment as
separator is okay ?  s-o-1036 has apparently 1*WSP as separator.

2: supersedes      = "Supersedes:" SP *WSP msg-id-core *WSP CRLF
3: injection-info  = "Injection-Info:" SP *WSP path-identity
                     *( [CFWS] ";" [CFWS] inj-info-para )
                     *WSP CRLF

Dubious.  I've added [CFWS] before and after the semicolon, your
syntax allowed "Injection-Info:" SP CFWS path-identity CFWS CRLF
among other things.

4: lines           = "Lines" ":" SP *WSP 1*DIGIT *WSP CRLF

Who needs a comment in a deprecated header, let alone folding ?

The remaining three [CFWS] cases:  xref, archive, and user-agent.
Especially the latter needs any "CWSP" it can get, but no FWS
before or after its header content.  TBD if you agree so far
with the 7+3 simple cases, and maybe the dubious Injection-Info.

                           Bye. Frank

P.S. and OT:  Sometimes something on your side adds ***SPAM-3***
              to the subject, and I forget to remove it.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: msg-id
Date: Thu, 23 Dec 2004 13:27:15 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 19
Sender: [email-masked]

                    Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: msg-id (was: Issues outstanding)
Date: Thu, 23 Dec 2004 10:58:30 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 38
Sender: [email-masked]

  [Charles said:]

                      Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: Issues outstanding
Date: Thu, 23 Dec 2004 09:51:51 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 17
Sender: [email-masked]

                       Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: Issues outstanding
Date: Thu, 23 Dec 2004 09:22:30 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 28
Sender: [email-masked]

                            Bye, Frank

Date: Thu, 23 Dec 2004 02:46:31 -0500 (EST)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Issues outstanding
Sender: [email-masked]

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: Issues outstanding
Date: Thu, 23 Dec 2004 08:14:08 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 65
Sender: [email-masked]

                        Bye, Frank

Date: Thu, 23 Dec 2004 00:01:55 +0100
From: Dirk Nimmich <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: Issues outstanding
Sender: [email-masked]

[...]

To: LIST: [email-masked];
Xref: clerew local.usefor:20496
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Issues outstanding
Date: Wed, 22 Dec 2004 19:04:31 GMT
Lines: 51
Sender: [email-masked]

Date: Wed, 22 Dec 2004 17:33:31 +0000
From: Alexey Melnikov <[email-masked]>
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: Issues outstanding
Sender: [email-masked]

I agree.

[...]

Date: Wed, 22 Dec 2004 17:27:41 +0000
From: Alexey Melnikov <[email-masked]>
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: Is this a STraw Poll?
Sender: [email-masked]

Charles Lindsey wrote:

>In <[email-masked]> Russ Allbery <[email-masked]> writes:
>  
>
>>Charles Lindsey <[email-masked]> writes:
>>    
>>
>>>Alexey Melnikov <[email-masked]> writes:
>>>      
>>>
>>>>This WG has enough work to do already, so I am Ok with defining
>>>>complaints-url in a separate document. However the syntax for
>>>>Injection-Info (or whatever we end up using) should allow for multiple
>>>>URLs.
>>>>        
>>>>
>>>OK, so it seems that a mail-complaints-to parameter in Injection-Info is
>>>the front runner. So we would need, in USEFOR:
>>>      
>>>
>>>   inj-info-param  =  post-host-param /
>>>                      post-account-param /
>>>                      sender-param /
>>>                      logging-param /
>>>                      mail-complaints-param
>>>      
>>>
>>>   mail-complaints-param = "mail-complaints-to" "=" address-list
>>>      
>>>
>>That looks like it's allowing multiple addresses, not multiple URLs.
>>    
>>
>
>Yes, that was the intention. The proposal, as already discussed, was to
>have separate parameters for the pure email case (mail-complaints-to=) and
>for the url case (complaints-url= or somesuch). Our chair proposed (see
>above) leaving complaints-url to a future extension, and that gave me the
>opportunity to write up the mail-complaints-to properly.
>  
>
My question is: do we *want* to allow for multiple complaints-to email 
addresses?

To: LIST: [email-masked];
Xref: clerew local.usefor:20494
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Is this a STraw Poll?
Date: Wed, 22 Dec 2004 14:24:06 GMT
Lines: 39
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Alexey Melnikov <[email-masked]> writes:

>>> This WG has enough work to do already, so I am Ok with defining
>>> complaints-url in a separate document. However the syntax for
>>> Injection-Info (or whatever we end up using) should allow for multiple
>>> URLs.

>> OK, so it seems that a mail-complaints-to parameter in Injection-Info is
>> the front runner. So we would need, in USEFOR:

>>    inj-info-param  =  post-host-param /
>>                       post-account-param /
>>                       sender-param /
>>                       logging-param /
>>                       mail-complaints-param

>>    mail-complaints-param = "mail-complaints-to" "=" address-list

>That looks like it's allowing multiple addresses, not multiple URLs.

Yes, that was the intention. The proposal, as already discussed, was to
have separate parameters for the pure email case (mail-complaints-to=) and
for the url case (complaints-url= or somesuch). Our chair proposed (see
above) leaving complaints-url to a future extension, and that gave me the
opportunity to write up the mail-complaints-to properly.

To: LIST: [email-masked];
Xref: clerew local.usefor:20495
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Issues outstanding
Date: Wed, 22 Dec 2004 14:44:00 GMT
Lines: 62
Sender: [email-masked]

Date: Wed, 22 Dec 2004 16:49:14 +0000
From: Alexey Melnikov <[email-masked]>
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: Issues outstanding
Sender: [email-masked]

Of course.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: Issues outstanding
Date: Tue, 21 Dec 2004 22:47:12 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 29
Sender: [email-masked]

                       Bye, Frank

Date: Tue, 21 Dec 2004 19:27:29 +0000
From: Alexey Melnikov <[email-masked]>
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: Issues outstanding
Sender: [email-masked]

7. FWS issue in headers.

Alexey

To: [email-masked]
Subject: Re: Is this a STraw Poll?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 21 Dec 2004 09:20:19 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Alexey Melnikov <[email-masked]> writes:

>> This WG has enough work to do already, so I am Ok with defining
>> complaints-url in a separate document. However the syntax for
>> Injection-Info (or whatever we end up using) should allow for multiple
>> URLs.

> OK, so it seems that a mail-complaints-to parameter in Injection-Info is
> the front runner. So we would need, in USEFOR:

>    inj-info-param  =  post-host-param /
>                       post-account-param /
>                       sender-param /
>                       logging-param /
>                       mail-complaints-param

>    mail-complaints-param = "mail-complaints-to" "=" address-list

That looks like it's allowing multiple addresses, not multiple URLs.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: LIST: [email-masked];
Xref: clerew local.usefor:20488
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Is this a STraw Poll? (was Standardize Complaints-To as deployed)
Date: Tue, 21 Dec 2004 13:48:52 GMT
Lines: 77
Sender: [email-masked]

>>>>>As a chair I would note that I see no consensus to use mail-complaint-to 
>>>>>over complaints-url.
>>>>>        
>>>>>
>>
>>I think the question is whether to provide both of them initially.
>>
>This WG has enough work to do already, so I am Ok with defining 
>complaints-url in a separate document. However the syntax for 
>Injection-Info (or whatever we end up using) should allow for multiple 
>URLs.

OK, so it seems that a mail-complaints-to parameter in Injection-Info is
the front runner. So we would need, in USEFOR:

   inj-info-param  =  post-host-param /
                      post-account-param /
                      sender-param /
                      logging-param /
                      mail-complaints-param

   mail-complaints-param = "mail-complaints-to" "=" address-list

and then:

   The specified <address>(es) is for sending complaints concerning the
   behaviour of the poster of the article; it SHOULD NOT be used for
   matters concerning propagation, protocol problems, etc. which should be
   addressed to "usenet@" or "news@" the <path-identity> (see [RFC 2142]).
   In the absence of this parameter, complaints concerning a poster's
   behaviour MAY be addressed to "abuse@" that <path-identity> (again, see
   [RFC 2142], though that address may not be available at injecting
   agents not provided for the use of the general public).

If we are agreed on that, then we can ask Ken to incorporate it into
USEFOR.

[Note that some related texts from draft-13 relating to mailability of
<path-identity>s have not yet made their way into USEFOR.]

The only other alternative remaining on the table is the Complaints-To
header as in draft-13. I received some mail from Alexey about this which I
am reproducing here, because it is part of the same discussion:

>....Charles Lindsey wrote:
 
>>Incidentally, one of the advantages of a Complaints-To header over a
>>parameter of Injection-Info is that it would be suitable for use in
>>Emails, if the email community wants to use it. 
  
>Can you elaborate on potential usage scenarios for email?

The situations I had in mind were where some ISP operated an email
submission agent, and had an AUP which forbade harrassment, spamming, and
other such evils. So it might add a Complaints-To header to emails passing
through it. Or else it might be added by a mailing list expander which had
rules about what could be posted to the list.

It is not for us to say whether that header should be used for email or
not, but it would clearly be suitable, whereas the Injection-Info header
is no way suited to email. Whether this factor should sway our decision on
the matter is for the WG to decide.

To: LIST: [email-masked];
Xref: clerew local.usefor:20490
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ***SPAM-3*** Re: FWS problem
Date: Tue, 21 Dec 2004 14:08:18 GMT
Lines: 61
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:
> 
>>|         approved = "Approved:" SP mailbox-list CRLF

>> You cannot reformulate that in your Approved: 1*WSP body CRLF
>> style because <mailbox-list> is defined in RFC 2822, where it
>> is already defined to start with '[CFWS]'.

>If you're free to replace [CFWS} by [FWS] because we don't want
>comments, then you're also free to replace [CFWS] by *WSP where
>both 2822 and we don't want a CRLF.  RfC 2822 didn't bother to
>get the sytax right, they only mentioned it in the text:

If you start to tinker with the RFC 2822 syntax to that extent, then you
will find that there is a considerable amount of it which would have to be
changed, and I think that is way beyond our remit. Hopefully the RFC 2822
situation can be improved if/when RFC 2822 comes up for revision, but we
have to work with it as it is.

>> We can't do anything about that without rewriting RFC 2822
>> syntax, and our Chair has forbidden that

>IIRC Alexej only said that whatever we do, the result must be
>a valid message/rfc822 as defined in 2822.  But not all valid
>message/rfc822 are also valid news articles.  If we have some
>restrictions, and that's the case, then it's perfectly okay to
>have a proper syntax reflecting these restrictions.

Well if Alexey were to say we could make these changes, then that would be
a different matter. But as it stands, with these particular cases covered
by verbiage, both in RFC 2822 and in USEFOR, nothing is actually broken
(apart from the definition of "header content", which needs to be worked
on).

>For those who don't get the idea by looking at the ABNF there
>is still the text with explicit MUSTs:  The header content of
>all header lines MUST contain a non-witespace (=> obs-FWS RIP).

>Back to your example:

>| mailbox-list = (mailbox *("," mailbox))

>BTW, please add the sytax again, as it was in old drafts, you
>probably don't want the obs-mbox-list in RfC 2822.

No, that is OK. USEFOR already says that we take only the syntax from
section 3 of RFC 2822, not that from section 4.

To: LIST: [email-masked];
Xref: clerew local.usefor:20489
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Tue, 21 Dec 2004 13:57:56 GMT
Lines: 55
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:
> 
>> I don't think there is any reason to forbid "reimport to
>> Usenet" as such (for example, it may come back into Usenet
>> in a different group and with a different message-id).
>.......................^^^

>>> you -> A - B - C -> me
>>>        |   |
>>>        a   b
>>>        |   |
>>>        other

>>> I never see some of your articles in group ABC.test
>[...]
>> you have not explained to me why it never got to server C

>"a" exported ABC.test to B.other in the other net,  Then "b"
>reimported B.other to B.local on server B.  C doesn't get the
>article in ABC.test.  The "and" in your statement (see above)
>is important.

Ah! So it was gatewayed back into a different newsgroup. In that case, a
new message identifier should have been provided somewhere along the line,
probably by b. Our draft does cover this (or something pretty close to it)
by the following text:

   Exceptionally, if there are multiple incoming gateways for a
   particular set of messages, each to a different newsgroup(s), each
   one SHOULD generate a message identifier unique to that gateway. Each
   incoming gateway nonetheless MUST ensure that it does not gate the
   same message twice.

        NOTE: Consider the example of two gateways of a given mailing
        list into the world-wide Usenet newsgroups, both of which
        preserve the email message identifier. Each newsgroup may then
        receive a portion of the messages (different sites seeing
        different portions).  In these cases, where there is no one
        "official" gateway, some other method of generating message
        identifiers has to be used to avoid collisions. It would
        obviously be preferable for there to be only one gateway which
        crossposts, but this may not be possible to coordinate.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: ***SPAM-3*** Re: FWS problem
Date: Tue, 21 Dec 2004 02:25:12 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 88
Sender: [email-masked]

Charles Lindsey wrote:
 
> I think somebody reported that there was software out there
> that barfed on "name: TAB body CRLF"

Okay, Henry already convinced me that s-o-1036 has a mandatory
SP in the text, not reflected in the syntax.

  [erroneous FWS] 
>> Let's fix it at least for our headers, replace [FWS] by *WSP
>> before and after any header content of "our" headers.

> No, we can't even do that. Consider the Approved header, for
> example:
>|         approved = "Approved:" SP mailbox-list CRLF

> You cannot reformulate that in your Approved: 1*WSP body CRLF
> style because <mailbox-list> is defined in RFC 2822, where it
> is already defined to start with '[CFWS]'.

If you're free to replace [CFWS} by [FWS] because we don't want
comments, then you're also free to replace [CFWS] by *WSP where
both 2822 and we don't want a CRLF.  RfC 2822 didn't bother to
get the sytax right, they only mentioned it in the text:

| However, where CFWS occurs in this standard, it MUST NOT be
| inserted in such a way that any line of a folded header field
| is made up entirely of WSP characters and nothing else.

CFWS minus C is FWS, and FWS minus CRLF is 1*WSP, or a case of
obs-FWS.  We don't allow obs-FWS, because that would be again a
line of folded header content without non-whitespace.

In other words, we want *WSP instead of [CFWS] before and after
the header content.  I don't see a problem with this approach.
It reflects 2822, minus obs-FWS, and minus the C in CFWS, these
restrictions are already a MUST in your draft.

> We can't do anything about that without rewriting RFC 2822
> syntax, and our Chair has forbidden that

IIRC Alexej only said that whatever we do, the result must be
a valid message/rfc822 as defined in 2822.  But not all valid
message/rfc822 are also valid news articles.  If we have some
restrictions, and that's the case, then it's perfectly okay to
have a proper syntax reflecting these restrictions.

For those who don't get the idea by looking at the ABNF there
is still the text with explicit MUSTs:  The header content of
all header lines MUST contain a non-witespace (=> obs-FWS RIP).

Back to your example:

| mailbox-list = (mailbox *("," mailbox))

BTW, please add the sytax again, as it was in old drafts, you
probably don't want the obs-mbox-list in RfC 2822.

| mailbox      = name-addr / addr-spec
| name-addr    = [display-name] angle-addr
| display-name = phrase
| phrase       = 1*word

Dito no obs-phrase here, please.

| word         = atom / quoted-string
| atom         = [CFWS] 1*atext [CFWS]

Okay, I see the general problem, to get it right you'd have to
move all [CFWS] from the bottom of the 2822 syntax to the top,
and then replace all [CFWS] before and after anything else by
*WSP, keeping only [CFWS] between something that's not empty.

That should be possible, but manually without a tool the result
would be probably worse than any dubious [CFWS] in 2822... :-(

> even draft-13 took the syntax of mailbox-list from RFC 2822
> without question

No obs-stuff in draft 7, where did you lose this improvement ?

Maybe it's impossible to fix the [CFWS] before and after the
mailbox-list, but there are still many explicit [FWS] in your
draft which can be fixed without touching 2822 terms. e.g. in
newsgroup-list, path-list, and poster-text.

                           Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Tue, 21 Dec 2004 01:04:36 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 33
Sender: [email-masked]

Charles Lindsey wrote:
 
> I don't think there is any reason to forbid "reimport to
> Usenet" as such (for example, it may come back into Usenet
> in a different group and with a different message-id).
.......................^^^

>> you -> A - B - C -> me
>>        |   |
>>        a   b
>>        |   |
>>        other

>> I never see some of your articles in group ABC.test
[...]
> you have not explained to me why it never got to server C

"a" exported ABC.test to B.other in the other net,  Then "b"
reimported B.other to B.local on server B.  C doesn't get the
article in ABC.test.  The "and" in your statement (see above)
is important.
  
What "b" could do is to create a new Message-ID, quite the
opposite of your idea to preserve Message-IDs in the body.

> Have you ever met a gateway admin who would admit that he
> "didn't know what he was doing" :-)

Not in the former German "gateways" and "gatebau" groups, all
GiGo users were legend, and dead gateway admins don't post :-)

                        Bye, Frank

To: LIST: [email-masked];
Xref: clerew local.usefor:20483
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ***SPAM-3*** Re: FWS problem
Date: Mon, 20 Dec 2004 14:53:46 GMT
Lines: 64
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>You can always convince me with this line.  But s-o-1036 says:

>| header          = start-line *continuation
>| start-line      = header-name ":" space [ nonblank-text ] eol
>[...]
>| space           = 1*( <HT (ASCII 9)> / <blank (ASCII 32)> )

>I don't see any mandatory SP in this syntax, I only see a WSP.

I think somebody reported that there was software out there that barfed on
"name: TAB body CRLF", which is why we currently insist on SP.

>Are you absolutely sure that we want to go back to the "blank"
>in 1036 ?  It's the old problem of "does RfC 2822 update 1036
>in some way ?"  IMHO a "name: 1*WSP body CRLF" is better than
>your "name: SP *WSP body CRLF".

>>> should be "name: SP *WSP header content CRLF" everywhere.

>> Maybe for our own news headers, but in some cases the syntax
>> for the [*WSP header] bit comes straight from RFC 2822, and
>> so it is not ours to change.

>Let's fix it at least for our headers, replace [FWS] by *WSP
>before and after any header content of "our" headers.

No, we can't even do that. Consider the Approved header, for example:

   approved        =  "Approved:" SP mailbox-list CRLF

You cannot reformulate that in your
    Approved: 1*WSP body CRLF
style because <mailbox-list> is defined in RFC 2822, where it is already
defined to start with '[CFWS]'. We can't do anything about that without
rewriting RFC 2822 syntax, and our Chair has forbidden that (and I think
everybody here accepts that - even draft-13 took the syntax of
mailbox-list from RFC 2822 without question).

>>> The syntax "name: SP [FWS] header content CRLF" is wrong,
>>> because "name: SP CRLF SP header content CRLF" is illegal.
> 
>> I don't think this prohibition can be made by syntax. The
>> corresponding provision in RFC 2822 (the one against empty
>> folded lines) was not made by syntax, and we are following
>> RFC 2822 wherever possible as a matter of policy.

>We're following the _intent_ of RfC 2822 whereever possible,
>but you're not forced to copy inaccurate syntax.

Yes we are, unless the RFC 2822 syntax is irretrievably broken, which it
isn't.

To: LIST: [email-masked];
Xref: clerew local.usefor:20484
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Mon, 20 Dec 2004 15:12:47 GMT
Lines: 54
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Let's just find a minimal _working_ model for Message-IDs.  In
>RfC 2822 you were not interested in these details, you only
>wanted some simple syntax covering wild and wonderful things
>like mail, s-o-1036, and X.400.

That idea is very appealing, but it is not what we have been instructed to
do, which is to follow RFC 2822 wherever possible. Now if there were
several more members of this WG in addition to yourself campaigning for a
more radical approach, then it might be considered; but there are not.

>Here and today we know it better, we want something working in
>Usenet, in news URLs, in cancel messages, and as many existing
>UAs as possible.  IMHO that disqualifies <whatever@invalid>,
>and it also doesn't allow NO-WS-CTL.

Now NO-WS-CTL is an interesting case, because I have just spotted the
following in the new NNTP draft:

   o  A message-id MUST NOT contain octets other than printable US-ASCII
      characters.

and I would not like to be proposing something that could not be
transported by NNTP.

So, since Russ is the Chair of the NNTP WG, could he please speak up on
this one? I think we would all agree that NO-WS-CTLs in message-ids are
ugly, but they are allowed in RFC 2822.

>RfC 2822 is too broad, they wanted a "working model" covering
>obsolete and broken software for mail, where Message-IDs aren't
>essential.  I'm only guessing why RfC 2822 got it wrong, you
>were there:  Why did they replace the old concept of "like an
>addr-spec, only unique" by a new "id-left @ id-right" ?

Because both <local-part> and <domain> have built-in [CFWS] before and
after them (and even in the middle), and the last thing we wanted (even
for email) was comments and folding in the middle of a message identifier.

Essentially, <id-left> and <id-right) are just <local-part> and <domain>
with all CFWS possibilities excluded.

To: LIST: [email-masked];
Xref: clerew local.usefor:20482
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Mon, 20 Dec 2004 14:36:31 GMT
Lines: 68
Sender: [email-masked]

>> The best means for preventing loops is to ensure that
>> Message-IDs are propagated correctly.

>Unfortunately that's not the case.  Loops cause two kinds of
>problems:  dupes and nopes.  Saving the Message-ID is only a
>part of a possible solution.  In your examples it would be
>better to have a "do NOT reimport to Usenet" flag instead of
>the Message-ID.

I don't think there is any reason to forbid "reimport to Usenet" as such
(for example, it may come back into Usenet in a different group and with a
different message-id).  What is important is to deal with it properly when
it comes back to the place where it was gatewayed out of usenet in the
first place.

>Here's a scenario where your strategy fails miserably:

>you -> A - B - C -> me
>       |   |
>       a   b
>       |   |
>       other

>I never see some of your articles in group ABC.test, they do
>not exist on server C (= "nope").  A, B, C, the gateways a
>and b, and the other net followed your strategy, they kept
>the Message-ID, but it didn't work as expected.

But you have not explained to me why it never got to server C, or indeed
exactly what a "nope" is. Please could you describe your scenario in more
detail.

>> we need a last-ditch backup, for use only in these really
>> nasty situations.

>No, the last ditch backup is "do not gateway if you don't know
>what you are doing".

Have you ever met a gateway admin who would admit that he "didn't know
what he was doing" :-). No, I think it is up to admins to decide when
the place he is supposed to be gatewaying to is just too bizarre that he
should give up the attempt. So long as it doesn't cause loops and so long
as the people on the other side of his gateway can accept what he sends,
he should be allowed to try.

>"Save the Message-ID somehow, maybe in the body" is completely
>misleading, because it's simply not good enough.

Well we are not actually recommending that in our draft. However, I would
not preclude that there might be some sufficiently bizarre "other medium"
where such a thing made good sense.

To: LIST: [email-masked];
Xref: clerew local.usefor:20481
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: News and nntp URI schemes
Date: Mon, 20 Dec 2004 14:20:35 GMT
Lines: 98
Sender: [email-masked]

>> If you want to comment, please do so to the URI list

>I'll do that separately, here I'm only interested in a
>technical detail relevant for Usefor:
> 
>>| <id-left> and <id-right> are defined in Section 3.6.4 of
>>| RFC 2822 [RFC2822]. They MUST be in a canonical form in
>>| which no <quoted-string> or <quoted-pair> is used in a
>>| context where the same semantic meaning could have been
>>| rendered without such quoting; moreover, no whitespace or
>>| ">" may be included, whether %-encoded or not and/or quoted
>>| or not.
> 
>>| For example, neither
>>|     news:"abcd"@example.com
>>| nor
>>|     news:"ab\cd"@example.com
>>| is in canonical form, because the form
>>|     news:[email-masked]
>>| is available.

>This text assumes that draft-usefor-02 is already on standards
>track, but that is not the case.  The "official" definition is
>still <unique@full_domain_name>, an "unofficial" definition is
>"<" local-part "@" domain ">"

The trouble is that there are just too many definitions of message-id
around the place. There is going to be strong pressure to keep to RFC 2822
or at least to a subset of it (which the RFC 1036 definition is not). I
don't think we should be writing news standards based on the RFC 1036
definition at this stage.

OTOH, if the URL definition is based on RFC 2822, then we have to warn
that certain "non-canonical" ones may cause problems.

The alternative is to give a very loose definition, on the grounds that
URLs are supposed to contain only whatever has been used in some existing
news article (and the agent that generated that article can worry about
which standard it conformed to). So any string of characters with an '@'
in the middle is good enough.

That is more or less what the new NNTP draft says:

   Each article MUST have a unique message-id; two articles offered by
   an NNTP server MUST NOT have the same message-id.  For the purposes
   of this specification, message-ids are opaque strings that MUST meet
   the following requirements:
   o  A message-id MUST begin with "<" and end with ">", and MUST NOT
      contain the latter except at the end.
   o  A message-id MUST be between 3 and 250 octets in length.
   o  A message-id MUST NOT contain octets other than printable US-ASCII
      characters.
   Two message-ids are the same if and only if they consist of the same
   sequence of octets.
.............
   This specification states that message-ids are the same if and only 
   if they consist of the same sequence of octets.  Other specifications 
   may define two different sequences as being equal because they are 
   putting an interpretation on particular characters.  RFC 2822 
   [RFC2822] has a concept of "quoted" and "escaped" characters.  It 
   therefore considers the three message-ids: 
      <[email-masked]>
      <"abcd"@example.com>
      <"ab\cd"@example.com>
   as being identical.  Therefore an NNTP implementation handing email
   articles must ensure that only one of these three appears in the
   protocol and the other two are converted to it as and when necessary,
   such as when a client checks the results of a NEWNEWS command against
   an internal database of message-ids.  Note that RFC 1036 [RFC1036]
   never treats two different strings as being identical.  Its draft
   successor restricts the syntax of message-ids so that, whenever RFC
   2822 would treat two strings as equivalent, only one of them is valid
   (in the above example only the first string is valid).

(And yes, I just noticed that NNTP will not accept Non-WS-Controls, but I
will deal with that elsewhere).

So yes, it might be better for the URL spec. to give a rather vague
definition of message-id, basing it on whatever the news article concerned
had actually used.

Anyway, the [email-masked] list is the proper place to continue this
discussion.

Date: Sun, 19 Dec 2004 20:55:32 +0000
From: Alexey Melnikov <[email-masked]>
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: Is this a STraw Poll? (was Standardize Complaints-To as deployed)
Sender: [email-masked]

Charles Lindsey wrote:

>>>>I would prefer #2, followed by #1 and #3.
>>>>
>>>>As a chair I would note that I see no consensus to use mail-complaint-to 
>>>>over complaints-url.
>>>>        
>>>>
>
>I think the question is whether to provide both of them initially.
>
This WG has enough work to do already, so I am Ok with defining 
complaints-url in a separate document. However the syntax for 
Injection-Info (or whatever we end up using) should allow for multiple 
URLs.

>>>Ah! You mean that those who want to give a (non-mailto) URL SHOULD give a
>>>mail-address as well. Yes, I could live with that.
>>>      
>>>
>>Yes.
>>Note, that this SHOULD belongs to USEAGE.    
>>
>
>Which is where we disagree.
>
The key reason for keeping syntax/protocol separate from policy is to be 
able to revise documents independently. Policy is more likely to change.

>  
>
>>And I am strictly against it becoming a MUST.    
>>
>
>Indeed.
>
Alexey

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: News and nntp URI schemes
Date: Sun, 19 Dec 2004 17:29:08 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 56
Sender: [email-masked]

Charles Lindsey wrote:

> If you want to comment, please do so to the URI list

I'll do that separately, here I'm only interested in a
technical detail relevant for Usefor:
 
>| <id-left> and <id-right> are defined in Section 3.6.4 of
>| RFC 2822 [RFC2822]. They MUST be in a canonical form in
>| which no <quoted-string> or <quoted-pair> is used in a
>| context where the same semantic meaning could have been
>| rendered without such quoting; moreover, no whitespace or
>| ">" may be included, whether %-encoded or not and/or quoted
>| or not.
 
>| For example, neither
>|     news:"abcd"@example.com
>| nor
>|     news:"ab\cd"@example.com
>| is in canonical form, because the form
>|     news:[email-masked]
>| is available.

This text assumes that draft-usefor-02 is already on standards
track, but that is not the case.  The "official" definition is
still <unique@full_domain_name>, an "unofficial" definition is
"<" local-part "@" domain ">"

| unique is any string of printing ASCII characters, not
| including "<" (left angle bracket), ">" (right angle 
| bracket), or "@" (at sign).
[...]
| Programmers are urged not to make assumptions about the
| content of Message-ID fields from other hosts, but to treat
| them as unknown character strings.

The proposed "canonical form" makes assumptions, and therefore
it's incompatible with RfC 1036 Message-IDs.  And the news URL
in RfC 1738 was about RfC 1036 Message-IDs:

| A <message-id> corresponds to the Message-ID of section 2.1.5
| of RFC 1036, without the enclosing "<" and ">"; it takes the
| form <unique>@<full_domain_name>.

No id-right, no id-left, no RfC 2822 nonsense.  Even in the old
<http://www.newsreaders.com/tech/draft-gilman-news-url-02.txt>
it is message = local-part "@" domain

No id-right, no id-left, and RfC 1036 in the references.  These
are FOUR relevant sources, 2 RfCs, one future FYI, and an old
draft, all using a sane concept of Message-ID.  Please replace
the RfC 2822 reference together with id-right and id-left by
the correct format defined in RfC 1036.

                           Bye, Frank

Date: Fri, 17 Dec 2004 23:37:34 -0500 (EST)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: FWS problem
Sender: [email-masked]

On Sat, 18 Dec 2004, Frank Ellermann wrote:
> But the new standard _should_ use a syntax reflecting the text
> where possible...

Yup, seems like a good idea.

> > message/rfc822 can be used to carry news without harm to its
> > readability.
> 
> If you're really determined to kill message/news...

I think it deserves killing.  There was some reason for it at the time,
but it hasn't worked out as well as hoped, and there has been some
favorable evolution in the definition of message/rfc822. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: FWS problem
Date: Sat, 18 Dec 2004 03:02:23 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 24
Sender: [email-masked]

Henry Spencer wrote:

> As in any standard, you have to read the text as well as the
> BNF; the text is not just commentary.

Not when I'm looking for a very fast shot. ;-)  Okay, I stand
corrected, s-o-1036 had SP in its text, RfC 1036 had it also,
and the new standard will keep it.

But the new standard _should_ use a syntax reflecting the text
where possible, name: SP *WSP ... instead of name: SP [FWS] ...
and ... *WSP CRLF instead of ... [FWS] CRLF

> message/rfc822 can be used to carry news without harm to its
> readability.

If you're really determined to kill message/news, my good old
"Mozilla 3" will survive it (it uses it in forwarded articles).

That's not under the top 10 of all "Mozilla 3" issues, I'm used
to fix its Content-Type: message/news manually if necessary.

                         Bye, Frank

Date: Fri, 17 Dec 2004 16:31:47 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Issues outstanding
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>You have so asserted many times. Several members of this WG who have taken
>the trouble to read your rants 

I am tired of you categorizing things you disagree with as a "rant". We've
covered this topic before, and it has been established that there is only
one way to know if an article is a followup or not, and that is to make
References header mandatory in a followup and prohibited in non-followups.
This is true no matter how you define a followup or what color the sky is
on the planet you live on. Any other set of requirements removes the
ability to determine what is and is not a followup. Now, while you may
consider that ability to be useless, that ability has been considered
important enough, prior to the latest drafts, of course, that it warranted
"MUST/MUST NOT" language. Either it is important to provide this
information or it is not. If it is not, then fine. I will not expect to
see it suddenly become important again in the next draft. Do not claim one 
thing now and suddenly change your mind later.

>...have agreed that what I stated is correct. I regard that issue as 
>closed.

Forty million frenchmen cannot be wrong, eh, Pascal?

Date: Fri, 17 Dec 2004 13:22:04 -0500 (EST)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: FWS problem
Sender: [email-masked]

On Fri, 17 Dec 2004, Frank Ellermann wrote:
> You can always convince me with this line.  But s-o-1036 says:
> | start-line      = header-name ":" space [ nonblank-text ] eol
> | space           = 1*( <HT (ASCII 9)> / <blank (ASCII 32)> )
> I don't see any mandatory SP in this syntax, I only see a WSP.

As in any standard, you have to read the text as well as the BNF; the text
is not just commentary.  Down in 4.2.3, son-of-1036 says: 

  In particular, RFC 1036 appeared to specify that the character
  immediately following the colon after a header name was required to be 
  a blank, and some news software insists on that, so this character MUST 
  be a blank. 

> Yes, but if message/news is a proper subset of message/rfc822,
> then why remove it from the IANA registry ?

Because message/rfc822 can be used to carry news without harm to its
readability.  If you want reliable bit-for-bit-correct transmission,
that's what application/news-transmission is for.

                                                          Henry Spencer
                                                       [email-masked]

To: LIST: [email-masked];
Xref: clerew local.nntp:3985 local.usefor:20472
Newsgroups: local.usefor,local.nntp
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: News and nntp URI schemes
Reply-To: [email-masked]
Date: Fri, 17 Dec 2004 12:56:30 GMT
Lines: 135
Sender: [email-masked]

http://www.ietf.org/internet-drafts/draft-hoffman-news-nntp-uri-03.txt has
recently appeared, but it does not yet reflect all the discussions that
have taken place on the uri.w3.org list.

What follows is a suggested text for the central portion of that draft,
taking those discussions into account, and which I have just posted to
uri.w3.org. I am also posting it to the nntp and Usefor lists for
information. If you want to comment, please do so to the URI list, to
which Reply-To has been set.

2.  The News URL Scheme

   The news URL scheme is used to refer to either news groups
   or individual Netnews articles, as defined in RFC 1036.

   The news URL takes the form:

      newsURL     = "news:" ( article / group / all-groups )
      article     = [ news-server "/" ] message-id
      group       = [ news-server "/" ] newsgroup-name
      all-groups  = news-server [ "/" [ "*" ] ] / "*"
      news-server = "//" server

   <server> is defined in [2396bis], and provides for a <host>, a <port>
   (defaulting to 119 in this scheme) and possibly a <userinfo>.

   If no <news-server> is specified, the resources are to be retrieved
   from whatever server has been configured for local use.

2.1  The newsURL contains an <article>

   A <message-id> corresponds to the <msg-id> of RFC 2822 and to the
   Message-ID of section 2.1.5 of RFC 1036, but without the enclosing
   "<" and ">".

   The resource retrieved by this URL is the Netnews article with the
   given <message-id>.  In a properly working Netnews system, the same
   article will be obtained whatever server is accessed for the purpose
   (assuming the server in question carried that article in the first
   place and that it has not expired).

   <id-left> and <id-right> are defined in Section 3.6.4 of RFC 2822
   [RFC2822]. They MUST be in a canonical form in which no
   <quoted-string> or <quoted-pair> is used in a context where the same
   semantic meaning could have been rendered without such quoting;
   moreover, no whitespace or ">" may be included, whether %-encoded or
   not and/or quoted or not.

   For example, neither

      news:"abcd"@example.com

   nor

      news:"ab\cd"@example.com

   is in canonical form, because the form

      news:[email-masked]

   is available.

2.2  The newsURL contains a <group>

   The <newsgroup-name> is a period-delimited hierarchical name, such as
   "comp.lang.perl.modules".

   The resource retrieved by this URL is some means to gain access to
   the articles in the given <newsgroup-name> that are available on the
   given <server> (usually by invoking a suitable news reading agent
   initialized to access that group).

2.2  The newsURL contains an <all-groups>

   If the newsURL is of one of the following forms:
      <URL:news:*>
      <URL:news://news.example.com/*>
      <URL:news://news.example.com/>
      <URL:news://news.example.com>
   it refers to "all available news groups".  The resource retrieved by
   this URL is some means to gain access to all the newsgroups that are
   available on the given <server> (usually by invoking a suitable news
   reading agent).

[Issue: Do we really want all those forms? Only the first was in RFC 1738.
Some agents are known to barf on anything with '*' in it; there exist
agents which recognize the last two forms. Maybe the '*' part of the
notation should be dispensed with.]

3.  The nntp URL scheme

   The nntp URL scheme is used to refer to individual Netnews articles,
   as defined in RFC 1036.

   The nntp URL takes the form:

      nntpURL     = "nntp"  ":" news-server "/" newsgroup-name "/" range
      news-server =  "//" server
      range       = article-number ["-" [article-number]]
      article-number = 1*DIGIT

   Observe, in contradistinction to the news scheme, that the
   <news-server> is not optional here, because the mapping from
   <article-numbers> to actual articles is established independently by
   each server.

3.1  The range is a single <article-number>

   The resource retrieved by this URL is the Netnews article numbered
   by the given <article-number> in the given <newsgroup-name> on the
   given <server>.

3.2  The range encompasses more than a single <article-number>

   The resource retrieved by this URL is some means to gain access to
   the articles numbered within the given <range> of <article-
   number>s in the given <newsgroup-name> on the given <server>
   (usually by invoking a suitable news reading agent initialized to
   access that range). A <range> of the form "nnnn-" provides access to
   all articles numbered "nnnn" and above.

To: LIST: [email-masked];
Xref: clerew local.usefor:20471
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Issues outstanding
Date: Fri, 17 Dec 2004 12:50:18 GMT
Lines: 60
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>No. Non-followups MUST NOT have a References header.

If you want to follow that principle, then you need definition #2.

I offered 2 definitions and described their consequences. You can take
your pick. So can everyone else.

>But I note that the section of the draft which used to state that
>References was mandatory for a followup and prohibited in a non-followup
>has dissappeared. I find it in neither usefor or usepro. All I see is a
>reference to RFC2822, which says that References SHOULD be used in a
>reply. It is no longer mandatory in a reply.

>When did we decide that References weren't mandatory in followups? Why did 
>this text, which has been in the draft for years, get removed? 

Yes, there are still several things missing from the USEFOR draft, and
that is one of them. I have suggested a suitable text to my co-author (two
texts, actually, to match the two definitions), and hopefully something
suitable will appear in the next draft.

>>these formulations. 

>That is untrue.

You have so asserted many times. Several members of this WG who have taken
the trouble to read your rants on the topic have agreed that what I stated
is correct. I regard that issue as closed. All we need to do now is to
decide which formulation to go ahead with.

>There are, of course, other issues outstanding, but since Charles doesn't 
>want to talk about them, they weren't listed.

There are indeed various smaller issues under discussion, but the four
that I singled out are the major ones which are holding up progress on the
drafts.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: msg-id
Date: Fri, 17 Dec 2004 14:06:08 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 117
Sender: [email-masked]

Charles Lindsey wrote:

> That implementation is broken

Indeed, fortunately I found an ISP allowing me to fix it... :-)

> changing our draft cannot fix it.

True, but if you'd say "domain" instead of "id-right" it's more
obvious.  Address literals are already on the border to utter
dubious, with different algorithms for "id-left", different
time zones, and dynamic IPs.  But <whatever@invalid> isn't only
a bad idea, it's broken.

>> Of course the UA is broken, but it's another argument for
>> RHS = domain as specified in 2821.

> Why? You are asking for further changes. Others are asking
> for the minimal change from RFC 2822. Which am I supposed to
> do?

Let's just find a minimal _working_ model for Message-IDs.  In
RfC 2822 you were not interested in these details, you only
wanted some simple syntax covering wild and wonderful things
like mail, s-o-1036, and X.400.

Here and today we know it better, we want something working in
Usenet, in news URLs, in cancel messages, and as many existing
UAs as possible.  IMHO that disqualifies <whatever@invalid>,
and it also doesn't allow NO-WS-CTL.

For the purists like Russ and Bruce the "working model" is most
probably RfC 1036 <unique@full_domain_name>.  For the heretics
like me the "working model" is s-o-1036, essentially the same:

|    message-id          = "<" local-part "@" domain ">"

RfC 2822 is too broad, they wanted a "working model" covering
obsolete and broken software for mail, where Message-IDs aren't
essential.  I'm only guessing why RfC 2822 got it wrong, you
were there:  Why did they replace the old concept of "like an
addr-spec, only unique" by a new "id-left @ id-right" ?

The RfC 2822 "addr-spec" is still fine, it mentions "domain":

| addr-spec  =       local-part "@" domain

But the RfC 2822 Message-ID invents an obscure "id-right" out
of thin air:

| msg-id     =       [CFWS] "<" id-left "@" id-right ">" [CFWS]

I just don't know why you did this in 2822, but it was wrong.
It's incompatible with 1036 and s-o-1036, we can't use it here.

We need the good old "<" unique "@" domain ">" minus RfC 2822
oddities like obs-domain or CFWS.  The RfC 2821 domain syntax
makes sense, it even includes address literals, just forget the
1 in its domain = (subdomain 1*("." subdomain)) / address-literal

Or copy the s-o-1036 definition of domain adding the address
literals as defined in 2821:

| domain        = unquoted-word *( "." unquoted-word )
| unquoted-word = 1*unquoted-char
| unquoted-char = <ASCII printable character except !()<>@,;:\".[]>

It's no big difference from 2822 or your text, and the syntax
still allows an insane <whatever@invalid>, but at least the
intention "domain" is clear and not obscured by an "id-right".

 [dubious <whatever@news-fe-01> and ...!news-fe-01!...]
> it is quite likely to be unique to that site, given that it
> was considered suitable as a path-identity.

It was more likely an error in the configuration of this news
server.

> It is not necessarily wrong.

Show me the UUCP world map entry for news-fe-01, and I consider
it :-)  That's what I said in the de.admin,news.misc.discussion
about it, all others said "unconditionally invalid Message-ID".

> A lot of news (and even some email) is still transported by
> UUCP and other protocols.

s-o-1036 mumbles something about using .uucp in this case, and
to stop this nonsense a.s.a.p.  That was 1994, and the dubious
"domain" and path entry was news-fe-01 (not news-fe-01.uucp) in
2004.

>    <[email-masked]>
>    <[email-masked]>

> They are DIFFERENT message-ids. Yes?

Yes.  And I would use the same reasoning for id-left:

     <[email-masked]>
     <"12345678"@foo.example.com>

Also different.  I'm a KISS extremist.  Some gateways trying
to "fix" the second version while others don't would be hell.

 [NO-WS-CTL in Message-ID]
> persuade someone else on this list to agree to the further
> restrictions you are asking for

If Russ really is a 1036 purist, and Henry still is a s-o-1036
heretic, then I simply assume that they also hate any NO-WS-CTL
in Message-IDs.  Only Bruce loves it, but I'm not sure why.

You have already said that you hate it, because NO-WS-CTL is a
PITA for the news URL.
                      Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: FWS problem
Date: Fri, 17 Dec 2004 11:50:34 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 71
Sender: [email-masked]

Charles Lindsey wrote:
 
> I am not sure how we ever got away from square one.

I wanted the ABNF to reflect your "MUST contain at least one
non-whitespace", and Bruce claimed that this is not the case
for "name: SP CRLF SP body CRLF".  But now "we" (you and me ;-)
obviously agree that this is really illegal.
 
> we have always concluded it needs to be so.

Okay, maybe it's a "we" excl. Bruce, and I can live with it,
if "we" (= you ;-) replace all corresponding [FWS] by *WSP in
the ABNF.

> It has indeed been so since Son-of-1036.

You can always convince me with this line.  But s-o-1036 says:

| header          = start-line *continuation
| start-line      = header-name ":" space [ nonblank-text ] eol
[...]
| space           = 1*( <HT (ASCII 9)> / <blank (ASCII 32)> )

I don't see any mandatory SP in this syntax, I only see a WSP.

Are you absolutely sure that we want to go back to the "blank"
in 1036 ?  It's the old problem of "does RfC 2822 update 1036
in some way ?"  IMHO a "name: 1*WSP body CRLF" is better than
your "name: SP *WSP body CRLF".

>> should be "name: SP *WSP header content CRLF" everywhere.

> Maybe for our own news headers, but in some cases the syntax
> for the [*WSP header] bit comes straight from RFC 2822, and
> so it is not ours to change.

Let's fix it at least for our headers, replace [FWS] by *WSP
before and after any header content of "our" headers.

>> The syntax "name: SP [FWS] header content CRLF" is wrong,
>> because "name: SP CRLF SP header content CRLF" is illegal.
 
> I don't think this prohibition can be made by syntax. The
> corresponding provision in RFC 2822 (the one against empty
> folded lines) was not made by syntax, and we are following
> RFC 2822 wherever possible as a matter of policy.

We're following the _intent_ of RfC 2822 whereever possible,
but you're not forced to copy inaccurate syntax.  If there's a
way to get it right in the ABNF, just do it please.  And as a
matter of policy I'm not happy with making a TAB illegal in a
place where it's allowed by both s-o-1036 and RfC 2822.

> The draft makes it perfectly clear what the rule is, and that
> it is different from RFC 2822.

Yes, but if message/news is a proper subset of message/rfc822,
then why remove it from the IANA registry ?  It's a practical
problem at least for gateways and moderated newsgroups, or in
your terminology for all injecting agents:

For "name: HT CRLF HT body CRLF" the injecting agent is forced
to format it as "name: SP body CRLF" somehow, and that's not
always trivial, e.g. what about a field body starting with a
maximal length encoded word ?
                                Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Fri, 17 Dec 2004 10:43:09 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 63
Sender: [email-masked]

Charles Lindsey wrote:

> The best means for preventing loops is to ensure that
> Message-IDs are propagated correctly.

Unfortunately that's not the case.  Loops cause two kinds of
problems:  dupes and nopes.  Saving the Message-ID is only a
part of a possible solution.  In your examples it would be
better to have a "do NOT reimport to Usenet" flag instead of
the Message-ID.

Here's a scenario where your strategy fails miserably:

you -> A - B - C -> me
       |   |
       a   b
       |   |
       other

I never see some of your articles in group ABC.test, they do
not exist on server C (= "nope").  A, B, C, the gateways a
and b, and the other net followed your strategy, they kept
the Message-ID, but it didn't work as expected.

I always considered nopes as worse than dupes, because dupes
are at least obvious.  With nopes the only hint you might get
are replies to articles you've never seen.

> it sometimes fails because message-ids get
> lost/changed/whatever somewhere in the other medium. So we
> need a backup method.

We don't need a backup method, we need a clear understanding
why loops generally don't work.  The text about Message-IDs in
comments only helps against dupes, but not nopes.

You could use the A - a - other - b - B - C example as shown
above in the draft, and then explain both problems, dupes and
nopes.

> X-Gatewayed-By: <name of gateway>

Yes, that's a good idea, it's the "do not reimport" flag.

> we need a last-ditch backup, for use only in these really
> nasty situations.

No, the last ditch backup is "do not gateway if you don't know
what you are doing".  And for a proper understanding of these
problems dupes _and_ nopes are the very minimum.  You could
use the same example A - a - other - b - B - C example to show
other problems like lost References (if the other net has no
concept of References and cannot preserve any X-References).

"Save the Message-ID somehow, maybe in the body" is completely
misleading, because it's simply not good enough.

BTW, if I'd "demand" to save FidoNet SeenBy and Path as some
X-SeenBy, X-Path, and X-Original-Fido-Zone, then that would be
also almost useless.  In FidoNet loops are _always_ illegal,
no matter how you do it.
                         Bye, Frank

Date: Thu, 16 Dec 2004 11:51:39 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Issues outstanding
Sender: [email-masked]

>   nevertheless have a References header.

No. Non-followups MUST NOT have a References header. That is the ONLY way 
to determine what is and is not a followup. That is the purpose of the 
References header. This is the position our draft (and precursors) have 
had for years.

But I note that the section of the draft which used to state that
References was mandatory for a followup and prohibited in a non-followup
has dissappeared. I find it in neither usefor or usepro. All I see is a
reference to RFC2822, which says that References SHOULD be used in a
reply. It is no longer mandatory in a reply.

When did we decide that References weren't mandatory in followups? Why did 
this text, which has been in the draft for years, get removed? 

>these formulations. 

That is untrue. It has been established that using (1) does not allow 
anyone to determine if an article is a followup or not, while (2) does.
However, (2) now puts an extraneous requirement on the header, which does 
not match up with the actual definition of the header in our draft.

There are, of course, other issues outstanding, but since Charles doesn't 
want to talk about them, they weren't listed.

5. "Poster" is "author and sender". Nope, just "sender".

6. Outgoing gateways ought to mess with the bodies of a message "to 
prevent loops". Nope.

7. "By email" being the only method permitted to get articles to a 
moderator. Nope.

8. The implied ability of injecting agents to detect valid email addresses 
for a poster. Amazing.

9. "Taken" vs. "Copied" re. Followup agents, plus defaulted "by default" 
actions. Redundant/redefinition & unrequested/unilateral change.

10. Duties of a Reading Agent which we have no business assigning duties 
to, and which we actually DON'T assign duties to, despite a section of the 
draft with that title pretending to assign them.

To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 16 Dec 2004 11:30:37 -0800
Sender: [email-masked]

John Stanley <[email-masked]> writes:
> "Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>> There is no mention of writing stuff into bodies ...

> 7.9.1.  Duties of an Outgoing Gateway

>    1. The message identifier of the news article should be preserved if
>       at all possible, preferably as or within the corresponding unique
>       identifier of the other medium, but if not at least as a comment
>       in the message. This helps greatly with preventing loops.

As the author of the original text here, I can say authoritatively that I
wasn't thinking of the body of the message, but rather some equivalent of
an X-Comment header in mail.  I certainly have no objections to a
clarification in that regard.  The goal is to get the message ID somewhere
where an aware gateway going the other direction could pick it out of the
message again, but without bothering the recipients of the message with
extraneous tracking information they don't care about.

If that isn't possible, it's not possible, and that's fine.  It mostly
just means that bidirectional gatewaying probably isn't a good idea for
that medium.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Thu, 16 Dec 2004 11:10:56 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

Thorfinn <thorfinn@xxxxxxxxxxxxxx>:

>Indeed there sometimes is, but often it's buried amongst so much
>extraneous random stuff that it's hard to find the point. 

I tend to respond to two or three things at one time. If the other two 
things don't interest you, fine, but they are not "random" and not 
"extraneous". 

As for "random" and "extraneous" ...

>Apart from all the quoted stuff below, I note that message-id in email
>appears to be *optional*, 

That's nice. Also irrelevant. Maybe even "random" and certainly 
"extraneous". 

>And even with that circumstance, such collisions could easily happen
>even *without* a past owner, since there are several places (and thus
>several different algorithms in use) where a message ID could be
>generated on the *same* host. 

The fact that there might be several different processes that create 
message ids does not mean that there are several different algorithms 
involved. It does not mean there will be collisions, either.

>And then we have the problem that a *lot* of user-agents (both mail and
>news ones) are operating behind firewalls, and do *not* have actual
>valid Internet domain names to use as the RHS,

Excuse me, but "firewall" and "no valid domain name" are not synonymous. 

>nor do they even have valid unique Internet IP addresses, 

Random and extraneous, I think you called it. So what? Where is the 
requirement for a unique IP address?

>Anyway, it is *entirely* feasible for those several different
>algorithms, even when used on a host with a proper FQDN, to generate
>colliding message-ids.

If I proposed several different algorithms for use at the same time, why 
certainly this repeated discussion about several different algorithms 
would be valid. But since I did not ...

>The point is, some of us *do* accept that even a "MUST" *does* bow to
>practicalities. 

The "practicality" in this case, as a reminder, is an algorithm that is
known to produce not only ids that look the same as some other algorithm
(sauce for the goose), but duplicates all by itself. Since there are known
algorithms that do NOT produce duplicates all by themselves (the one I
proposed being just one) this "practicality" is just another way of saying
"MUST doesn't mean MUST". Otherwise, you'd say "it's broken".

>So, I guess we come to the meat of it:

Were I to have taken offense at your "random" and "extraneous" comments, 
I'd make a pithy remark here, something about getting to "the meat of it" 
after such a long wind-up.

>I think that a unique message id is a *theoretically impossible* thing
>to generate. I don't care *what* algorithm you use, someone can design
>a counter algorithm, which also generates "unique" ids, which collides
>with the previous algorithm. 

It does not matter what algorithm YOU design, it will not change the 
product of MY algorithm. Now, you can deliberately create duplicates by 
using your algorithm, but that does not disprove mine, only yours. Design 
an algorithm that is: 1) get latest message in group 'news.groups' 2) 
copy the message id from that. That doesn't mean the algorithm you are 
copying from is broken, it means YOURS is broken.

>Now, whilst we *can* override the mail standards to say that the
>uniqueness is now a "SHOULD" rather than a "MUST"...

When did we do that? Who has even suggested it?

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>So what does "MUST be unique" actually mean, when it comes to be
>interpreted in the Real World (TM)? It means that, if ever you find a
>system that has generated a duplicate message-id, you have a perfect
>excuse to go and beat him about the head for being in clear violation of
>the standard.

This is going to be fun. I'm going to write a system that posts a flood of 
messages using message ids from Charles's message id space, and when he 
finds his system producing a "duplicate", he'll beat himself up. After 
all, the duplicate is clearly his fault.

>In practical terms, if a site is persistently generating bad message-ids,
>then complaining to its provider/ISP/upstream is likely to be effective,

Since duplicates are not accepted at injection, it would be pretty hard 
for anyone but the person running the injecting agent or the posting agent 
to ever see one. Why would they complain to their upstream news site, or 
to their ISP?

>... because even the more clueless providers can see that interoperability is
>being affected.

Excuse me? If my incoming gateway correctly detects and prevents a loop by 
using the message id in the incoming message, and that results in the 
injecting agent not being able to inject the article, how has 
"interoperability" been affected? Isn't that the "Right Thing" to do?

Date: Thu, 16 Dec 2004 10:37:24 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>There is no mention of writing stuff into bodies ...

7.9.1.  Duties of an Outgoing Gateway

   1. The message identifier of the news article should be preserved if
      at all possible, preferably as or within the corresponding unique
      identifier of the other medium, but if not at least as a comment
      in the message. This helps greatly with preventing loops.

>The whole of this text is little changed from what Russ wrote years ago,
>and I see no reason to make any significant change to it now.

I do not accept the "it's too late" excuse for not correcting mistakes or 
making things clearer.

To: LIST: [email-masked];
Xref: clerew local.usefor:20466
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Thu, 16 Dec 2004 11:20:45 GMT
Lines: 50
Sender: [email-masked]

In <[email-masked]> Thorfinn <[email-masked]> writes:

>On Wed 15 Dec 2004 at 08:23:00AM -0500, in <[email-masked]>,
>Bruce Lilly <[email-masked]> wrote:

>> I haven't the time to go into detail now, but I strongly
>> suggest that reading RFC 2822 section 3.6.4, RFC 822
>> section 4.1 and section 6, and draft-crocker-email-arch-01
>> sections 3.2 and 3.3 may help to clear up some of the
>> contradictory remarks made on this topic in this WG.

>Apart from all the quoted stuff below, I note that message-id in email
>appears to be *optional*, which it certainly shouldn't be for us.

As the USEFOR draft stands, it inherits RFC section 3.6.4, except insofar
as it explicitly modifies it.

So USEFOR _does_ make that header a MUST, and it _does_ restrict the
syntax. But it is silent on the 'uniqueness' issue, and therefore the RFC
2822 wording on uniqueness is binding for us. I see no particular reason
to change that situation.

So what does "MUST be unique" actually mean, when it comes to be
interpreted in the Real World (TM)? It means that, if ever you find a
system that has generated a duplicate message-id, you have a perfect
excuse to go and beat him about the head for being in clear violation of
the standard.

Now "beating about the head" is easier said than done in cyberspace, but
that is true of all standards violations (and let no one claim that they
"never happen").

In practical terms, if a site is persistently generating bad message-ids,
then complaining to its provider/ISP/upstream is likely to be effective,
because even the more clueless providers can see that interoperability is
being affected. But if it is one of those accidental cases that only
happens once in a thousand years, and you manage to spot it, then by all
means attempt to beat the perpetrator about his virtual head, but do not
get over-excited if his real head remains intact.

Date: Thu, 16 Dec 2004 09:30:53 +1100
From: Thorfinn <[email-masked]>
To: [email-masked]
Cc: Alexey Melnikov <[email-masked]>
Subject: Re: msg-id
Sender: [email-masked]

On Wed 15 Dec 2004 at 08:23:00AM -0500, in <[email-masked]>,
Bruce Lilly <[email-masked]> wrote:
> On Mon December 13 2004 19:09, Thorfinn wrote:
> > We mean MUST, and we also accept that MUST applies in the universe of
> > practicalities, where nothing is actually of zero probability, so "close
> > enough" is good enough.
> RFC 2119 keywords, especially MUST/MUST NOT, are applicable
> in specific types of circumstances as detailed in 2119 (q.v.).

Yes.  They are.

> Disappointment of a purchaser of second-hand goods is not one
> of the criteria.

Are you talking about the second hand domain name case?  Anyway,
more response further down, after the RFC quoting.

> > You don't accept that, that's fine.  Everyone else, AFAICT, does accept
> > that.  If anyone else doesn't, speak up now.
> I object to such an ultimatum from other than the chair.
> There is merit to some of what John has to say,

Indeed there sometimes is, but often it's buried amongst so much
extraneous random stuff that it's hard to find the point.  If you have
the time to wade through that to find the gems, then please, feel free.
I, for one, no longer do.

> and there are serious problems with what some others have claimed. Any
> ultimata should come from the chair, not individual WG members, and
> should carry a reasonable time period for response, not demanding an
> instant (i.e. "now") response.

Fair enough.  I won't issue such a comment again.  It was more of a
rhetorical device than an actual ultimatum, but it's *also* true that if
I feel the need to use rhetorical devices in what I'm saying, then I
shouldn't bother replying.

> I haven't the time to go into detail now, but I strongly
> suggest that reading RFC 2822 section 3.6.4, RFC 822
> section 4.1 and section 6, and draft-crocker-email-arch-01
> sections 3.2 and 3.3 may help to clear up some of the
> contradictory remarks made on this topic in this WG.

Apart from all the quoted stuff below, I note that message-id in email
appears to be *optional*, which it certainly shouldn't be for us.

===== begin quoted stuff =====

RFC2119:

1. MUST   This word, or the terms "REQUIRED" or "SHALL", mean that the
   definition is an absolute requirement of the specification.

3. SHOULD   This word, or the adjective "RECOMMENDED", mean that there
   may exist valid reasons in particular circumstances to ignore a
   particular item, but the full implications must be understood and
   carefully weighed before choosing a different course.

RFC2822:
--------

3.6.4 (part of)

The message identifier (msg-id) itself MUST be a globally unique
identifier for a message.  The generator of the message identifier
MUST guarantee that the msg-id is unique.

RFC822:
-------

 4.1.  SYNTAX

     msg-id      =  "<" addr-spec ">"            ; Unique message id

4.6.  REFERENCE FIELDS

     4.6.1.  MESSAGE-ID / RESENT-MESSAGE-ID

             This field contains a unique identifier  (the  local-part
        address  unit)  which  refers to THIS version of THIS message.
        The uniqueness of the message identifier is guaranteed by  the
        host  which  generates  it.  This identifier is intended to be
        machine readable and not necessarily meaningful to humans.   A
        message  identifier pertains to exactly one instantiation of a
        particular message; subsequent revisions to the message should
        each receive new message identifiers.

6.  ADDRESS SPECIFICATION

     6.1.  SYNTAX
     addr-spec   =  local-part "@" domain        ; global address

     local-part  =  word *("." word)             ; uninterpreted
                                                 ; case-preserved

... and a lot more in words in section 6, all superseded by 2822 anyway.

draft-crocker-email-arch-01:
----------------------------

3.2  Domain Names

   A domain name is a global reference to an Internet resource, such as
   a host, a service or a network.  A name usually maps to one or more
   IP Addresses.  A domain name can be administered to refer to
   individual users, but this is not common practice.  The name is
   structure as a hierarchical sequence of sub-names, separated by dots
   (".").

   When not part of a mailbox address, a domain name is used in Internet
   mail to refer to a node that took action upon the message, such as
   providing the administrative scope for a message identifier, or
   performing transfer processing.

3.3  Message Identifers

   Message identifiers have two distinct parts, divided by an at-sign
   ("@").  The right-hand side contains a globally interpreted name for
   the administrative domain assigning the identifier.  The left-hand
   side of the at-sign contains a string that is globally opaque and
   serves to uniquely identify the message within the domain referenced
   on the right-hand side.  The duration of uniqueness for the message
   identifier is undefined.

   The identifier may be assigned by the user or by any component of the
   system along the path.  Although Internet mail standards provide for
   a single identifier, more than one is sometimes assigned.

===== end quoted stuff =====

I note one major item that differs about news and email.  Email messages
rarely get collected into large collections, and it's not, thus, as
simportant for them to have a unique message id.  They aren't even
required to have one.  It's optional.

Usenet ones *do*... that's the whole point, after all.  So everything
*does* have a message-id.  Now, if a message-id *is* generated, that
MUST be unique is sitting there (and has been sitting there for some
time, and isn't about to go away).  That's fine and dandy, and as far as
I can tell, most people are entirely happy with that being a MUST.
Please correct me (in an appropriately timely fashion) if I'm wrong on
that.

Back to the second hand domain case... Firstly, it may not even be
possible for someone obtaining a domain to know whether the domain has
been used previously, as domain names often go back to a registrar
first, and some registrars are reknowned for being rather poor at giving
out information at times.  At least in the .au space they are *required*
to go back to the registrar with no on selling allowed, rather than
being on-sold directly.

That being the case, are you actually suggesting that even with that
problem, a domain name owner must find all the past algorithms that have
been used to generate IDs for that domain, and then design a new
algorithm to avoid colliding with *any* of them, and make sure that any
software installed uses that algorithm?

And even with that circumstance, such collisions could easily happen
even *without* a past owner, since there are several places (and thus
several different algorithms in use) where a message ID could be
generated on the *same* host.  Mail-transfer-agents make them, any one
of a zillion different mail-user-agents will make them on their own, and
existing news servers and news clients generate their own too.

And then we have the problem that a *lot* of user-agents (both mail and
news ones) are operating behind firewalls, and do *not* have actual
valid Internet domain names to use as the RHS, nor do they even have
valid unique Internet IP addresses, due to them generally being in the
"private" address spaces allocated.  And they can't even use ethernet
MAC addresses, because a lot of them are dialup users, and don't *have*
a MAC address...

Anyway, it is *entirely* feasible for those several different
algorithms, even when used on a host with a proper FQDN, to generate
colliding message-ids.  I think that's actually got a significantly a
higher likelihood than the algorithm "use SHA256 of message body and
existing known headers as message-id-lhs" (which incidentally also has
the property that if the message is the same, the message-id-lhs will be
the same).

Are you suggesting that all those algorithms (including the use SHA256
one, which has a theoretical collision possibility immensely smaller
than the "once per thousand years" being bandied about) do not satisfy
the MUST that is in RFC2822, and that people should not use them?

The point is, some of us *do* accept that even a "MUST" *does* bow to
practicalities.  Yes, it *is and should be* an absolute requirement of
the specification that message-id generators make "unique" message-ids.
But no, we're not blind about it, and we make judgements about
probabilities, and certain sufficiently low probability events, to us,
are not worth worrying about.

This is all moot anyway... Our job is to specify, and to leave it up to
the implementors to implement.  Sure, we have to make sure that what we
specify is feasible and reasonable.

So, I guess we come to the meat of it:

I think that a unique message id is a *theoretically impossible* thing
to generate.  I don't care *what* algorithm you use, someone can design
a counter algorithm, which also generates "unique" ids, which collides
with the previous algorithm.  And multiple algorithms *are* in use out
there.

However, I also think that a unique message id is a *theoretically
desireable* thing to have.

Now, whilst we *can* override the mail standards to say that the
uniqueness is now a "SHOULD" rather than a "MUST"... I think that's
something which would convey *entirely* the wrong message to
implementors.

After all, it's *even more important* for us to have a unique message-id
for every message, since we collect who knows how many millions of
usenet messages in the one place all the time, and they're *all*
supposed to have unique message-ids.

Later,

  Thorf

-- 
<a href="http://tertius.net.au/~thorfinn/">[email-masked]</a>
"Everyone knows that bloodstained walls improve productivity."
    -- Thorfinn(tertius.net.au

To: LIST: [email-masked];
Xref: clerew local.usefor:20463
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Issues outstanding
Date: Wed, 15 Dec 2004 19:14:01 GMT
Lines: 86
Sender: [email-masked]

1. Complaints-To

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Thorfinn <[email-masked]>
Subject: Re: msg-id
Date: Wed, 15 Dec 2004 08:23:00 -0500
Cc: Alexey Melnikov <[email-masked]>, [email-masked]
Sender: [email-masked]

On Mon December 13 2004 19:09, Thorfinn wrote:

> We mean MUST, and we also accept that MUST applies in the universe of
> practicalities, where nothing is actually of zero probability, so "close
> enough" is good enough.

RFC 2119 keywords, especially MUST/MUST NOT, are applicable
in specific types of circumstances as detailed in 2119 (q.v.).
Disappointment of a purchaser of second-hand goods is not one
of the criteria.

> You don't accept that, that's fine.  Everyone else, AFAICT, does accept
> that.  If anyone else doesn't, speak up now.

I object to such an ultimatum from other than the chair. There
is merit to some of what John has to say, and there are
serious problems with what some others have claimed. Any
ultimata should come from the chair, not individual WG
members, and should carry a reasonable time period for
response, not demanding an instant (i.e. "now") response.
I haven't the time to go into detail now, but I strongly
suggest that reading RFC 2822 section 3.6.4, RFC 822
section 4.1 and section 6, and draft-crocker-email-arch-01
sections 3.2 and 3.3 may help to clear up some of the
contradictory remarks made on this topic in this WG.

To: LIST: [email-masked];
Xref: clerew local.usefor:20462
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: FWS problem
Date: Wed, 15 Dec 2004 11:36:50 GMT
Lines: 90
Sender: [email-masked]

>> the whole point of "header content" is that it does not
>> include the header name

>Fine, that's what you have in the draft, we're now back to
>square one.

I am not sure how we ever got away from square one.

>>>> The header content of every header line (including the
>>>> first and any that are subsequently folded) MUST contain
>>>> at least one non-whitespace character.

>>> Yes, that's the erroneous paragraph

You are confusing two issues. Whether that provision is correct, and the
manner in which we document it (and other SP-related matters).

So no, it is not erroneous. We have discussed it many times, including
once in the last 6 months, and we have always concluded it needs to be so.
It has indeed been so since Son-of-1036.

>Now that's also back to the start of this discussion.  In
>other words "name: SP CRLF body CRLF" is invalid, because
>the first line "name: SP CRLF" has no non-whitespace header
>content, it has only a single trailing SP.

Yes.

>JFTR, Bruce said that this is okay.  Your draft says that
>this is invalid.  If you are right the syntax in the draft
>should be "name: SP *WSP header content CRLF" everywhere.

Maybe for our own news headers, but in some cases the syntax for the
[*WSP header] bit comes straight from RFC 2822, and so it is not ours to
change.

>The syntax "name: SP [FWS] header content CRLF" is wrong,
>because "name: SP CRLF SP header content CRLF" is illegal.

I don't think this prohibition can be made by syntax. The corresponding
provision in RFC 2822 (the one against empty folded lines) was not made by
syntax, and we are following RFC 2822 wherever possible as a matter of
policy.

>> some "helpful" intermediate site that thinks trailing
>> white space is evil might well change it to
>> "name: CRLF SP body CRL", at which point it _will_ break
>> existing software.

>My text editor does this when I forget to SET TRAILING ON.
>Is it the duty of injecting agents to fix any "name: CRLF
>SP body CRLF", i.e. insert the missing SP after the colon ?

It is not just text editors and injecting agents. Some half-baked
relaying/serving agent or some gateway might remove a trailing whitespace
somewhere en route, turning a possibly acceptable article into one that
would break something further on. We could not take that risk, and hence
the prohibition of :name: SP CRLF".

>> I think some existing software also breaks with
>> name: TAB body CRLF
>> which is why we insist on SP.

>In other words, the syntax of message/news is very different
>from message/rfc822.  I have no problem with this consequence,
>but please say so in the draft, and don't delete message/news.

The draft makes it perfectly clear what the rule is, and that it is
different from RFC 2822.

What IS wrong with the present USEFOR draft is the definition of the term
"header content", and in particular how much of the whitespace it
includes, and that needs to be fixed.

To: LIST: [email-masked];
Xref: clerew local.usefor:20461
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Wed, 15 Dec 2004 11:15:58 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>This is what my original text about gatewaying said, long, long ago, so I
>don't understand why we're even discussing it.  It's completely outside
>the scope of our document to specify how gateways into other media should
>deal with the shortcomings of that media.  All we can say is what's
>important to Usenet and leave it to the person writing the gateway, or to
>a separate specification aimed only at gatewaying.

Indeed so. The text in the current draft contains various SHOULD/MUST
[NOT] provisions. It also tries to be helpful by explaining why these
provisions are necessary, and gives a few hints where it is not obvious
how some provision might be implemented (including a rather full example
of one particular gateway). I think it makes it clear that gateway
implementors need to have a thorough understanding of the foibles of the
"other medium", and that there is no "one size fits all" solution.

There is no mention of writing stuff into bodies (so I am not sure how we
came to be dicussing that), except for one suggestion that the news
message-id should be included in a "comment" if there is nowhere else to
put it (though that might turn out to be of more use to human debuggers
that to automated loop detectors).

The whole of this text is little changed from what Russ wrote years ago,
and I see no reason to make any significant change to it now.

To: LIST: [email-masked];
Xref: clerew local.usefor:20456
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Ad hominem
Date: Tue, 14 Dec 2004 19:15:17 GMT
Lines: 19
Sender: [email-masked]

In local.usefor you write:

If there are any specific topics within this message from John that some
member of this WG would like me to reply to explicitly, then I will gladly
do so.

But since it seems to contain nothing that has not been responded to many
times before, I am not minded to make any other reply.

To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 14 Dec 2004 14:03:38 -0800
Sender: [email-masked]

John Stanley <[email-masked]> writes:

> If we are going to talk about gating articles into a horribly broken
> messaging system (one that has no concept of message identifiers), then
> we have to accept that it is broken and isn't our responsibility or
> within our ability to fix.

Right.

This is what my original text about gatewaying said, long, long ago, so I
don't understand why we're even discussing it.  It's completely outside
the scope of our document to specify how gateways into other media should
deal with the shortcomings of that media.  All we can say is what's
important to Usenet and leave it to the person writing the gateway, or to
a separate specification aimed only at gatewaying.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Tue, 14 Dec 2004 13:49:52 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>And the only last-ditch possibility left is to include something in the
>body of the message, such as:

>THIS ARTICLE HAS ALREADY PASSED ONCE THROUGH <name of gateway>.

>Which will look pretty ugly to people who see that in the body, but what
>else could be done to prevent the loop?

Unfortunately, this method is also very likely to prevent responses from 
that message to be gated. Once that string appears in the body, it is ripe 
for being quoted. Just as it was quoted in this response. 

You might think that you could simply look for that string at the start of
a line. But the "other medium" might not have the concept of "start of a
line"! Or it might change the lines somehow. Or something else. You cannot
rely on it appearing at the start of a line, or even remaining on one
line. In the same encoding. Or even in the same language.

Then you have to rely on the INCOMING gateway to look for this line and 
take appropriate action. If it does not, it will generate a news message 
with a different message id, which IS a DIFFERENT MESSAGE according to the 
news system. If your gateway treates messages with different message ids 
as the same message, it is broken.

If we are going to talk about gating articles into a horribly broken 
messaging system (one that has no concept of message identifiers), then we 
have to accept that it is broken and isn't our responsibility or within 
our ability to fix. Since it also requires enforcing structure on an 
entirely unstructured message element, it is impossible to accomplish.

If you thought the arguments were harsh when you wanted structure in an 
unstructured subject header, just think what it will be like when group A 
that uses "other medium" realizes that you are trying to enforce structure 
on THEIR messaging system's unstructured components.

Date: Tue, 14 Dec 2004 13:22:51 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

Thorfinn <thorfinn@xxxxxxxxxxxxxx>:

>Go back and read again.  Particularly regarding field order in the two
>algorithms.

What "two" algorithms? I proposed only one. 

> He doesn't use an algorithm which duplicates message ids. However, when
>compared to *your* algorithm, at some point, it is possible that your
>"no duplicates" algorithm will have generated IDs which happen to be
>identical to his "no duplicates" algorithm.

So, my algorithm produces duplicates because at some time in the future 
someone else's algorithm might duplicate something I've already produced? 
No, sorry. The duplicates are not produced by my algorithm. They come from 
a different algorithm. If that algorithm produces duplicates, then that 
algorithm is broken.

>Both, considered on their own, will generate unique IDs. 

Actually, not they don't both produce unique IDs. You've just said that
the second algorithm might produce an ID that has already been produced by
mine. Thus the second algorithm produces duplicates. But that's
irrelevant, since the discussion is about whether the algorithm I
suggested would do so. And it does not.

>So, where is your uniqueness?  Non existent, unless you can
>guarantee that you will own that domain name forever and never give it
>up.

Get a clue. I cannot guarantee uniqueness in the actions of someone else, 
if they choose to use an algorithm that can produce duplicates. Their 
choice to use such an algorithm is a) not under my control and b) not a 
failure in my algorithm. I CAN guarantee uniqueness in the algorithm I 
use.

>> Yep. Got that. Unfortunately, "MUST" and "unique" combine to zero. Either 
>> you don't mean "unique" or you don't mean "MUST". Which is it?

>We mean MUST, and we also accept that MUST applies in the universe of
>practicalities, where nothing is actually of zero probability, so "close
>enough" is good enough.

Until you show me an operating system that produces the same process id 
for two different processes, then the algorithm I presented has zero 
chance of producing duplicates. Even then, that same operating system 
would have to provide no means of locking such that unique sequence 
numbers could be produced, and then do all of this within one second.

Zero is better than "good enough", when zero is the goal. If anyone 
doesn't agree with that, please speak up.

>> >They are DIFFERENT message-ids. Yes?
>> Not as far as I can tell.

>Well, now that you've actually changed what Charles wrote, they look the
>same.  

Changing what someone said is a pretty serious claim, pal. You better have 
facts to back that up. I changed nothing. I cut and paste exactly what I 
saw. The two look identical to me. Do they look different to you?

><12345678@xxxxxxxxxxxxxxx>
><12345678@XXXXXXXXXXXXXXX>

>are different Message-IDs, though.

Yep. I've not said otherwise. In fact, I think I was one of the people who
was convincing Charles et.al. that the RHS is case sensitive. Did you 
think otherwise?

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>Consider two computers on the same LAN, mounting files from a common
>source by NFS. They might well be using the same pid at the same instant
>(once in a thousand years, maybe).

Yep. And if they are on the same lan, I will have given them different 
names, so the RHS will be different. On one system, one must guarantee 
that the LHS is unique; between systems, the RHS takes care of it.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: FWS problem
Date: Tue, 14 Dec 2004 18:46:23 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 61
Sender: [email-masked]

Charles Lindsey wrote:

> the whole point of "header content" is that it does not
> include the header name

Fine, that's what you have in the draft, we're now back to
square one.

>>> The header content of every header line (including the
>>> first and any that are subsequently folded) MUST contain
>>> at least one non-whitespace character.

>> Yes, that's the erroneous paragraph
[...]
> No, that is definitely the effect we want

Now that's also back to the start of this discussion.  In
other words "name: SP CRLF body CRLF" is invalid, because
the first line "name: SP CRLF" has no non-whitespace header
content, it has only a single trailing SP.

JFTR, Bruce said that this is okay.  Your draft says that
this is invalid.  If you are right the syntax in the draft
should be "name: SP *WSP header content CRLF" everywhere.

The syntax "name: SP [FWS] header content CRLF" is wrong,
because "name: SP CRLF SP header content CRLF" is illegal.

That's what I said in my first article about this problem:
replace [FWS] by *WSP before and after any header content,
you want FWS only *within* the header content.

> Things that will certainly break existing software are:
> name: body CRLF
> name: CRLF

Okay, then "name: FWS body CRLF" is unfortunately dead.

> some "helpful" intermediate site that thinks trailing
> white space is evil might well change it to
> "name: CRLF SP body CRL", at which point it _will_ break
> existing software.

My text editor does this when I forget to SET TRAILING ON.
Is it the duty of injecting agents to fix any "name: CRLF
SP body CRLF", i.e. insert the missing SP after the colon ?

It's a major difference from RfC 2822, at least gateways
should know this problem.  But it's probably better to say
that it's a duty of all injecting agents.

> I think some existing software also breaks with
> name: TAB body CRLF
> which is why we insist on SP.

In other words, the syntax of message/news is very different
from message/rfc822.  I have no problem with this consequence,
but please say so in the draft, and don't delete message/news.

                         Bye, Frank

To: LIST: [email-masked];
Xref: clerew local.usefor:20453
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: FWS problem
Date: Tue, 14 Dec 2004 11:49:25 GMT
Lines: 82
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:
> 
>> This specification uses the terms "header", "header name",
>> and "header content" which are synonymous with the [RFC2822]
>> terms "header field", "field name", and "field body" 
>> respectively.

>For your MUST you either need "header content = field name
>plus body", or simply use "header" (if that's name plus body):

No, the whole point of "header content" is that it does not include the
header name (if you want to refer to the header inclusing its name, then
we already have the term "header" for that). The question at issue is how
much of the FWS (including the obligatory SP) is to be included in the
term.

>> The header content of every header line (including the first
>> and any that are subsequently folded) MUST contain at least
>> one non-whitespace character.

>Yes, that's the erroneous paragraph.  You could replace it by

No, that is definitely the effect we want, given the intended meaning of
"header content".

>>> why this fuzz about a mandatory SP ?
> 
>> Because much deployed software breaks without it. ANd for
>> the same reason it is also required by the new NNTP draft.

>Please confirm, you're saying that "name: SP CRLF SP body CRLF"
>is okay, but "name: CRLF SP body CRLF" breaks some obscure NNTP
>servers ?  I have great difficulties to believe this, because
>for me it's "obvious" that trailing white space is always evil.

Things that will certainly break existing software are:

name: body CRLF
name: CRLF

It may well be that the case you mention "name: SP CRLF SP body CRLF" will
not actually break anything, but OTOH some "helpful" intermediate site
that thinks trailing white space is evil might well change it to
"name: CRLF SP body CRL", at which point it _will_ break existing
software.

So that is why we forbid empty content on the first line. RFC 2822 also
forbids enpty content on folded lines for essentially the same reason,
namely that the line "SP CRLF" might well be changed to "CRLF" by that
"helpful" intermediate site, whereupon it becomes indistinguishable from
the divede between headers and body.

>Is this NNTP draft something we can kill ?  The 2822 style of
>folding is much better than a mandatory trailing white space.

No. But the only point about it is that it requires the SP after the
name:. It does not concern itself with folding.

>Only the 2822 obs-style "name:body CRLF" is bad, but a simple
>"name: CRLF SP body CRLF" should be okay.  And all variants
>with TAB instead of SP of course.

I think some existing software also breaks with

name: TAB body CRLF

which is why we insist on SP.

To: LIST: [email-masked];
Xref: clerew local.usefor:20455
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Tue, 14 Dec 2004 12:03:51 GMT
Lines: 52
Sender: [email-masked]

>> Will the rest of the Working Group please tell me whether
>> it wants me to continue to reply in detail

>Maybe s/perverse/strange/ or odd, peculiar, idiosyncratic.

>I'm still not sure what you have in mind with a gateway saving
>Message-IDs as comments.  Maybe replace it by ten repetitions
>of "above all avoid loops".  Or add some jokes:  "If you're a
>news to avian carrier gateway make sure that you're the only
>one."  "While it's funny to gateway news to IRC you'll be shot
>if somebody else gateways IRC back to news (and a Message-ID
>in a comment is no excuse in this case)".

The best means for preventing loops is to ensure that Message-IDs are
propagated correctly. That is the first line of defence, but it sometimes
fails because message-ids get lost/changed/whatever somewhere in the other
medium. So we need a backup method.

The commonest backup is to insert an extra header in the gatewayed
article, such as:

X-Gatewayed-By: <name of gateway>

which the named gateway can then check for before gatewaying any message.

That is the second line of defence, but if fails if the other medium does
not allow X-headers of that nature, or even any headers at all, or if it
cannot be relied upon to propagate such headers reliably. So we need a
last-ditch backup, for use only in these really nasty situations.

And the only last-ditch possibility left is to include something in the
body of the message, such as:

THIS ARTICLE HAS ALREADY PASSED ONCE THROUGH <name of gateway>.

Which will look pretty ugly to people who see that in the body, but what
else could be done to prevent the loop?

To: LIST: [email-masked];
Xref: clerew local.usefor:20454
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Tue, 14 Dec 2004 11:52:52 GMT
Lines: 23
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>I read your example. Since the pid is part of the id, the only way that my 
>algorithm could duplicate message ids is for TWO processes to have the 
>same message id at the same epochtime AND for the locking mechanism that 
>prevents duplicate sequences to fail. I know of NO operating system that 
>assigns the same pid to two different processes, so even if the sequence 
>number fails, the pids will be different.

Consider two computers on the same LAN, mounting files from a common
source by NFS. They might well be using the same pid at the same instant
(once in a thousand years, maybe).

Date: Tue, 14 Dec 2004 11:09:25 +1100
From: Thorfinn <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

On Mon 13 Dec 2004 at 10:49:55AM -0800, in <[email-masked]>,
John Stanley <[email-masked]> wrote:
> Seth Breidbart <sethb@xxxxxxxxx>:
[snippage]
> >You might read my example more carefully.  
> I read your example. Since the pid is part of the id, the only way that my 
> algorithm could duplicate message ids is for TWO processes to have the 
> same message id at the same epochtime AND for the locking mechanism that 
> prevents duplicate sequences to fail. I know of NO operating system that 
> assigns the same pid to two different processes, so even if the sequence 
> number fails, the pids will be different.

Go back and read again.  Particularly regarding field order in the two
algorithms.

> >Because the new owner used a _different_ algorithm,
> What new owner? What so what if he uses an algorithm that will duplicate 
> message ids? We are talking about THIS algorithm, which does not.

He doesn't use an algorithm which duplicates message ids.  However, when
compared to *your* algorithm, at some point, it is possible that your
"no duplicates" algorithm will have generated IDs which happen to be
identical to his "no duplicates" algorithm.

Both, considered on their own, will generate unique IDs. Taken together,
they don't.  So, where is your uniqueness?  Non existent, unless you can
guarantee that you will own that domain name forever and never give it
up.

> >I'm saying that I'm not worried about sufficiently-low failure
> >probabilities. 
> Yep. Got that. Unfortunately, "MUST" and "unique" combine to zero. Either 
> you don't mean "unique" or you don't mean "MUST". Which is it?

We mean MUST, and we also accept that MUST applies in the universe of
practicalities, where nothing is actually of zero probability, so "close
enough" is good enough.

You don't accept that, that's fine.  Everyone else, AFAICT, does accept
that.  If anyone else doesn't, speak up now.

[snippage]
> "Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:
> >Consider the two message-ids:
> >   <12345678@xxxxxxxxxxxxxxx>
> >   <12345678@xxxxxxxxxxxxxxx>
> >They are DIFFERENT message-ids. Yes?
> Not as far as I can tell.

Well, now that you've actually changed what Charles wrote, they look the
same.  

<12345678@xxxxxxxxxxxxxxx>
<12345678@XXXXXXXXXXXXXXX>

are different Message-IDs, though.

Later,

  Thorf

-- 
<a href="http://tertius.net.au/~thorfinn/">[email-masked]</a>
"I miss the box full of gaffer tape that Neef used to have."
    -- Hobbes#vurt.net

Date: Mon, 13 Dec 2004 13:16:47 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Ad hominem
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>Will the rest of the Working Group please tell me whether it wants me to
>continue to reply in detail to John's long rants,

They are "rants" when you do not agree. They are "responses" when you make 
them. 

>he deliberately
>misreads/misunderstands what is said to him (or else he is even stupider
>than I thought),

Thanks for making your position on this discussion clear, Charles.

>I have replied fully to this one, but I see little point in continuing to
>do so in the future.

I see little point in you doing so. You either change the text so it says
the same wrong thing when I say something is wrong, or refuse to change
anything when I point out that it is unclear. Or abdicate responsibility.

>As usual, you fail to distinguish between what some particular agents are
>able to do, and what agents in general are able to do. SOME agents may be
>able to determine (or think they are able) that an address is "invalid",

Of course some might be able to actually do that for some addresses, and
we both know I've not said otherwise. Those agents are not the issue. The
issue is the agent you mention next -- those that THINK they are able to.
Telling them they SHOULD do something based on information they do not
have is wrong. It makes them think they can do something they cannot.
There is not an agent in the world that could determine anything about the 
validity of an address I use from one of the domains I own, so the ONLY
correct behaviour on their part is the MAY behaviour. MAY do the right 
thing, instead of MUST or SHOULD?

>It nowhere says that ALL agents
>are able to make that determination, which inevitably means that there
>will exist SOME agents that are unable to take advantage of that "MAY".

Where do you see in the draft anything about any limitation on what agents 
can make use of the "MAY" clause in that paragraph? What draft are you 
reading? 

>>Consequently, people who use perfectly valid addresses may well find their
>>articles rejected, because we've told server admins they can pretend to
>>know if they are valid or not.

>We have told server admins no such thing.

That's what that paragraph tells them. Maybe it isn't what you intend it 
to tell them, but it certainly does. When you say that something SHOULD be 
done with certain information, you imply that the information is 
available. When it is not, they ought not be doing anything different.

>For those who are confused by John's snipping, the discussion has suddenly
>moved on to communication with moderators. ...

In USENET, it's called "editing", and it is the normal thing to do. If you 
want the full context, read the original.

>That is the only implementation currently deployed on Usenet. You
>yourself agreed that other arrangements would only be available on Private
>networks. 

The fact that it is the only CURRENT implementation does not mean it must 
be the ONLY implementation ever. And no, I did not "agree" to anything of 
the sort. I gave one example that disproved the requirement for "by email" 
which involved what you call a "private network", but wasn't specified by
me as such.

It is possible that tomorrow a moderation system could be written to use 
anonymous ftp to accept articles. New and updated injecting agents could 
be written to do that directly; meanwhile, isc.org implements a mail->ftp
gateway to forward the email it gets to the ftp system.

>but for the moment the only
>universally accepted way on Usenet to send articles to a moderator is by
>email, 

Do you really think that removing the "by email" limitation in the
standard will cause things to break overnight? Will software suddenly
start, I don't know, printing messages on the line printer just hoping
they make it to the moderator? I don't think so.

But will a "MUST do X" in the standard allow people to think about better 
ways of doing things? Why would they? They MUST do it this way, why waste 
time thinking of ways to improve it?

So, nothing breaks if you take out "by email" and innovation would be 
allowed. Seems like a no-brainer.

>>No, it does not. "Take" means that something is removed from one place and
>>moved to another.

>Take v.t. to lay hold of; ...
>A very versatile word, as you can see.

And you dare accuse ME of "long rants". 

Walk up to any person on the street. Hold out a dollar bill and say 
"please take this dollar bill". Report back if ANY of them ask you to go 
into the nearby Kinko's so they can copy the dollar, instead of simply 
taking it out of your hand like you told them to.

"Copy" has no such connotation, and "copy" is what you are doing. Say 
"copy". After all, nobody asked you to make the change in the first place.

>I think, of the bunch listed above, "to ascertain something from"
>fits my defined meaning best.

And "copy" fits it better. You aren't "ascertaining" anything, you are 
copying it.

>A followup agent is, by definition, a special case of a posting agent, so
>its duties are not complete until it has handed over the final text to an
>injecting agent.
 
It does not, however, have any duties CHANGING the subject, newsgroups,
or other headers after it has presented the article to the user. Thus,
its duties in that regard are solely to copy the various contents of the 
precursor and then let the user fiddle with them. It does nothing with 
them other than pass them to the injecting agent after that. 

Thus, the ONLY action specified for the followup agent when preparing the 
headers is the DEFAULT action BY DEFAULT, and repeating this "by default"
is meaningless. 

>>>But OK, I will buy that, except that it has to be the "last two" (i.e. you
>>>MUST keep the last one from the precursor).

>>Why? What breaks if the parent of the precursor is no longer threaded? 
>>Justify the MUST.

>The original text said "... but the first and last message identifiers
>from the precursor MUST NOT be removed". So if the precursor had:

>References: <A> <B> <C>
>Message-ID: <D>

>then the followup article will have

>References: <A> <B> <C> <D>
>Message-ID: <E>

Yes, that's right. So why must <C> be kept? What breaks? 

>The original wording allowed the trimming of <B>, but not of <A> or <C>.

Yes, the original wording was wrong and I thought we were changing it. 

>The new wording allows precisely the same. 

So I'll ask again. What breaks if <C> is removed? I understand that <D> 
MUST be kept (to keep the thread). <A> to link everything to the parent.
But WHY is there a MUST for keeping <C>?

>You asked me for a change of
>wording, not for a change of meaning, so that is what I did.

What kind of game are you playing? I pointed out that the text was WRONG.
That's not asking you for the same WRONG text written a different way.
Anyone with a brain would know that means CHANGE THE MEANING.

>If somebody wants a change of meaning, then that is a separate issue, to
>be discussed. However, allowing both <B> and <C> to be trimmed away might
>well cause threading with the earlier articles to fail if the precursor
>had failed to be delivered. 

Wrong. That's why <A> is kept. If nether B nor C have arrived, they cannot
be threaded against anyway. And you can extend your same wrong argument
back three, five, or even eightythree levels. That's not an excuse for a
MUST.

>There was a time when our draft suggested keeping at least 20, plus the
>first one (and USEAGE still says that, although some recent experiences
>with excessively long References breaching the 998 limit makes me think 20
>is too large). So I could easily be persuaded that keep "last two" should
>actually be increased to, maybe, "last three".

So I could easily be persuaded to keep asking WHAT BREAKS? What breaks?

>>Then say that. But that's so obvious that I don't think it needs to be 
>>said. That whole section about "reading agent" is a definition seeking a 
>>purpose.

>Maybe so, but that is a separate issue for separate discussion (there
>exist some people wanting to keep the whole section and some to remove
>it).

No, that is an issue for the current discussion, since it deals with the 
paragraph being discussed. What function does the current paragraph serve?
It make general statements about something we actually have no control 
over. Why bother? Define "reading agent" if you have to and leave it at 
that.

>But what you asked me to do, and which I did do, was to weaken the
>over-strong wording thjat was originally present. You said nothing then
>about removing the whole section.

You said nothing ten years ago about making any changes to the draft. Yes, 
seeing the wording made it clear that the section was irrelevant and ought 
to be removed completely.

>Now you have suddenly switched to gateways. ...

I didn't "suddenly" do anything, Charles. I got to the point in the 
discussion where we were talking about gateways.

>>The only content to the entire first paragraph of that section is to say:
>>loops are possible, please prevent them. We've already said that. There is 
>>no need for the entire paragraph. It adds no meaning to the draft, so 
>>while it may be completely true, it is also completely meaningless.

>That text was originally written by Russ. You have to persuade him if you
>want it taken out.

No, I don't. Are YOU the editor or are you NOT? You will gladly change his
text when you feel like it; when you don't, you create fictional
requirements for outside approval.

>You have already said that, and I have already changed it at your request.
>Why are you still complaining? It now says (as I have posted previously):

It is still irrelevant to the section of the document it is in (Duties), 
since it lists no duties! It was covered by an earlier section, so it is 
redundant. Changing it so it is still redundant does not solve the base 
problem.

>>That kind of loop is easy to catch: simply do not allow an outgoing 
>>gateway to accept incoming messages.

>Since the whole purpose of the outgoing gateway (which is presumably
>attached to some serving agent) is normally to gateway the whole of some
>newsgroup to the "other medium", then if it (or rather its serving agent)
>never receives any messages it will never gateway anything.

Are you that stupid, or are you being deliberately dense? I said "incoming
messages". The word "incoming" has a meaning, and it is important, just 
like it is important when talking about "incoming gateways". For a 
news gateway, an INCOMING message would be one that comes from the 
other medium. You know, something that an INCOMING gateway would process.

The simple way to prevent loops from INCOMING messages in an outgoing 
gateway is to simply not allow them. No comments in the body of the 
outgoing message are necessary.

The outgoing gateway has the message id of the news message, so it 
does not need to scan the body of the messages hoping for structure, it 
just looks at the MANDATORY message id.

>>In fact, if a gateway accepts INCOMING messages, then it is an INCOMING 
>>gateway, and is dealt with elsewhere.

>NO! 

What, it is NOT dealt with elsewhere? Or an incoming gateway doesn't deal 
with incoming messages?

>Incoming messages don't just come from incoming gateways.

I didn't say they did. Incoming messages are dealt with by incoming 
gateways BY DEFINITION. They don't come from incoming gateways, they are 
processed by them. Incoming messages come FROM the other medium. They are 
coming IN to news. 

>They arrive by the usual Usenet relaying mechanisms.

An incoming message to news that arrives via the usual USENET relaying 
mechanisms is not going through a gateway. 

>>>>However, if you meant that the same outgoing gateway is going to 
>>>>see the same message outgoing more than once, then it already has the 
>>>>mandatory message id (not in the body) to work with.

>>>There might not be such a thing as a message id in the other medium. 

>>There is a MANDATORY message id in news.

>There is no MANDATORY message id in the other medium.

The outgoing gateway is processing NEWS articles and sending them out. In 
NEWS, there is a MANDATORY message id. If you are concerned that the 
outgoing gateway is going to see the same NEWS message more than once and 
send it out again, then the gateway solves THIS problem by keeping track 
of the message id OF THE NEWS ARTICLE. It's pretty simple. About three 
lines of perl code and the problem is solved. It doesn't matter if the 
"other medium" has no concept of message id, news does, and it is 
MANDATORY. If the outgoing gateway doesn't find a message id in the 
headers of the news message it is gating, then it drops it. It isn't a 
news message.

>Then what do YOU want outGate to do with the News message-id? 

I don't care what it does with the news message id. And that is not the 
issue that started this discussion. You claimed that putting the message 
id as a comment in the body of the outgoing message prevented loops. It 
does nothing of the sort. 

>And how can
>you be sure that, wherever you hide it, inGate will subsequently put it
>back in the News message-id before handing it over to Site B?

Who said anyone could be sure? And how can YOU be sure that by hiding it 
in the body of the message that inGate will do anything with it? You must 
be sure if you are so sure that putting it there will prevent loops.

>You have to
>allow for the possibility that the 2nd time outGate sees the message it
>will have a different message-id.

If it has a different message id, by definition, it is a different 
message. That's what news says. You cannot otherwise determine the 
difference between an article posted twice and the same message arriving 
twice.

If you want anything else, then you've just put a requirement on the 
structure of the body of the "other medium" in a draft that covers news. 
Will the FIDO or whoever else is involved groups appreciate you doing 
this, and would it be binding on them anyway?

>>So you expect outgoing gateways to parse the BODIES of the messages it 
>>converts looking for things that look like something it understands?

>Yes, if it can't find a better way to do it (which hopefully it can).

An undefined structure in an unstructured message element is not a very 
good way of defining something. 

>But fortunately inGate is in a position to invent the little bit of
>structure that it needs.

inGate can invent whatever structure it wants, but if the other medium 
also believes that the body is unstructured, it is violating their 
standards. And even when it invents this structure, who is to say that it 
is the same structure that outGate imposes? outGate can put all the 
comments in the body that it wants, if nobody knows how to decipher them, 
or deciphers them differently, nothing is gained. 

>>>It is nowhere stated that a separate injecting agent is involved in an
>>>incoming gateway. 

>>Nowhere is it stated that it is not.

>True. So what?

So by making the gateway responsible for ALL compliance you've ruled it
out implicitely. We apparently did not want to rule it out, so the 
statement about it being responsible must be incorrect.

>>By trying to inject it and getting a rejection, because the duplicate ID 
>>is already in the history. 

>Injecting agents don't keep histories.

The agents that they inject to DO, and just what did you think was meant
by "by trying to inject it" if it wasn't that it tried to inject it?

>With a bit of luck their friendly
>neighbourhopod serving agent will keep one and silently reject it.

A bit of luck? By the way, how can the injecting agent know that an 
article has already been injected unless it has a history available to it?
You say it MUST NOT forward such an article to a moderator, and it cannot 
know how to obey that MUST NOT without a local history (or one always 
available to it). It could, I suppose, be a history of just articles 
appearing in moderated groups, but that would be more difficult to 
maintain than a full history.

>But that is no use if the message-id got changed to a different message-id
>within the other medium.

Yes, we cannot prevent problems in other media. We know this.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Mon, 13 Dec 2004 21:51:06 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 17
Sender: [email-masked]

Charles Lindsey wrote:

> Will the rest of the Working Group please tell me whether
> it wants me to continue to reply in detail

Maybe s/perverse/strange/ or odd, peculiar, idiosyncratic.

I'm still not sure what you have in mind with a gateway saving
Message-IDs as comments.  Maybe replace it by ten repetitions
of "above all avoid loops".  Or add some jokes:  "If you're a
news to avian carrier gateway make sure that you're the only
one."  "While it's funny to gateway news to IRC you'll be shot
if somebody else gateways IRC back to news (and a Message-ID
in a comment is no excuse in this case)".

                          Bye, Frank

Date: Mon, 13 Dec 2004 10:56:55 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Address Validation
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>I am certain that nobody cares that this is the case.

>Then why make all this fuss? 

Because _I_ care if the standard says contradictory things and tells 
people they can pretend to know things they cannot.

>>This solves the problem of de-facto invalidation of perfectly valid 
>>addresses exactly how? 

>It was not intended to solve the problem of de-facto invalidation of
>perfectly valid addresses. 

So it was a rewrite for rewrite's sake, making it look like you were 
trying to compromise when none was intended and none was offered. Typical.

>>And "perverse"? Foof.

>You think it would not be perverse?

It would be stupid, it would be wrong, and it would be a conforming
application. So yes, in THAT sense, the entire thing is perverse, but
standing by itself, no. You'd have to know intent to label it thus, and
you have no knowledge of the server admin's intent, especially when it
might be a server admin who has net yet been born.

Date: Mon, 13 Dec 2004 10:49:55 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

Seth Breidbart <sethb@xxxxxxxxx>:

>You are making the assumption that a new domain owner will continue to
>use the same algorithm. 

No, I don't care what the "new domain owner" decides to do. We cannot 
prevent people from choosing to duplicate message ids, we can only specify 
that they MUST NOT do so. And we can point out non-conforming 
implementations, like those that are designed to do so. And we can look at 
conforming algorithms, like the one I showed.

>> There is no other owner at that epochtime. 

>So what?

So that algorithm will not duplicate message ids. Some other algorithm 
might, but that does not make mine broken.

>>>...and gets his pids randomly from 32-bit space (to avoid potential
>>>attacks based on known-pids).  Oops.
>>
>> I'm interested in hearing which operating system allows two different 
>> processes to have the same process id at the same time.

>You might read my example more carefully.  

I read your example. Since the pid is part of the id, the only way that my 
algorithm could duplicate message ids is for TWO processes to have the 
same message id at the same epochtime AND for the locking mechanism that 
prevents duplicate sequences to fail. I know of NO operating system that 
assigns the same pid to two different processes, so even if the sequence 
number fails, the pids will be different.

>Because the new owner used a _different_ algorithm,

What new owner? What so what if he uses an algorithm that will duplicate 
message ids? We are talking about THIS algorithm, which does not.

>> The question was about a MUST be unique clause, which doesn't actually
>> need to be unique.

>It does need to be unique.  The protocol fails when it isn't unique.

Well, the algorithm that brought this up doesn't guarantee unique ids. 
Thus it does not meet the "MUST" requirement. If it is "good enough", then 
you are saying that the MUST really isn't a must, and thus the "unique" 
doesn't really need to be unique.

>I'm saying that I'm not worried about sufficiently-low failure
>probabilities. 

Yep. Got that. Unfortunately, "MUST" and "unique" combine to zero. Either 
you don't mean "unique" or you don't mean "MUST". Which is it?

>You say that you won't accept them, except that your
>examples of what is acceptable to you still have non-zero failure
>probabilities.

Until you can show me an operating system that assigns the same pid to two 
different processes, I'd say that the odds of a duplicate using my 
algorithm are zero.

>> If mozilla creates message ids for a domain that may never see the
>> message at all, then it cannot guarantee it will be unique.

>I own a domain that does not run a news server. That domain will never 
>see a usenet message. 

That's nice. What does it have to do with the news system?

>Yet use of it in the RHS of a Message-Id
>by me can certainly guarantee uniqueness (modulo other people doing
>stuff they're not entitled to, which can break any algorithm).

That's very nice. Also completely irrelevant. The issue you are 
responding to here is a client creating message ids IN SOME OTHER DOMAIN,
which may never see the message at all. Your client creating ids in
YOUR domain is not the same thing, obviously. Please pay attention.

Richard Clayton <richard@xxxxxxxxxxxxxx>:

>there also appears to be an assumption that "domain" is the same as
>"host" 

Yes, and I should have corrected that earlier. Instead of 
fully.quallified.domain.name, I said "domain". I thought it would be 
understood. My mistake.

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>Consider the two message-ids:

>   <12345678@xxxxxxxxxxxxxxx>
>   <12345678@xxxxxxxxxxxxxxx>

>They are DIFFERENT message-ids. Yes?

Not as far as I can tell.

Date: Mon, 13 Dec 2004 12:43:53 -0500 (EST)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

On Mon, 13 Dec 2004, Charles Lindsey wrote:
> Will the rest of the Working Group please tell me whether it wants me to
> continue to reply in detail to John's long rants...

I long ago stopped reading John's endless screeds, never mind replying to
them, and think the list would be better off if everyone ignored them
until he learns to accept defeat on his old issues and present any new
ones clearly and concisely. 

                                                          Henry Spencer
                                                       [email-masked]

To: LIST: [email-masked];
Xref: clerew local.usefor:20445
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Mon, 13 Dec 2004 16:55:22 GMT
Lines: 463
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

Will the rest of the Working Group please tell me whether it wants me to
continue to reply in detail to John's long rants, given that he keeps
repeating the same complaints ad nauseam, he deliberately
misreads/misunderstands what is said to him (or else he is even stupider
than I thought), and jumbles up unrelated topics so that it is hard to
spot what he is talking about at any given moment.

I have replied fully to this one, but I see little point in continuing to
do so in the future.

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>No, it implies no such thing. Agents may or may not be able to detect
>>validity/invalidity of various classes of address, as Seth has pointed
>>out. The "MAY" permits them to make use of such information as they may
>>have available.

>Since they do not have the information that says "your address is invalid" 
>in the general case, there is no use to be made of that "information". 
>Pretending they CAN know this is the problem. 

As usual, you fail to distinguish between what some particular agents are
able to do, and what agents in general are able to do. SOME agents may be
able to determine (or think they are able) that an address is "invalid",
in which case they MAY choose to reject. It nowhere says that ALL agents
are able to make that determination, which inevitably means that there
will exist SOME agents that are unable to take advantage of that "MAY". So
what?

>Consequently, people who use perfectly valid addresses may well find their
>articles rejected, because we've told server admins they can pretend to
>know if they are valid or not.

We have told server admins no such thing.

For those who are confused by John's snipping, the discussion has suddenly
moved on to communication with moderators. ...

>>No, there is nothing else that will nearly always work on Usenet as it
>>presently exists.

>Well, DOH! But that is not an excuse for preventing it from being created.
>There is no reason to demand in the standard that "by email" be the only
>method of sending articles to the moderator. If that is the only current 
>implementation, that's one thing, but to close out any other 
>implementation is just ridiculous.

That is the only implementation currently deployed on Usenet. You
yourself agreed that other arrangements would only be available on Private
networks. A Private network is, by definition, a "cooperating subnet"
which is, by definition, permitted to use additional protocols not defined
by the standard. Such additional protocols may, at some future date,
become defined as extensions to the standard; but for the moment the only
universally accepted way on Usenet to send articles to a moderator is by
email, and no alternative mechanism is on the table; that is therefore
what we are going to standardize.

>I'm sorry, but in another section of the draft you try to "standardize" 
>the "exceptional" cases of reinjection.

That is because there was some small existing usage of reinjection,
because there were good reasons why it should happen in a small number of
situations, and because we discussed it and concluded that the possibility
should be mentioned (because there needed to be some restrictions on
exactly how it was done). None of those preconditions applies in the case
of novel methods of communicating with moderators.

Now we have switched to Injection-Date. ...

>>>>So if the Injection-Date is earlier than the expiry date
>>>>(however computed) that it implements (probably of a per group basis), 
>>>>then it MUST drop it.

>>>I asked once, and I expected an answer. Why is this a MUST? Expiration 
>>>policy is the site's responsibility, not ours. 

>>This is nothing to do with controlling the expiration dates of sites. It
>>is a matter of preventing loops, for which purpose that MUST is essential.

>I see nothing in that statement about "loops", and everything about 
>"expiry date". 

The word "expiry" does not appear in the text under discussion. That text
was an agreed part of the protocol as discussed when we introduced the
Injection-Date header, and before we added that header it was an agreed
part of the protocol applying to the Date header.

Note that the critical date for a site is that of "the earliest articles of
which it normally keeps record", which will usually be earlier than that
of the earliest expired article, since it is usual to keep message-ids in
the history file for a further period even after those articles have been
expired. So if an article arrives with an even earlier Injection-Date than
that, then it MUST be dropped.

Now we are back on ".invalid" again. ...

>>... both Frank and Seth seem to he saying that it is about
>>right as I currently have it. 

>>I agree that the
>>way it is currently worded seems to be pulling two ways at once and will
>>probably need to be changed. But I need guidance as to which direction to
>>change it in.

>Well, that's what I've been giving you. 

And other people have been giving different guidance.

Now we have switched to "take". ...

>>>There is no reason to redefine a standard english word when there is 
>>>already a standard english word that conveys the correct meaning.

>>"Take" is also a standard english word, and conveys the intended meaning
>>in this case. 

>No, it does not. "Take" means that something is removed from one place and
>moved to another.

Take v.t. to lay hold of; to get into one's possession: to seize: to
catch: to capture: to captivate: to receive or come to have willngly or by
an act of one's own; to appropriate: to assume, adopt: to accept: to
receive: to admit: to submerge (scot.): to have normally assigned to one:
to find out, come upon, surprise, detect: to swallow or inhale: to apply
to oneself: to obtain: to engage, secure: to seek and receive: to have
recourse to: to attend a course in: to visit: to call for, necessitate,
use up: to remove: to cause to go: to subtract: to convey: to escort: to
detract: to derive: to understand: to apprehend: to assume, suppose: to
mistake: to conceive: to accept as true: to tolerate: to ascertain: to
observe or measure: to ascertain something from: to execute, perform: to
set down: to portray: to photograph: to charge oneself with: to
asseverate: to strike: to come upon and affect: to bewitch: to blight: to
deliver, give (obs.): to betake - and then there are the intransitive
forms ...

A very versatile word, as you can see. To determine its meaning in any
given case, you have to examine its context. Fortunately, in this case,
the context is clear, because I have given an explicit definition of the
term.  I think, of the bunch listed above, "to ascertain something from"
fits my defined meaning best.

Switching now to followup agents. ...

>>>Nonsense. We tell it what it must use. We do NOT tell the user what he 
>>>uses. Since there is only ONE thing that the followup agent is allowed to 
>>>do ("copy the content") that is the default. There is no other default.

>>There are TWO things the followup agent can do. One is to accept (without
>>demur) what the user types in. 

>The user has typed NOTHING in at the point in question. We are telling it 
>what it must do when it prepares the article for user action. There is 
>only ONE THING that it does -- and that means that ONE action is the 
>default. Saying that it must, by default, do the only thing we tell it it 
>can do is just silly. 

A followup agent is, by definition, a special case of a posting agent, so
its duties are not complete until it has handed over the final text to an
injecting agent. Therefore, its duties include accepting text from the
user and using it in place of whatever default it might otherwise have
used. Our draft carefully defines the allowed defaults and, in the
circumstances "default" is a perfectly proper word to use.

And now References. ...

>>>You want to trim the one in the followup. "If the resulting 
>>>References-header is excessively long, it may be trimmed by removing 
>>>message identifiers other than the first and the last."

>>But OK, I will buy that, except that it has to be the "last two" (i.e. you
>>MUST keep the last one from the precursor).

>Why? What breaks if the parent of the precursor is no longer threaded? 
>Justify the MUST.

The original text said "... but the first and last message identifiers
from the precursor MUST NOT be removed". So if the precursor had:

References: <A> <B> <C>
Message-ID: <D>

then the followup article will have

References: <A> <B> <C> <D>
Message-ID: <E>

The original wording allowed the trimming of <B>, but not of <A> or <C>.
The new wording allows precisely the same. You asked me for a change of
wording, not for a change of meaning, so that is what I did.

If somebody wants a change of meaning, then that is a separate issue, to
be discussed. However, allowing both <B> and <C> to be trimmed away might
well cause threading with the earlier articles to fail if the precursor
had failed to be delivered. Consider the case where <C> and <E> are
available, bur <D> is missing, for example.

There was a time when our draft suggested keeping at least 20, plus the
first one (and USEAGE still says that, although some recent experiences
with excessively long References breaching the 998 limit makes me think 20
is too large). So I could easily be persuaded that keep "last two" should
actually be increased to, maybe, "last three".

Now we are at Duties of a Reading Agent. ...

>>The only limitations are those imposed by the lack of appropriate
>>facilities in the reading agent. The whole point of the wording is to
>>indicate that the poster cannot assume that every reading agent will be
>>able to make sense of everything, however bizarre, that he might choose to
>>put in the article.

>Then say that. But that's so obvious that I don't think it needs to be 
>said. That whole section about "reading agent" is a definition seeking a 
>purpose.

Maybe so, but that is a separate issue for separate discussion (there
exist some people wanting to keep the whole section and some to remove
it).

But what you asked me to do, and which I did do, was to weaken the
over-strong wording thjat was originally present. You said nothing then
about removing the whole section.

Now you have suddenly switched to gateways. ...

>>>We already cover the possibility of loops, so there is no need for this 
>>>paragraph at all.

>>There are various ways in which loops can arise. It is necessary to draw
>>attention to them.

>We already draw attention to them, in the previous section where it is 
>stated as "the primary dictat": Above all, prevent loops.

>The only content to the entire first paragraph of that section is to say:
>loops are possible, please prevent them. We've already said that. There is 
>no need for the entire paragraph. It adds no meaning to the draft, so 
>while it may be completely true, it is also completely meaningless.

That text was originally written by Russ. You have to persuade him if you
want it taken out.

> But it 
>is wrong in that it says that the limits are due to the PRESENCE of 
>another incoming gateway, and that is not true. The operation MUST 
>consider the POSSIBILITY of an incoming gateway, not just the PRESENCE of 
>one.

You have already said that, and I have already changed it at your request.
Why are you still complaining? It now says (as I have posted previously):

7.9.1.  Duties of an Outgoing Gateway

   From the perspective of Netnews, an outgoing gateway is just a
   special type of reading agent. The exact nature of what the outgoing
   gateway will need to do to articles depends on the medium to which
   the articles are being gated. The operation of the outgoing gateway
   is subject to additional constraints due to the possibility of one or
   more corresponding incoming gateways back from that medium to
   Netnews, since this opens the possibility of loops.

>So, to solve the problem, and make things shorter (something you like to 
>do), remove it. One paragraph shorter.

>>>It is unlikely that an outgoing gateway is going to see an incoming 
>>>message.

>>On the contrary, that is just the kind of loop we are trying to catch.

>That kind of loop is easy to catch: simply do not allow an outgoing 
>gateway to accept incoming messages.

Since the whole purpose of the outgoing gateway (which is presumably
attached to some serving agent) is normally to gateway the whole of some
newsgroup to the "other medium", then if it (or rather its serving agent)
never receives any messages it will never gateway anything. Normally,
these will be messages that have arrived via other relaying agents in the
normal manner (and that includes the ones which might turn out to be parts
of loops).

> Problem solved. In fact, I doubt 
>there are many outgoing gateways that do accept incoming messages. I know 
>the ones I have written do not. There is, in fact, no way to SEND an 
>incoming message to my outgoing gateway. 

So your outgoing gateway never has any messages to process at all then?
Why did you go to all that trouble to write it?

>In fact, if a gateway accepts INCOMING messages, then it is an INCOMING 
>gateway, and is dealt with elsewhere.

NO! Incoming messages don't just come from incoming gateways. They arrive
by the usual Usenet relaying mechanisms. Consider the following setup:

In Usenet:

    Site A ---> Site B ---> Site C ---> Site D --->
                  ^                       |
                  |                       |
                  | inGate                | outGate
In the other      |                       |
medium:           |                       |
                  |                       |
    Site E <--- Site F <--- Site G <--- Site H <---

>>>However, if you meant that the same outgoing gateway is going to 
>>>see the same message outgoing more than once, then it already has the 
>>>mandatory message id (not in the body) to work with.

>>There might not be such a thing as a message id in the other medium. 

>There is a MANDATORY message id in news.

There is no MANDATORY message id in the other medium. Maybe even no
message-id concept at all.

>An outgoing gateway takes NEWS 
>articles and sends them to the other medium. If you think an outgoing 
>gateway author is going to throw the message id away prior to checking for 
>dupes just because the "other medium" doesn't understand message ids, then 
>you are being foolish.

Then what do YOU want outGate to do with the News message-id? And how can
you be sure that, wherever you hide it, inGate will subsequently put it
back in the News message-id before handing it over to Site B? You have to
allow for the possibility that the 2nd time outGate sees the message it
will have a different message-id.

>>> A comment in the body 
>>>of a messsage is not likely to prevent loops. Saying it does is silly.

>>No. If the gateway to the "other medium" (which presumably understands the
>>rules for contructing messages in the other medium) can manage to insert
>>some distinct identifier into whatever passes for a body in the other
>>medium, then it is quite capable (and indeed it is the only agent in the
>>whole world with that capability) of recognizing the identifier that it
>>itself placed there, should it ever come back again. 

>So you expect outgoing gateways to parse the BODIES of the messages it 
>converts looking for things that look like something it understands?

Yes, if it can't find a better way to do it (which hopefully it can).

> Well 
>now, that's a serious requirement. But since the content of the body is 
>UNSTRUCTURED, you've got a long row to hoe defining structure of headers 
>in the body. But before you can claim that a comment in the body of a 
>message (in some other medium than news, to boot) is likely to prevent 
>loops, you need that structure.

But fortunately inGate is in a position to invent the little bit of
structure that it needs.

>>It is nowhere stated that a separate injecting agent is involved in an
>>incoming gateway. 

>Nowhere is it stated that it is not.

True. So what?

>>If there is, then it can be relied on to do the checks.

>Not when we tell the gateway author that the gateway author that the 
>articles the gateway forms must follow all the standards.

Why ever not? It has the responsibility of causing the necessary checks to
be done. It has a choice. Either it does them itself, or it ensures that
the injecting agent it hands off to will do it. What is so wrong with
that? It is not our job to tell implementors how to do it - just to tell
them that it must be done somehow.

>>>It takes a bit of thought there to realize that a duplicated message id is 
>>>an illegal content for the message id header. And most injecting agents 
>>>do, indeed, reject duplicates.

>>How?

>By trying to inject it and getting a rejection, because the duplicate ID 
>is already in the history. 

Injecting agents don't keep histories. With a bit of luck their friendly
neighbourhopod serving agent will keep one and silently reject it.

But that is no use if the message-id got changed to a different message-id
within the other medium.

>>>>I would suggest checking for and noting the presence of the cancel message
>>>>_before_ removing it.

>>>There is no cancel message before removing it. I am told I must remove it 
>>>before I can inject it.

>>Eh?

>I have an incoming gateway for group X. User Y sends a message to the 
>gateway containing the header "Control: cancel <[email-masked]>". My 
>gateway is told I MUST NOT pass that message without removing or renaming 
>that header. On the other hand, since I can determine that this is a valid
>cancel message, I can create a cancel message of my own. How? By passing
>the message WITH the control header. 

You are postulating another medium whose cancel messages are in the same
format as Netnews cancel messages. I agree that a typical mail2news
gateway has this property.

Yes, if you follow the draft, you notice two things:

1. The incoming message is a cancel message, so you generate an equivalent
(brand new) News cancel message.

2. The incoming message contains a Control-header, so you remove it before
you pass it on (so you have now inserted two messages into the News
system). OTOH, in the case of a pure cancel message there is little point
in passing on the emasculated message (which is why I used a message with
a Supersedes header in my example) so maybe you just drop it.

And yes, in this very particular case of a mail2news gateway there is an
obvious optimization just to let it through. But strictly speaking that is
wrong. When Russ wrote this, he concluded that letting Control messages
through as received was a Bad Thing. If you disagree with him, then please
try to persuade him otherwise. Otherwise, I will not change it.

>Now, YOU told me that I was supposed to look for a cancel message before I
>remove the header. I ask my server for that message and, not surprisingly,
>since I have not yet posted one, it does not yet exist. So how does this 
>help the problem? If the cancel message already existed, yes, I'm out of 
>the woods, I don't have to process the incoming message. But since it does 
>not, and I MUST NOT/SHOULD process it, I'm stuck.

You are in fact hoist with your own petard. Nobody required you to
implement your gateway in such a stupid way.

>>If an article with a Supersedes header arrives at an outgoing gateway,

>So what? We're talking about duties of an INCOMING gateway here.

The draft has discouraging things to say about control messages for BOTH
kinds of gateway.

To: LIST: [email-masked];
Xref: clerew local.usefor:20443
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Mon, 13 Dec 2004 13:17:34 GMT
Lines: 73
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>That might be true, or let's say I don't care.  But we have a
>practical problem with some very old UAs like my "Mozilla 3":

>This monster creates Message-IDs based on the RHS of the From:
>address.  Now we tolerate something like spamhater@invalid as
>an "address".  With these UAs that would result in Message-IDs
>like <41AE974D.1DCF@invalid>.  We can't have that.

But sadly you have got it :-( . That implementation is broken (i.e. the
probability that it will go wrong is way way worse than once in a
thousand years). It already contravenes our draft (more specifically, it
contravenes RFC 2822), so changing our draft cannot fix it.

>Of course the UA is broken, but it's another argument for RHS =
>domain as specified in 2821.

Why? You are asking for further changes. Others are asking for the minimal
change from RFC 2822. Which am I supposed to do?

>>> Message-ID <xyz@news-fe-01>, and the consensus was that it's
>>> wrong.  The ..!news-fe-01!.. in his Path: wasn't much better.

>...  This RHS was
>"identified" as invalid, because it's no domain.

But it is quite likely to be unique to that site, given that it was
considered suitable as a path-identity. It is not necessarily wrong.
Please bear in mind also that Usenet sites are not obliged to possess
domain names. A lot of news (and even some email) is still transported by
UUCP and other protocols.

>> But the id-right is NOT case-sensitive, whether it looks like
>> it came from a domain-name or not (I know Bruce will disagree
>> with this, but he is wrong).

>No, I disagree, and you are wrong, it _is_ now case-sensitive,

Yes, those two words "sensitive" and "insensitive" are just too similar. I
should have said

 But the id-right is NOT case-insensitive, whether it looks like
 it came from a domain-name or not (I know Bruce will disagree
 with this, but he is wrong).

Consider the two message-ids:

   <[email-masked]>
   <[email-masked]>

They are DIFFERENT message-ids. Yes?

>So the last thing I'm unhappy with is the very complex special
>syntax for the LHS, especially the idea to allow any NO-WS-CTL
>where it never before was allowed in news.  It's just bad for
>the news URL.

Then persuade someone else on this list to agree to the further
restrictions you are asking for. If there is consensus, then I shall
change it.

To: LIST: [email-masked];
Xref: clerew local.usefor:20444
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Address Validation (Was: Re: draft-ietf-usefor-usepro-02)
Date: Mon, 13 Dec 2004 14:15:21 GMT
Lines: 45
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>Thorfinn <thorfinn@xxxxxxxxxxxxxx>:

>>I also think that nobody else cares that this is the case,

>I am certain that nobody cares that this is the case.

Then why make all this fuss? 

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>So how about, for injecting agents:

>>   2. It SHOULD verify that the article is from a trusted source, and
>>      MAY reject articles in which headers contain unverified email
>>      addresses, that is, addresses which are not known to be valid for
>>      the trusted source, though it would be perverse to reject
>>      intentionally unverifiable addresses such as those ending in
>>      ".invalid" (7.5).

>This solves the problem of de-facto invalidation of perfectly valid 
>addresses exactly how? 

It was not intended to solve the problem of de-facto invalidation of
perfectly valid addresses. Others (Thorfinn in particular) said they were
perfectly happy that a site might reject an article with a from address
which satisfied *its* concept of invalidity, irrespective of the fact
that mail sent to that address might well be delivered correctly.

>And "perverse"? Foof.

You think it would not be perverse?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: FWS problem
Date: Mon, 13 Dec 2004 12:29:57 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 59
Sender: [email-masked]

Charles Lindsey wrote:
 
> This specification uses the terms "header", "header name",
> and "header content" which are synonymous with the [RFC2822]
> terms "header field", "field name", and "field body" 
> respectively.

For your MUST you either need "header content = field name
plus body", or simply use "header" (if that's name plus body):

> The header content of every header line (including the first
> and any that are subsequently folded) MUST contain at least
> one non-whitespace character.

Yes, that's the erroneous paragraph.  You could replace it by

| The header content of every header line (including any that
| are subsequently folded) MUST contain at least one
| non-whitespace character.

That's for the definition "header content = field name + body".
Or just say this without changing the existing definitions:

| Every header line (including any that are subsequently
| folded) MUST contain at least one non-whitespace character.

It's shorter, it must be correct... ;-)

>> why this fuzz about a mandatory SP ?
 
> Because much deployed software breaks without it. ANd for
> the same reason it is also required by the new NNTP draft.

Please confirm, you're saying that "name: SP CRLF SP body CRLF"
is okay, but "name: CRLF SP body CRLF" breaks some obscure NNTP
servers ?  I have great difficulties to believe this, because
for me it's "obvious" that trailing white space is always evil.

If my server refuses to accept extremely long References: I'm
sometimes forced to edit the file "outbox" manually.  And my
text editor strips all trailing spaces.  I know that I must
watch signatures, but I never considered any folded headers.

Is this NNTP draft something we can kill ?  The 2822 style of
folding is much better than a mandatory trailing white space.

Only the 2822 obs-style "name:body CRLF" is bad, but a simple
"name: CRLF SP body CRLF" should be okay.  And all variants
with TAB instead of SP of course.

Instead of copying excessively bad ideas like NO-WS-CTL in a
Message-ID from RfC 2822 it would be nice to have at least a
_similar_ concept of header and folding in news and mail.

Or do we want mail2news gateways to insert a mandatory trailing
space in the first line of any "name: CRLF SP body CRLF" ?

                        Bye, Frank

Date: Sun, 12 Dec 2004 01:01:50 +0000
To: Seth Breidbart <[email-masked]>
Cc: [email-masked], [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: msg-id
Sender: [email-masked]

In message <[email-masked]>, Seth Breidbart
<[email-masked]> writes
>
>John Stanley <[email-masked]> wrote:

>> Yes, by using an algorithm that does not have the possibility of 
>> duplicates. "epochtime.sequence.pid.yy.mm.dd.hh.mm.ss@domain" will not 
>> repeat.
>
>You are making the assumption that a new domain owner will continue to
>use the same algorithm.  That is not a valid assumption.  A new domain
>owner might use a _different_ algorithm.

there also appears to be an assumption that "domain" is the same as
"host" :-(  For example, if the co-owner of my domain posts at the same
moment as I do (from his machine 100 miles to the south of here) there
may be a clash. However, there's an obvious set of possible fixes to
John's design to tackle that, so it's not a killer observation.

Now in practice we all use software that avoids this problem by other
methods...  (in the case of co-owner and I, there's a unique product ID
included in the algorithm.  I do wonder if @aol.com people can be so
sanguine about this ?)

However since the standard isn't going to specify the exact mechanism as
to how duplicates are avoided, why not just have it concentrate on why
duplicates are a bad thing (stressing that this now means over
substantial time periods) and impress upon implementors that they should
address this issue properly and not make a mess of it by skimping on
message ID length (which I think worries some people)

viz: say what the result should be, stress that it's not a trivial thing
to achieve, and leave exploring the rest of the rathole alone

>I'm saying that I'm not worried about sufficiently-low failure
>probabilities.  

perhaps the text should specifically address the effect of the failures
(viz: that your new message may fail to propagate, or fail to be
archived)... 

>I own a domain that does not run a news server.  That domain will
>never see a usenet message.  Yet use of it in the RHS of a Message-Id
>by me can certainly guarantee uniqueness (modulo other people doing
>stuff they're not entitled to, which can break any algorithm).

... so what the text should be saying is that using a RHS which you
don't control is especially undesirable because it means that you may
adversely affect complete strangers.

If we didn't believe that the whole aim is to ensure we only screw up
ourselves, not strangers, then for efficiency of implentations we'd
avoid x@y style IDs altogether and start recommending IDs of the form
opaque@usenet where the opaque value was created by running SHA-1 over
an identifier to which consisted of - let's say - a sequence number
appended to our birth date and GPS co-ordinates of our current location.

However, this opaqueness feels like something that might go wrong from
time to time (in reality, many eons, but the feel would be otherwise)
and we'd never even meet the person who caused it to go wrong for us ...
whereas if we're using our domains, even with a really poor mechanism,
then at least we can go upstairs to the kid's bedroom (or head down
south for the afternoon) and berate the obstructer personally :)

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

Date: Fri, 10 Dec 2004 16:23:06 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

John Stanley <[email-masked]> wrote:
> Seth Breidbart <sethb@xxxxxxxxx>:
>>> What is important is that the ERROR is known and can be avoided completely
>>> by fixing it now.
>
>>No, it can't be avoided completely.
>
> Yes, by using an algorithm that does not have the possibility of 
> duplicates. "epochtime.sequence.pid.yy.mm.dd.hh.mm.ss@domain" will not 
> repeat.

You are making the assumption that a new domain owner will continue to
use the same algorithm.  That is not a valid assumption.  A new domain
owner might use a _different_ algorithm.

>>> Hmmm. Let's see. I use "sequence number dot epoch time dot process id dot
>>> month dot day dot year @ domain". Since THIS epoch time will never
>>> reoccur, it cannot be the same as what any other owner of my domain would
>>> use. Guaranteed unique.
>>
>> Except that another owner of the domain 
>
> There is no other owner at that epochtime. 

So what?  I didn't say there was.

>>(or another piece of client
>>software in use in that domain) used (or will use) "sequence number
>>dot process id dot epoch time dot day dot month dot year @ domain",
>
> No, I've already told you the algorithm being used in that domain.

That's the algorithm being used by the current owner now.  A new owner
later might use a different algorithm.

>>...and gets his pids randomly from 32-bit space (to avoid potential
>>attacks based on known-pids).  Oops.
>
> I'm interested in hearing which operating system allows two different 
> processes to have the same process id at the same time.

You might read my example more carefully.  Because the new owner used
a _different_ algorithm, different fields matched (e.g. his PID
matched your epoch-time).  Sure, the odds are low; but you don't find
that acceptable.

>>Genetic testing shows that the results were not the same.
>
> "Your wife had a baby" does not depend on genetic testing. The content of 
> the message does not have any effect on the message id.

If you choose to leave out critical data, then any events are "the
same".  The issue here is "are the Message Ids unique?"

>>No, he didn't.  If a coin is expected to come up heads once per
>>thousand years, then either it's extremely biased, or it's expected to
>>be tossed twice in a thousand years.
>
> Well, it might be extremely biased, but it certainly doesn't need to be 
> tossed twice for the odds to be stated the way they were. If we say the 
> odds of you graduating college are unity, does that mean you have to try 
> graduating twice?

No, if the odds are unity, then I have to try one time that has a 100%
chance of success.  If the expected number of times is unity, and each
attempt has a probability of 50%, then the expected number of attempts
is 2.

>>You keep claiming that other people don't understand what "expected"
>>means, with no evidence other than your own statements.
>
> And theirs.

And your misunderstanding of their statements or of probability
theory.

>>Another red herring.  Who said we shouldn't or can't specify "MUST"?
>
> The question was about a MUST be unique clause, which doesn't actually
> need to be unique.

It does need to be unique.  The protocol fails when it isn't unique.

Note that "need" is always relative to a goal.

> So, I guess the correct statement should have been:
>
> "We cannot specify a MUST that we really mean because  ...".

That's your statement, not mine.  I say we can specify the MUST.

>>We can't _implement_ it perfectly, because we can't implement
>>_anything_ perfectly. 
>
> A nice variant on what I just said. We shouldn't worry about implementing 
> what we CAN correctly because there are other things that can make it 
> break anyway.

Is that what you're saying?  It certainly isn't what I'm saying.

I'm saying that I'm not worried about sufficiently-low failure
probabilities.  You say that you won't accept them, except that your
examples of what is acceptable to you still have non-zero failure
probabilities.

> If mozilla creates message ids for a domain that may never see the
> message at all, then it cannot guarantee it will be unique.

I own a domain that does not run a news server.  That domain will
never see a usenet message.  Yet use of it in the RHS of a Message-Id
by me can certainly guarantee uniqueness (modulo other people doing
stuff they're not entitled to, which can break any algorithm).

Seth

Date: Fri, 10 Dec 2004 12:17:34 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Address Validation (Was: Re: draft-ietf-usefor-usepro-02)
Sender: [email-masked]

Thorfinn <thorfinn@xxxxxxxxxxxxxx>:

>I *think* that John's point is that users with otherwise valid email
>addresses may will find their articles rejected also by some sites
>(those that choose to allow only addresses *they* know to be valid).

And by rejecting them, make then effectively invalid.

>I also think that nobody else cares that this is the case,

I am certain that nobody cares that this is the case. 

>There is plenty of choice available of injecting points for users,

I guess the fact that the draft implies abilities that are completely 
fictional doesn't matter.

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>So how about, for injecting agents:

>   2. It SHOULD verify that the article is from a trusted source, and
>      MAY reject articles in which headers contain unverified email
>      addresses, that is, addresses which are not known to be valid for
>      the trusted source, though it would be perverse to reject
>      intentionally unverifiable addresses such as those ending in
>      ".invalid" (7.5).

This solves the problem of de-facto invalidation of perfectly valid 
addresses exactly how? 

And "perverse"? Foof.

>Where the relevant wording in 7.5 is:

>   Contrary to [RFC 2822], which states that the mailbox(es) in the From
>   header is that of the poster(s), a poster who does not, for whatever
>   reason, wish to use his own mailbox MAY use any mailbox ending in the
>   top level domain ".invalid" [RFC 2606].

And this solves the problem of telling users that they MAY do one thing
and server admins that they don't exactly how?

Date: Fri, 10 Dec 2004 12:10:53 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

Seth Breidbart <sethb@xxxxxxxxx>:
>> What is important is that the ERROR is known and can be avoided completely
>> by fixing it now.

>No, it can't be avoided completely.

Yes, by using an algorithm that does not have the possibility of 
duplicates. "epochtime.sequence.pid.yy.mm.dd.hh.mm.ss@domain" will not 
repeat.

>> Hmmm. Let's see. I use "sequence number dot epoch time dot process id dot
>> month dot day dot year @ domain". Since THIS epoch time will never
>> reoccur, it cannot be the same as what any other owner of my domain would
>> use. Guaranteed unique.

> Except that another owner of the domain 

There is no other owner at that epochtime. 

>(or another piece of client
>software in use in that domain) used (or will use) "sequence number
>dot process id dot epoch time dot day dot month dot year @ domain",

No, I've already told you the algorithm being used in that domain. And 
since only one process has the process id, two different processes on the 
same system cannot generate the same message id.

>...and gets his pids randomly from 32-bit space (to avoid potential
>attacks based on known-pids).  Oops.

I'm interested in hearing which operating system allows two different 
processes to have the same process id at the same time.

>Genetic testing shows that the results were not the same.

"Your wife had a baby" does not depend on genetic testing. The content of 
the message does not have any effect on the message id.

>> No, you said they caused the SAME violation of the same MUST. Even
>> were that true, the two causes are very different.

>So what?

So you cannot even correctly represent what you said.

>> No, actually it doesn't. It someone tosses one coin per thousand years and 
>> you say it has a once per thousand year chance of coming up heads, you've
>> just said it is a certainty to come up heads.

>No, he didn't.  If a coin is expected to come up heads once per
>thousand years, then either it's extremely biased, or it's expected to
>be tossed twice in a thousand years.

Well, it might be extremely biased, but it certainly doesn't need to be 
tossed twice for the odds to be stated the way they were. If we say the 
odds of you graduating college are unity, does that mean you have to try 
graduating twice?

>You keep claiming that other people don't understand what "expected"
>means, with no evidence other than your own statements.

And theirs.

>> Ah, yes, this old "we cannot specify MUST because a computer somewhere 
>> might crash and violate it anyway."

>Another red herring.  Who said we shouldn't or can't specify "MUST"?

The question was about a MUST be unique clause, which doesn't actually
need to be unique. So, I guess the correct statement should have been:

"We cannot specify a MUST that we really mean because  ...".

>We can't _implement_ it perfectly, because we can't implement
>_anything_ perfectly. 

A nice variant on what I just said. We shouldn't worry about implementing 
what we CAN correctly because there are other things that can make it 
break anyway.

Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx>:

>That might be true, or let's say I don't care.  But we have a
>practical problem with some very old UAs like my "Mozilla 3":

>This monster creates Message-IDs based on the RHS of the From:
>address.  Now we tolerate something like spamhater@invalid as
>an "address".  

This monster was in violation of the standards long before we started
"tolerating" .invalid addresses. If mozilla creates message ids for a
domain that may never see the message at all, then it cannot guarantee it
will be unique.

Date: Fri, 10 Dec 2004 11:52:58 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>No, it implies no such thing. Agents may or may not be able to detect
>validity/invalidity of various classes of address, as Seth has pointed
>out. The "MAY" permits them to make use of such information as they may
>have available.

Since they do not have the information that says "your address is invalid" 
in the general case, there is no use to be made of that "information". 
Pretending they CAN know this is the problem. 

>Consequently, people who use invalid addresses may well
>find their articles rejected. Or not.

Consequently, people who use perfectly valid addresses may well find their
articles rejected, because we've told server admins they can pretend to
know if they are valid or not.

If you don't know it is invalid, you ought to do nothing about it. Knowing 
it is invalid takes more than saying "you must use an example.com email
address".

>>Excuse me? Nothing else can exist that can do the same thing? Sorry, but
>>you are patently wrong.

>No, there is nothing else that will nearly always work on Usenet as it
>presently exists.

Well, DOH! But that is not an excuse for preventing it from being created.
There is no reason to demand in the standard that "by email" be the only
method of sending articles to the moderator. If that is the only current 
implementation, that's one thing, but to close out any other 
implementation is just ridiculous.

>Other mechanisms may be used in small cooperating subnets, but until they
>become commonplace we have no way of standardizing them.

I'm sorry, but in another section of the draft you try to "standardize" 
the "exceptional" cases of reinjection. Which is it? Something must be 
commonplace before it can be standardized, or standards can be written to 
allow exceptional cases?

>>>So if the Injection-Date is earlier than the expiry date
>>>(however computed) that it implements (probably of a per group basis), 
>>>then it MUST drop it.

>>I asked once, and I expected an answer. Why is this a MUST? Expiration 
>>policy is the site's responsibility, not ours. 

>This is nothing to do with controlling the expiration dates of sites. It
>is a matter of preventing loops, for which purpose that MUST is essential.

I see nothing in that statement about "loops", and everything about 
"expiry date". 

>... both Frank and Seth seem to he saying that it is about
>right as I currently have it. 

So? They are wrong, if that is what they are saying. It is
self-contradictory if we tell people in one place that they may do
something and then not tell the admins in another place that they need to
allow it. It would be like allowing the followup agent to create message
ids of the form "<something.somethingelse.but.no.at.sign>" and then
telling injecting agents that they don't have to accept them.

>I.e. we encourage people to use ".invalid",

No, we tell them explicitely they may use it. There is a difference.

>I agree that the
>way it is currently worded seems to be pulling two ways at once and will
>probably need to be changed. But I need guidance as to which direction to
>change it in.

Well, that's what I've been giving you. 

>>There is no reason to redefine a standard english word when there is 
>>already a standard english word that conveys the correct meaning.

>"Take" is also a standard english word, and conveys the intended meaning
>in this case. 

No, it does not. "Take" means that something is removed from one place and
moved to another. "Take" is how SBC convinced the courts to levy such
harsh consequences against those people who "took" copies of telco
documents. It is only after you redefine "take" to mean "copy" that "take"
comes even close to the correct meaning. And yet, "copy" already has that
meaning. There is no reason to redefine "take" so it means "copy"; just
use "copy".

>I do not see your problem with it. Does anyone else?

And yet, the draft used to say "copy", before you decided unilaterally to 
change it.

>>Nonsense. We tell it what it must use. We do NOT tell the user what he 
>>uses. Since there is only ONE thing that the followup agent is allowed to 
>>do ("copy the content") that is the default. There is no other default.

>There are TWO things the followup agent can do. One is to accept (without
>demur) what the user types in. 

The user has typed NOTHING in at the point in question. We are telling it 
what it must do when it prepares the article for user action. There is 
only ONE THING that it does -- and that means that ONE action is the 
default. Saying that it must, by default, do the only thing we tell it it 
can do is just silly. 

>>You want to trim the one in the followup. "If the resulting 
>>References-header is excessively long, it may be trimmed by removing 
>>message identifiers other than the first and the last."

>But OK, I will buy that, except that it has to be the "last two" (i.e. you
>MUST keep the last one from the precursor).

Why? What breaks if the parent of the precursor is no longer threaded? 
Justify the MUST.

>The only limitations are those imposed by the lack of appropriate
>facilities in the reading agent. The whole point of the wording is to
>indicate that the poster cannot assume that every reading agent will be
>able to make sense of everything, however bizarre, that he might choose to
>put in the article.

Then say that. But that's so obvious that I don't think it needs to be 
said. That whole section about "reading agent" is a definition seeking a 
purpose. And the title implies that a reading agent HAS some duties, which 
it does not. There is nothing we can say about what a reading agent MUST 
or SHOULD do, since we already acknowledge that "reading agent" doesn't 
even necessarily display anything to anyone. 

The only place we need to mention "reading agent" is in the definitions, 
and then we can say:

	A "reading agent" downloads articles from a serving agent for 
	reading or other processing outside the news system. 

>Neither sentence is meaningless. Does either of them say anything that is
>untrue?					

Do you understand the difference between "meaningless" and "untrue"? If 
the only justification for putting things in the draft is that they be 
true, then we ought to be saying things like "the euro is the current 
monetary standard of the EU" and "upwelling on the Oregon Coast is 
responsible for the vast nature of abundant sea life found there.".

>>We already cover the possibility of loops, so there is no need for this 
>>paragraph at all.

>There are various ways in which loops can arise. It is necessary to draw
>attention to them.

We already draw attention to them, in the previous section where it is 
stated as "the primary dictat": Above all, prevent loops.

The only content to the entire first paragraph of that section is to say:
loops are possible, please prevent them. We've already said that. There is 
no need for the entire paragraph. It adds no meaning to the draft, so 
while it may be completely true, it is also completely meaningless. But it 
is wrong in that it says that the limits are due to the PRESENCE of 
another incoming gateway, and that is not true. The operation MUST 
consider the POSSIBILITY of an incoming gateway, not just the PRESENCE of 
one.

So, to solve the problem, and make things shorter (something you like to 
do), remove it. One paragraph shorter.

>>It is unlikely that an outgoing gateway is going to see an incoming 
>>message.

>On the contrary, that is just the kind of loop we are trying to catch.

That kind of loop is easy to catch: simply do not allow an outgoing 
gateway to accept incoming messages. Problem solved. In fact, I doubt 
there are many outgoing gateways that do accept incoming messages. I know 
the ones I have written do not. There is, in fact, no way to SEND an 
incoming message to my outgoing gateway. 

In fact, if a gateway accepts INCOMING messages, then it is an INCOMING 
gateway, and is dealt with elsewhere.

>If
>the article is gatewayed to the "other medium" and there exists some other
>gateway elsewhere that puts it back into Netnews, then the flooding
>algorithm is very likely indeed to bring it back to the first gateway.

You mean like how I mentioned in the very next sentence? 

>>However, if you meant that the same outgoing gateway is going to 
>>see the same message outgoing more than once, then it already has the 
>>mandatory message id (not in the body) to work with.

>There might not be such a thing as a message id in the other medium. 

There is a MANDATORY message id in news. An outgoing gateway takes NEWS 
articles and sends them to the other medium. If you think an outgoing 
gateway author is going to throw the message id away prior to checking for 
dupes just because the "other medium" doesn't understand message ids, then 
you are being foolish.

>> A comment in the body 
>>of a messsage is not likely to prevent loops. Saying it does is silly.

>No. If the gateway to the "other medium" (which presumably understands the
>rules for contructing messages in the other medium) can manage to insert
>some distinct identifier into whatever passes for a body in the other
>medium, then it is quite capable (and indeed it is the only agent in the
>whole world with that capability) of recognizing the identifier that it
>itself placed there, should it ever come back again. 

So you expect outgoing gateways to parse the BODIES of the messages it 
converts looking for things that look like something it understands? Well 
now, that's a serious requirement. But since the content of the body is 
UNSTRUCTURED, you've got a long row to hoe defining structure of headers 
in the body. But before you can claim that a comment in the body of a 
message (in some other medium than news, to boot) is likely to prevent 
loops, you need that structure.

>It is nowhere stated that a separate injecting agent is involved in an
>incoming gateway. 

Nowhere is it stated that it is not.

>If there is, then it can be relied on to do the checks.

Not when we tell the gateway author that the gateway author that the 
articles the gateway forms must follow all the standards. 

>>It takes a bit of thought there to realize that a duplicated message id is 
>>an illegal content for the message id header. And most injecting agents 
>>do, indeed, reject duplicates.

>How?

By trying to inject it and getting a rejection, because the duplicate ID 
is already in the history. 

>And if it is the same message come round again, then it is not even
>illegal to have the same message id.

So this discussion about MUST generate unique ids is frivolous?

>>>I would suggest checking for and noting the presence of the cancel message
>>>_before_ removing it.

>>There is no cancel message before removing it. I am told I must remove it 
>>before I can inject it.

>Eh?

I have an incoming gateway for group X. User Y sends a message to the 
gateway containing the header "Control: cancel <[email-masked]>". My 
gateway is told I MUST NOT pass that message without removing or renaming 
that header. On the other hand, since I can determine that this is a valid
cancel message, I can create a cancel message of my own. How? By passing
the message WITH the control header. 

I MUST NOT pass the message, but I SHOULD pass it.

Now, YOU told me that I was supposed to look for a cancel message before I
remove the header. I ask my server for that message and, not surprisingly,
since I have not yet posted one, it does not yet exist. So how does this 
help the problem? If the cancel message already existed, yes, I'm out of 
the woods, I don't have to process the incoming message. But since it does 
not, and I MUST NOT/SHOULD process it, I'm stuck.

>If an article with a Supersedes header arrives at an outgoing gateway,

So what? We're talking about duties of an INCOMING gateway here.

>The word "poster" is in widespread usage throughout usenet and does not
>seem to cause any real problems out there, 

The word "log in" is in widespread usage on the web, even for sites that 
have no login/password function. It has not seem to cause any real 
problems in that usage, but it is still completely incorrect.

To: LIST: [email-masked];
Xref: clerew local.usefor:20435
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Address Validation (Was: Re: draft-ietf-usefor-usepro-02)
Date: Fri, 10 Dec 2004 14:03:09 GMT
Lines: 43
Sender: [email-masked]

In <[email-masked]> Thorfinn <[email-masked]> writes:

>I *think* that John's point is that users with otherwise valid email
>addresses may will find their articles rejected also by some sites
>(those that choose to allow only addresses *they* know to be valid).

>I also think that nobody else cares that this is the case, and in fact,
>that everyone expects this to be the case on some sites, and that is
>fine by them.

If that is the general view here, then my present text can more or less
stand. But John has a slight point that, if we are encouraging people to
use ".invalid" then we should not write things in such a way as to suggest
that their good efforts might nevertheless lead to rejection. It's a
matter of giving the right "feel", without trying to tie things down with
too many MUSTs.

So how about, for injecting agents:

   2. It SHOULD verify that the article is from a trusted source, and
      MAY reject articles in which headers contain unverified email
      addresses, that is, addresses which are not known to be valid for
      the trusted source, though it would be perverse to reject
      intentionally unverifiable addresses such as those ending in
      ".invalid" (7.5).

Where the relevant wording in 7.5 is:

   Contrary to [RFC 2822], which states that the mailbox(es) in the From
   header is that of the poster(s), a poster who does not, for whatever
   reason, wish to use his own mailbox MAY use any mailbox ending in the
   top level domain ".invalid" [RFC 2606].

To: LIST: [email-masked];
Xref: clerew local.usefor:20436
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Fri, 10 Dec 2004 14:12:16 GMT
Lines: 40
Sender: [email-masked]

In <[email-masked]> Henry Spencer <[email-masked]> writes:

>On Fri, 10 Dec 2004, Frank Ellermann wrote:
>> > I believe some relaying agents do still check the id-right
>> > case insensitively. But many (most?) don't, and no way should
>> > our draft require them to.
>> 
>> Please mention this in the text, it's an important difference
>> from s-o-1036 or from 1036 plus 822.

>Yes, that's a good point -- it deserves mention in a NOTE, as a warning
>to new implementers, even if the specs are believed to be clear about it.

There was indeed such a NOTE in 5.3 of draft-13:

        NOTE: Some old software may treat message identifiers that
        differ only in case within their id-right part as equivalent,
        and implementors of agents that generate message identifiers
        should be aware of this.

But it got lost in the great rush to chuck out all unnecessary texts as
part of the split :-( .

I have now pot it back at the end of Step 5 of the duties of a relaying
agent.

Note that both USEFOR and USEPRO now make the point that a simple
case-sensitive comparison of octets is all that is ever required to
determine the identity of two message-ids.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: msg-id
Date: Fri, 10 Dec 2004 06:44:03 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 45
Sender: [email-masked]

Charles Lindsey wrote:

> one failure per thousand years
[...]
> could be regarded as satisfying that MUST for all practical
> purposes

That might be true, or let's say I don't care.  But we have a
practical problem with some very old UAs like my "Mozilla 3":

This monster creates Message-IDs based on the RHS of the From:
address.  Now we tolerate something like spamhater@invalid as
an "address".  With these UAs that would result in Message-IDs
like <41AE974D.1DCF@invalid>.  We can't have that.

Of course the UA is broken, but it's another argument for RHS =
domain as specified in 2821.

>> Message-ID <xyz@news-fe-01>, and the consensus was that it's
>> wrong.  The ..!news-fe-01!.. in his Path: wasn't much better.

> Well I suppose "xyz" COULD be the first in a long sequence of
> strings all provably different, but it is not a convincing
> example, as you say.

The discussion wasn't about the xyz, and the real LHS was one
of the usual constructs, it was about the RHS.  This RHS was
"identified" as invalid, because it's no domain.

> But the id-right is NOT case-sensitive, whether it looks like
> it came from a domain-name or not (I know Bruce will disagree
> with this, but he is wrong).

No, I disagree, and you are wrong, it _is_ now case-sensitive,
you said so in another article. ;-)  That's fine, I like the
opaque "magic token" concept for Message-IDs.  If the UA uses a
decent domain as RHS, not invalid stuff, or if it accepts the
proposed Message-ID from its injecting agent.

So the last thing I'm unhappy with is the very complex special
syntax for the LHS, especially the idea to allow any NO-WS-CTL
where it never before was allowed in news.  It's just bad for
the news URL.
               Bye, Frank

Date: Fri, 10 Dec 2004 00:28:17 +0000
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: MD5 collision (was msg-id)
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes

>Now that is rather interesting, because I have never seen a report of a
>genuine MD5 collisionw.

One was reported this August in the rump session of Crypto 2004

        http://www.tcs.hut.fi/~mjos/md5/

>AIUI, there are suspicions that the MD5 hashing scheme is not as secure as
>it was once thought to be.

AIUI, there's still no known way of creating a clash with a previously
provided hash value, but it seems that the Chinese researchers can
provide collisions (ie they provide both of the pair) pretty much at
will. AFAIK, their methods are still not published, but study of what
they hash shows the sort of properties (minor changes to a few high
order bits) that they are exploiting.

SHA-0 also has collisions known, but despite rumours nothing firm is
known against SHA-1.  No-one from the trade has been recommending MD5
since the results found in the mid 1990s (and its output length is a
little bit too short anyway), but if you are using it there's still no
real reason to panic -- you should just be making plans to move away, at
present I would suggest to SHA-1, when it is convenient to do so.

- -- 
richard @ highwayman . com                       "Nothing seems the same
                          Still you never see the change from day to day
                                And no-one notices the customs slip away"

Date: Thu, 9 Dec 2004 19:06:18 -0500 (EST)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: msg-id
Sender: [email-masked]

On Fri, 10 Dec 2004, Frank Ellermann wrote:
> > I believe some relaying agents do still check the id-right
> > case insensitively. But many (most?) don't, and no way should
> > our draft require them to.
> 
> Please mention this in the text, it's an important difference
> from s-o-1036 or from 1036 plus 822.

Yes, that's a good point -- it deserves mention in a NOTE, as a warning
to new implementers, even if the specs are believed to be clear about it.

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Fri, 10 Dec 2004 00:49:24 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 9
Sender: [email-masked]

Charles Lindsey wrote:

> The word "MAY", as defined in RFC 2119, is nearly always used
> to deal with matters of policy. Protocols can and do contain
> optional features.

RfC 2476 taught me to read "maybe not" whenever I see a MAY in
an RfC, the effects are sometimes devastating. :-(  Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: msg-id
Date: Fri, 10 Dec 2004 00:36:20 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 17
Sender: [email-masked]

Charles Lindsey wrote:

> I believe some relaying agents do still check the id-right
> case insensitively. But many (most?) don't, and no way should
> our draft require them to.

Please mention this in the text, it's an important difference
from s-o-1036 or from 1036 plus 822.  I recall discussions with
experts, where some argued that 2822 updated 822, and therefore
1036 implicitly inherited this update, and others argued that
2822 does not yet replace STD 11, and I argued that s-o-1036
is THE LAW and anything else nonsense, unless it's in the book
of Mozilla.

Okay, not all of these discussions were _that_ irrational, but
still funny and technically inconclusive.  Bye, Frank

Date: Fri, 10 Dec 2004 09:23:24 +1100
From: Thorfinn <[email-masked]>
To: Charles Lindsey <[email-masked]>
Cc: LIST: [email-masked];
Subject: Address Validation (Was: Re: draft-ietf-usefor-usepro-02)
Sender: [email-masked]

On Thu 09 Dec 2004 at 03:07:02PM +0000, in <[email-masked]>,
Charles Lindsey <[email-masked]> wrote:
> In <[email-masked]> John Stanley <[email-masked]> writes:
[snippage]
> >No, it says "it MAY allow articles in which headers contain unverified
> >email addresses, that is, addresses which are not known to be valid for
> >the trusted source,...". The implication is that the agent can tell which
> >are valid (and thus which are not.)
> No, it implies no such thing. Agents may or may not be able to detect
> validity/invalidity of various classes of address, as Seth has pointed
> out. The "MAY" permits them to make use of such information as they may
> have available. Consequently, people who use invalid addresses may well
> find their articles rejected. Or not.

I *think* that John's point is that users with otherwise valid email
addresses may will find their articles rejected also by some sites
(those that choose to allow only addresses *they* know to be valid).

I also think that nobody else cares that this is the case, and in fact,
that everyone expects this to be the case on some sites, and that is
fine by them.

There is plenty of choice available of injecting points for users, even
if they only have one ISP they can connect to, all with different
policies.

Later,

  Thorf

-- 
<a href="http://tertius.net.au/~thorfinn/">[email-masked]</a>
"Sorry I'm late, I was held up by that fuckwit Harradine."
    -- unnamed Liberal MP, to morgan(vurt.net

Date: Fri, 10 Dec 2004 09:11:21 +1100
From: Thorfinn <[email-masked]>
To: [email-masked]
Subject: [Mostly OffTopic] Re: MD5 collision (was msg-id)
Sender: [email-masked]

On Thu 09 Dec 2004 at 12:17:41PM +0000, in <[email-masked]>,
Charles Lindsey <[email-masked]> wrote:
> In <[email-masked]> Thorfinn <[email-masked]> writes:
> >A "MUST" isn't some magical stick to be implemented in airy fairy theory
> >land.  It butts up against the reality that is practice.  And in
> >practice, hardly anyone is going to give a shit about once in every
> >thousand years events (hell - I should know - I got an MD5 collision on
> >two randomly generated numbers once on an automated MD5 module install
> >test, shrugged and tried again), when there are far more critical issues
> >affecting the problem.
> Now that is rather interesting, because I have never seen a report of a
> genuine MD5 collisionw.

Hrm.  Should I be reporting it somewhere in particular?  I definitely
saw it, but obviously, it was unrepeatable when I tried again.

The module in question was the Perl Digest::MD5 module, and it was in
its automated test that grabs a random chunk out of /dev/random twice
and compares them that I got a collision...  Now, it's *possible* that
/dev/random isn't so random... but also, as you say:

> AIUI, there are suspicions that the MD5 hashing scheme is not as secure as
> it was once thought to be. And apparently the nature of the problem is
> that, given a truly random set of strings to be hashed, they do not map
> uniformly on to the 128-bit space of hash values. I.e. there exist hash
> values that are more likely to be mapped into than others.

*nodnod* Quite possible.

> Which probably explains how you came to observe an event that, on pure
> probability grounds, was otherwise expected to occur about once before the
> heat-death of the universe.

Yeps.  There are also plenty other factors that could explain it such as
bugs in a number of possible different places.  And that's the point
with Message-ID generation... we're a whole lot more likely to get
duplicate Message-ID generation due to ordinary software bugs than
anything else.

Meep,

 Thorf

-- 
<a href="http://tertius.net.au/~thorfinn/">[email-masked]</a>
"Sorry I'm late, I was held up by that fuckwit Harradine."
    -- unnamed Liberal MP, to morgan(vurt.net

Date: Thu, 9 Dec 2004 14:36:24 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

John Stanley <[email-masked]> wrote:
> Seth Breidbart <sethb@xxxxxxxxx>:

>>> It does not mean it cannot happen tomorrow or the day after. It does
>>> not mean it will only happen in one thousand years.
>
>>I understand that.  It means it will happen, on average, one thousand
>>time in one million years.
>
> No, it does not. It means that has an X percent chance of happening FOR
> EVERY ID GENERATED, which, assuming a rate of Y ids generated per thousand
> years, multiplies out to 1.

Where X varies, because the chance of collision is different each time
a new ID is generated.

> What is important is that the ERROR is known and can be avoided completely
> by fixing it now.

No, it can't be avoided completely.

>>So is any other, since whatever method
>>you use might generate a Message-ID that was previously used by a
>>previous owner of the domain (or might sometime in the future be used
>>by a subsequent owner).
>
> Hmmm. Let's see. I use "sequence number dot epoch time dot process id dot
> month dot day dot year @ domain". Since THIS epoch time will never
> reoccur, it cannot be the same as what any other owner of my domain would
> use. Guaranteed unique.

Except that another owner of the domain (or another piece of client
software in use in that domain) used (or will use) "sequence number
dot process id dot epoch time dot day dot month dot year @ domain",
and gets his pids randomly from 32-bit space (to avoid potential
attacks based on known-pids).  Oops.

>>>>They both cause the same violation of the same MUST.
>>> No, they don't,
>>They both cause a non-unique Message-ID.  
>
> Your wife has a baby. It could either be caused by YOUR romantic actions, 
> or the romantic actions of the milkman. There is no difference because the 
> result was the same.

Genetic testing shows that the results were not the same.

Duplicate Message-IDs are duplicate Message-IDs.

> The breakdown in the piece of hardware causes a mangled or
> non-existant message id. How can a broken-down computer generate
> anything?

Remember the Pentium multiply bug?  A broken chip can still work
correctly most of the time.

> A flawed implementation generates a valid-looking but non-unique
> message id.

A good implementation on flawed hardware generates a valid-looking but
non-unique message id.

> There's a significant difference between the two.

"non-unique"

> No, you said they caused the SAME violation of the same MUST. Even
> were that true, the two causes are very different.

So what?

> "We shouldn't require adherance to a MUST because something else
> might break and violate the MUST" is just silly.

We shouldn't worry about one potential cause of violating a MUST
because its actual real-world probability is low enough not to matter.

>>You don't think ISPs will know what software they're running?  Maybe
>>you should avoid such ISPs.  I do.
>
> I don't think many ISPs know what software their outsourced news servers 
> run.

Then either don't use one that outsources news, or ask them who they
outsource it to and ask the outsourcer what it runs.

Or generate your own.

>>I am not at all bothered by the expected once in the next thousand
>>years Message-ID collision.
>
> I know. Your error.

Do you avoid all risks greater than those duplicate Message-IDs during
your lifetime?

> <thorfinn@xxxxxxxxxxxxxx>:

>>> Then you would know that "once per thousand years" is 1) a pretty 
>>> meaningless statistic
>
>>Err, no, it's not meaningless.  It means that it's an extremely low
>>probability event.  
>
> No, actually it doesn't. It someone tosses one coin per thousand years and 
> you say it has a once per thousand year chance of coming up heads, you've
> just said it is a certainty to come up heads.

No, he didn't.  If a coin is expected to come up heads once per
thousand years, then either it's extremely biased, or it's expected to
be tossed twice in a thousand years.  (Or it's a little biased and
expected to be tossed a little over twice, or has only heads and
expected to be tossed once, etc.)

You keep claiming that other people don't understand what "expected"
means, with no evidence other than your own statements.

>>"The computer crashes" isn't an implementation of MUST NOT, either.
>
> Ah, yes, this old "we cannot specify MUST because a computer somewhere 
> might crash and violate it anyway."

Another red herring.  Who said we shouldn't or can't specify "MUST"?

We can't _implement_ it perfectly, because we can't implement
_anything_ perfectly.  That doesn't mean it shouldn't be in the spec.

Seth

To: [email-masked]
Subject: Re: msg-id
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 09 Dec 2004 11:05:35 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:
>> Charles Lindsey <[email-masked]> writes:

>>> Indeed, injecting agents may well have to check for these funny cases
>>> (but then that is what I said). But relaying and serving agents have
>>> no time to perform such checks, and so will proceed on the assumption
>>> that any broken cases will have been weeded out beforehand.

>> Charles is indeed mistaken about what many relaying and serving agents
>> do.  John is correct, and not just for injecting agents.

> Please can you be more explicit? Are you claiming that all relaying and
> serving agents make full and exhaustive checks on the format of all
> message-ids? To the same extent than injecting agents do, or should do?

I'm saying that your generalization is false, and many relaying and
serving agents do indeed perform these checks to the same level of
scrutiny as injecting agents.

> Henry Spencer seems to agree that what I said in my last paragraph
> quoted above was more or less correct.

Henry said that many relaying agents perform these checks, but not all of
them do, and therefore we need to proceed on the assumption that they
might not.  That seems reasonable to me, and is not quite what you said.
You were just disagreeing with John in more absolute terms than are
actually warranted.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: LIST: [email-masked];
Xref: clerew local.usefor:20420
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Thu, 9 Dec 2004 12:03:55 GMT
Lines: 39
Sender: [email-masked]

In <[email-masked]> Henry Spencer <[email-masked]> writes:

>On Tue, 7 Dec 2004, Charles Lindsey wrote:
>> >> But the id-right is NOT case-sensitive, 
>> >I see nothing in RFC2822 that says it is not case-sensitive.
>> 
>> Then I think we are in agreement. But I am aware of people who do not
>> share that view.

>Hmm, interesting.  This changed from 822 to 2822.  RFC 2821, SMTP, is
>explicit that the domain of an addr-spec is case-insensitive, but 2822 is
>silent about it... and the body of a message ID is no longer an addr-spec
>anyway. 

>> Indeed, injecting agents may well have to check for these funny cases (but
>> then that is what I said). But relaying and serving agents have no time to
>> perform such checks, and so will proceed on the assumption that any broken
>> cases will have been weeded out beforehand.

>I think it would be more accurate to say "might not have time" and "may
>proceed on the assumption".  Some will undoubtedly wish to be fussier, but
>this should be optional rather than mandatory.

Indeed. That is what I meant. 

For example, I believe some relaying agents do still check the id-right
case insensitively. But many (most?) don't, and no way should our draft
require them to.

To: LIST: [email-masked];
Xref: clerew local.usefor:20426
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Thu, 9 Dec 2004 15:30:16 GMT
Lines: 53
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>Seth Breidbart <sethb@xxxxxxxxx>:

>>>> To whom are you attributing that error?
>>> To everyone who says that "once per thousand years" is good enough.

>>Then you're wrong.   I think one collision, total, on average every
>>thousand years is good enough. 

>The "error" is not whether you think that is "good enough", but on the
>meaning of "once per thousand years". It does not mean it cannot happen 
>tomorrow or the day after. It does not mean it will only happen in
>one thousand years.

>> I also understand probability theory quite well.

>Then you would know that "once per thousand years" is 1) a pretty 
>meaningless statistic and 2) not in any way an implementation of "MUST 
>NOT".

>>> A breakdown in a piece of hardware is different than a programmed, known
>>> flaw.

>>They both cause the same violation of the same MUST.

>No, they don't, and even were that true, they are still very different 
>things. The existance of one does not mean the other is unimportant.

Technically speaking, using a sufficiently large random number as a means
of constructing message-ids is in violation of that MUST, and that MUST is
still there (in RFC 2822 actually).

But in practice is is perfectly good enough that we do not really need to
care about it. And in fact our draft is currently totally silent on this
issue, relying on the wording in RCC 2822 (except that USEFOR now adds
that uniqueness now applies over both Email and Netnews).

I see no point in writing anything different in our drafts, but if I were
writing an informational document explaining ways of generating
message-ids, I would probably say that random generation was a good enough
approximation to unique for all practical purposes.

To: LIST: [email-masked];
Xref: clerew local.usefor:20424
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Thu, 9 Dec 2004 15:16:14 GMT
Lines: 39
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Yes, we're agreed that the draft is in error and that the reality
>(as reflected in earlier WG discussion but not reflected in the
>draft text) is that the poster corresponds to the RFC 2822
>sender.

The word "poster" as actually used in our draft means either "author" or
"sender", and in most cases means both. Where the distinctin is important
we can and do use more precise wording. So please draw attention to cases
where there might still be problems.

I agree that the definition may need to be cleaned up, but let us identify
any problematic cases before doing that.

>Separate from the issue of the "poster" keyword in the
>Followup-To header field (as mentioned by Sebastian Brocks),
>which does not necessarily refer to the poster or the author(s).
>We could mention that oddity as an historical artifact, and/or
>we could plan a strategy for correcting the unfortunate
>mistake (long-term; keeping the term for backwards
>compatibility).

The word "poster" is in widespread usage throughout usenet and does not
seem to cause any real problems out there, and there would be nothing at
all to be gained trying to use anything else as the special keyword in the
Followup-To header.

To: LIST: [email-masked];
Xref: clerew local.usefor:20422
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: MD5 collision (was msg-id)
Date: Thu, 9 Dec 2004 12:17:41 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> Thorfinn <[email-masked]> writes:

>A "MUST" isn't some magical stick to be implemented in airy fairy theory
>land.  It butts up against the reality that is practice.  And in
>practice, hardly anyone is going to give a shit about once in every
>thousand years events (hell - I should know - I got an MD5 collision on
>two randomly generated numbers once on an automated MD5 module install
>test, shrugged and tried again), when there are far more critical issues
>affecting the problem.

Now that is rather interesting, because I have never seen a report of a
genuine MD5 collisionw.

AIUI, there are suspicions that the MD5 hashing scheme is not as secure as
it was once thought to be. And apparently the nature of the problem is
that, given a truly random set of strings to be hashed, they do not map
uniformly on to the 128-bit space of hash values. I.e. there exist hash
values that are more likely to be mapped into than others.

Which probably explains how you came to observe an event that, on pure
probability grounds, was otherwise expected to occur about once before the
heat-death of the universe.

To: LIST: [email-masked];
Xref: clerew local.usefor:20425
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Thu, 9 Dec 2004 15:19:58 GMT
Lines: 22
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Tue December 7 2004 19:56, Seth Breidbart wrote:

>> "MAY" means it's a matter of local policy.

>The entire mess being discussed is policy.  The draft is supposed
>to be about protocol. Policy doesn't belong in it.

The word "MAY", as defined in RFC 2119, is nearly always used to deal with
matters of policy. Protocols can and do contain optional features.

To: LIST: [email-masked];
Xref: clerew local.usefor:20419
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Thu, 9 Dec 2004 12:00:32 GMT
Lines: 37
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> John Stanley <[email-masked]> writes:

>>> I don't think so. When I was running a mail to news gateway, one of the
>>> first steps I was forced to take was to convert the mail message id
>>> into something legal for news, since a large number of them seemed to
>>> contain spaces. The injecting agent didn't accept that at all. It did
>>> not just compare everything from the first '<' to the next '>' or SP or
>>> just look for an '@' somewhere.

>> Indeed, injecting agents may well have to check for these funny cases
>> (but then that is what I said). But relaying and serving agents have no
>> time to perform such checks, and so will proceed on the assumption that
>> any broken cases will have been weeded out beforehand.

>Charles is indeed mistaken about what many relaying and serving agents do.
>John is correct, and not just for injecting agents.

Please can you be more explicit? Are you claiming that all relaying and
serving agents make full and exhaustive checks on the format of all
message-ids? To the same extent than injecting agents do, or should do?

Henry Spencer seems to agree that what I said in my last paragraph quoted
above was more or less correct.

To: LIST: [email-masked];
Xref: clerew local.usefor:20423
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Thu, 9 Dec 2004 15:07:02 GMT
Lines: 378
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>>If it is "undocumented", how does one know it is a tracing header?

>>With difficulty. But people have always managed to discover that
>>NNTP-Posting-Host was a tracing header.

>Were we talking only about NNTP-Posting-Host, you might have a point. We 
>are not. In fact, NNTP-Posting-Host is mentioned explicitely, so it is not
>included in the "other undocumented tracing header", and thus we aren't 
>talking about NNTP-Posting-Host at all. The question stands, if it is 
>undocumented, how does one know it is a "tracing header" versus any
>other kind? (Do we define 'tracing header' anywhere?)

The concept of "tracing header" is well known (it is described in RFC 2822
amongst other places). If an implementor of an injecting agent is aware of
some tracing header not explicitly mentioned in our draft, then it is
entirely reasonable for him to remove it, as stated.

>No, it says "it MAY allow articles in which headers contain unverified
>email addresses, that is, addresses which are not known to be valid for
>the trusted source,...". The implication is that the agent can tell which
>are valid (and thus which are not.)

No, it implies no such thing. Agents may or may not be able to detect
validity/invalidity of various classes of address, as Seth has pointed
out. The "MAY" permits them to make use of such information as they may
have available. Consequently, people who use invalid addresses may well
find their articles rejected. Or not.

>What is "exceptional" about a service that makes routine use of 
>reinjection?

The fact that very few services do that, and then only on a very limited
class of articles.

>>Do you really want me to change the first sentence to read
>>   "An Injecting Agent is responsible for taking a (proto-)article from a
>>   posting (or other) agent ..."   ?

>I'm sorry, but yes, I want the standard to accurate and correct. If that
>means changing a paragraph that says what kind of articles an agent deals 
>with, yes.

OK, I have made that change, even though I don't think it was necessary.

>>>Here's this artificial limitation again. Why must it be via EMAIL?

>>Because, for the worldwide Usenet, that is the way the protocol has been
>>defined, 

>> and there is no other machinery that is guaranteed to work.

>Excuse me? Nothing else can exist that can do the same thing? Sorry, but
>you are patently wrong.

No, there is nothing else that will nearly always work on Usenet as it
presently exists. A site that is smart enough to know that this group in
the Mozambiquean hierarchy is moderated, but has no other knowledge of
that hierarchy, will just email the article to moderators.isc.org, with a
very good chance that it will work. There is absolutely no other mechanism
that it can even try to use.

Other mechanisms may be used in small cooperating subnets, but until they
become commonplace we have no way of standardizing them. Cooperating
subnets, by definition, use mechanisms that are outside of our standard.

>The standard does not say 'again', it simply says "all groups likely to be 
>crossposted to". And yes, Charles, crossposting done once makes it very 
>likely to be done again, even if it had not been done already. But once is
>enough. 

>>The use of the word "likely" indicates that the
>>admin of the serving agent has to exercise some judgement here,

>Get real. The admin is not going to look at every report of every unknown 
>group to determine if it ought to be listed in the active list.

If he is getting 6 crossposts a day to the Mozambiquean hierarchy, then he
probably ought to be doing something about it. If he is getting one every
6 months, then there is little point in bothering.

In the event that he does inject it without an Approved header, no great
harm arises because it will get discarded as soon as it hits a relaying or
serving agent anywhere near Mozambique (where they are likely to care
about such things). And if it continues to propagate around some sites in
China, then who cares?

The fact is that it is impossible to prevent this sort of thing from
happening, hence there is no point pretending in our draft that we can.

>>So if the Injection-Date is earlier than the expiry date
>>(however computed) that it implements (probably of a per group basis), 
>>then it MUST drop it.

>I asked once, and I expected an answer. Why is this a MUST? Expiration 
>policy is the site's responsibility, not ours. 

This is nothing to do with controlling the expiration dates of sites. It
is a matter of preventing loops, for which purpose that MUST is essential.

Say the Injection-Date was 4 days ago. Suppose we expire everything (i.e.
remove all trace from our history file) after 3 days. Then it would be
folly to accept this article now. No problem it it had arrived yesterday,
at which point we would likely have seen that we had already got it.

>>>This paragraph is meaningless without an accompanying MUST for the 
>>>injection agent. 

>>Yes, you may have a point there. Would you suggest that I weaken the
>>wording here, or strengthen the wording under injecting agent ("SHOULD"
>>would probably suffice)? 

>SHOULD does not suffice. Regular people will see the RECOMMENDATION 
>synonym for SHOULD and configure their servers to reject .invalid 
>addresses, even though we explicitely tell people they MAY use them.
>In order for them to be able to use them, they have to work. That's
>a MUST.

>>AIUI, we don't want to prevent people doing that kind of munge, but we
>>don't want to force injecting agents to accept it either.

>We cannot have our cake and eat it, too. If we want people to use a 
>standard way of munging, then we have to make it a standard way of doing 
>it. It's patently silly to say "here's the authorized way of doing this, 
>but it might not be authorized...".

Whilst this is a legitimate topic for discussion (and indeed we are
discussing it), both Frank and Seth seem to he saying that it is about
right as I currently have it. I.e. we encourage people to use ".invalid",
but we make no guarantee that even that will always work.

I think we first need to decide whether that is the attitude we want to
take, and then adjust the wording to fit what we decide. I agree that the
way it is currently worded seems to be pulling two ways at once and will
probably need to be changed. But I need guidance as to which direction to
change it in.

>>Now that I have removed the text regarding MIME-style parameters that used
>>to be in there, I considered using "copied" instead of "taken". But since
>>it is the header content rather than the whole header that gets copied, I
>>concluded on balance that defining the special term "taken" would shorten
>>the wording, on balance. 

>There is no reason to redefine a standard english word when there is 
>already a standard english word that conveys the correct meaning.

"Take" is also a standard english word, and conveys the intended meaning
in this case. I do not see your problem with it. Does anyone else?

>>>And once again, when we specify ONE ACTION for an agent, that is THE 
>>>DEFAULT. We don't need to keep saying "by default", we already know
>>>that the one action we specify IS THE DEFAULT.

>>There are TWO actions for this agent. Either it uses the user's override
>>(if any) or else it uses the "default".

>Nonsense. We tell it what it must use. We do NOT tell the user what he 
>uses. Since there is only ONE thing that the followup agent is allowed to 
>do ("copy the content") that is the default. There is no other default.

There are TWO things the followup agent can do. One is to accept (without
demur) what the user types in. The other (if the user takes no action) is
to use the specified default (which is precisely why we call it the
"default"). But note that there is another ONE thing that it cannot (or
SHOULD NOT) do, and that is to use anything else as the default (such as
"Re[2]:").

>>      If the References header of the precursor has grown to excessive
>>      length, it MAY first be trimmed, but its first and last message
>>      identifiers MUST NOT be removed.

>No. It serves no purpose to trim the References header of the precursor.

I don't think anybody is going to take it that literally. However ...
 
>You want to trim the one in the followup. "If the resulting 
>References-header is excessively long, it may be trimmed by removing 
>message identifiers other than the first and the last."

But OK, I will buy that, except that it has to be the "last two" (i.e. you
MUST keep the last one from the precursor).

>>>A reading agent is free to show articles to the user in any manner the 
>>>user so desires. This is not limited to character sets.

>>It does not say it is limited to character sets. It says "such as
>>availability of charsets". 

>The point remains, the reading agent is under NO limitations on how it may 
>display articles. The "such as" implies there are limits.

The only limitations are those imposed by the lack of appropriate
facilities in the reading agent. The whole point of the wording is to
indicate that the poster cannot assume that every reading agent will be
able to make sense of everything, however bizarre, that he might choose to
put in the article.

>  We have no 
>business saying what it must or should do -- there are "reading 
>agents" we discuss later which display NOTHING to the user, they are 
>called "outgoing gateways".

Indeed, and that is covered by the wording "displays them (or processes
them in some other manner)". That covers gatewaying, printing, converting
into Braille, and whatever else.

>>>Irrelevant and meaningless. Elide.

>>The next sentence would be meaningless without it.

>And the next paragraph I wrote already showed the next sentence is
>meaningless and ought to be removed. "Keep this meaningless sentence
>because otherwise the next meaningless sentence would still be
>meaningless" doesn't convince me we ought to keep either one.

Neither sentence is meaningless. Does either of them say anything that is
untrue?					

>>   .... The operation of the outgoing gateway
>>   is subject to additional constraints due to the possibility of one or
>>   more corresponding incoming gateways back from that medium to
>>   Netnews, since this opens the possibility of loops.

>We already cover the possibility of loops, so there is no need for this 
>paragraph at all.

There are various ways in which loops can arise. It is necessary to draw
attention to them.

>>>I fail to see how a comment in the body of a message helps to prevent 
>>>loops.

>>There might be no other place to put it in the other medium. 

>Oh, ok. Wait. I still don't see how a comment in the body of a message 
>helps prevent loops.

>>But the
>>gateway that put it there should be able to find and recognize it if it
>>ever comes back to the same gateway,

>It is unlikely that an outgoing gateway is going to see an incoming 
>message.

On the contrary, that is just the kind of loop we are trying to catch. If
the article is gatewayed to the "other medium" and there exists some other
gateway elsewhere that puts it back into Netnews, then the flooding
algorithm is very likely indeed to bring it back to the first gateway.

> However, if you meant that the same outgoing gateway is going to 
>see the same message outgoing more than once, then it already has the 
>mandatory message id (not in the body) to work with.

There might not be such a thing as a message id in the other medium. There
might not even be a distinction between headers and body in the other
medium.

> A comment in the body 
>of a messsage is not likely to prevent loops. Saying it does is silly.

No. If the gateway to the "other medium" (which presumably understands the
rules for contructing messages in the other medium) can manage to insert
some distinct identifier into whatever passes for a body in the other
medium, then it is quite capable (and indeed it is the only agent in the
whole world with that capability) of recognizing the identifier that it
itself placed there, should it ever come back again. Of course, it would
have been much better to put the identifier into the "headers" of the
other medium if that were possible, but if it were not then it has to go
in the "body".

>>>>   The incoming gateway has the serious responsibility of ensuring that
>>>>   all of the requirements of this standard are met by the articles that
>>>>   it forms.

>>>ALL? Or do you mean that it must meet the standards expected by the 
>>>injecting agent that it will be using?

>>It is responsible for ensuring that proper checks are done. If it can
>>trust a local injecting agent to do the checks properly, then that is
>>fine.

>That's not what our draft says. It says that IT has the responsibility, 
>not the injecting agent. And it talks about articles IT forms, not the 
>final form as it would appear on the wire.

It is nowhere stated that a separate injecting agent is involved in an
incoming gateway. If there is, then it can be relied on to do the checks.
If there is not, then the gateway has to shoulder the responsibilities of
an injecting agent. That is exactly what it says, so what is your problem?

>>>I guess you did mean "all". I disagree. An incoming gateway does NOT need 
>>>to be an injecting and relaying agent too. It can function quite well 
>>>below the injection agent. 

>>How it organizes itself internally is none of our business, just so long
>>as its duties and responsibilities are taken care of.

>Then why are we saying that IT has the responsibility for ALL of the 
>standard, and not just the parts for the function it is performing? 

Because we do not state which parts for the function it is performing. It
can do the whole thing itself, or it can subcontract parts of it to other
agents available to it. The internal details are none of our business,
just so long as it gets done.

>>>What is the reason for this RFC2119 mandate? The injecting agent will stop 
>>>a duplicate.

Injecting agents are not normally responsible for stopping duplicates;
that is usually the function of a serving agent. But in gatewaying
situations the article may have got so mangled that no other agent would
be able to stop them.

>7.2.2.  Procedure to be followed by Injecting Agents

>   4. It MUST reject any article that does not have the proper mandatory
>      headers for a proto-article (or, when reinjecting, all the
>      mandatory headers other than Injection-Date), or which contains
>      any header that does not have legal contents.  

>It takes a bit of thought there to realize that a duplicated message id is 
>an illegal content for the message id header. And most injecting agents 
>do, indeed, reject duplicates.

How?

And if it is the same message come round again, then it is not even
illegal to have the same message id.

>>>Cool. My gateway's method of determining a "valid equivalent" is by seeing
>>>a Control header. But I must remove it, even though I am permitted to 
>>>put it in. Which is it?

>>I would suggest checking for and noting the presence of the cancel message
>>_before_ removing it.

>There is no cancel message before removing it. I am told I must remove it 
>before I can inject it.

Eh? If an article with a Supersedes header arrives at an outgoing gateway,
then it had better notice the presence of Supersedes (so that it can
generate whatever passes for a cancel message in the other medium) BEFORE
it removes that header (for the purpose of passing the rest of the headers
and the body to the other medium). If it tries to do things in the wrong
order, then it will certainly fail to do a proper job.

Date: Wed, 8 Dec 2004 10:28:57 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

Seth Breidbart <sethb@xxxxxxxxx>:

>> It does not mean it cannot happen tomorrow or the day after. It does
>> not mean it will only happen in one thousand years.

>I understand that.  It means it will happen, on average, one thousand
>time in one million years.

No, it does not. It means that has an X percent chance of happening FOR
EVERY ID GENERATED, which, assuming a rate of Y ids generated per thousand
years, multiplies out to 1. It can happen one thousand times in the next
two days, for all you know.

What is important is that the ERROR is known and can be avoided completely
by fixing it now.

>> Then you would know that "once per thousand years" is 1) a pretty 
>> meaningless statistic

>It has a clear meaning (providing the details are spelled out: how
>many users posting how often, etc.)

You would have to know how many users there will be TOMORROW and every day 
for the next thousand years, AND how many articles they will be posting 
tomorrow and every day after that for the next thousand years. Since that 
cannot be known, it is a meaningless statistic. Now, "X % chance" has some 
meaning. "Once per thousand years" does not.

>It's a flawed implementation.

Sorry, I guess for the truly pedantic I should have said it is not in any 
way a correct implementation of "MUST NOT". Yes, it is flawed. It can be 
fixed. We ought not to support flawed implementations.

>So is any other, since whatever method
>you use might generate a Message-ID that was previously used by a
>previous owner of the domain (or might sometime in the future be used
>by a subsequent owner).

Hmmm. Let's see. I use "sequence number dot epoch time dot process id dot
month dot day dot year @ domain". Since THIS epoch time will never
reoccur, it cannot be the same as what any other owner of my domain would
use. Guaranteed unique.

>>>They both cause the same violation of the same MUST.
>> No, they don't,

>They both cause a non-unique Message-ID.  

Your wife has a baby. It could either be caused by YOUR romantic actions, 
or the romantic actions of the milkman. There is no difference because the 
result was the same.

The breakdown in the piece of hardware causes a mangled or non-existant 
message id. How can a broken-down computer generate anything? A flawed 
implementation generates a valid-looking but non-unique message id. 
There's a significant difference between the two.

>> and even were that true, they are still very different things.

>I didn't say they were the same thing, I said they had the same
>effect.

No, you said they caused the SAME violation of the same MUST. Even were
that true, the two causes are very different. "We shouldn't require
adherance to a MUST because something else might break and violate the
MUST" is just silly.

>You don't think ISPs will know what software they're running?  Maybe
>you should avoid such ISPs.  I do.

I don't think many ISPs know what software their outsourced news servers 
run.

>> You are choosing to bother by participating in this discussion. Make up 
>> your mind.

>I am not at all bothered by the expected once in the next thousand
>years Message-ID collision.

I know. Your error.

<thorfinn@xxxxxxxxxxxxxx>:

>> The "error" is not whether you think that is "good enough", but on the
>> meaning of "once per thousand years". It does not mean it cannot happen 
>> tomorrow or the day after. It does not mean it will only happen in
>> one thousand years.

>You seem to not get it.

Casinos make millions off of people like you. "I'll bet '20' on the
roulette wheel because it hasn't shown up for a long time and it must show
up once every thirty-something spins." "I'll not bet '20' on the next spin
because it hit this time, and it cannot show up for another thirty spins."

>> Then you would know that "once per thousand years" is 1) a pretty 
>> meaningless statistic

>Err, no, it's not meaningless.  It means that it's an extremely low
>probability event.  

No, actually it doesn't. It someone tosses one coin per thousand years and 
you say it has a once per thousand year chance of coming up heads, you've
just said it is a certainty to come up heads.

>> and 2) not in any way an implementation of "MUST 
>> NOT".

>"The computer crashes" isn't an implementation of MUST NOT, either.

Ah, yes, this old "we cannot specify MUST because a computer somewhere 
might crash and violate it anyway."

Hey, I know. Let's stop mandating seat belt use, because you know, a 16 
ton weight might fall on the car and you'd be dead anyway!

>Or, how about the probability that a meteor will land on your head? 

And I thought MY example was being ridiculous. Ok, you win. Remove any 
mention that the message id MUST be unique, obviously a meteor might fall 
on my head and cause me to create one that is not, and it is more 
important that the message get posted than the system be able to prevent 
duplicates.

Date: Wed, 8 Dec 2004 10:01:01 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

Seth Breidbart <sethb@xxxxxxxxx>:

>No, the implication is that the agent can _sometimes_ tell that an
>email address is (known to be) valid, and sometimes cannot tell.

Were that as far as it goes, fine. It isn't. The next step is the 
pretense. "You may not use this address".

>An unverified email address is one that has not been verified.  It
>might not be verified because it's bogus, or it might not be verified
>because nobody got around to verifying it.

This statement demonstrates the farce. It might not be verified because
it is impossible to verify. The ISP through which I post has no idea what
other domains I own, much less have access to email through, and in the 
domains I own, I can create addresses at whim. They have no way of 
verifying any of them. But we'll tell them they can pretend to know which 
ones are invalid. 

> There is no requirement for a general solution.

This is a draft convering the standard for news, not just your private 
implimentation.

>No, it is saying "I do not know this to be valid so _I_ won't let you
>use it."

Thus the pretense. If it cannot make the determination, it should do 
nothing different. 

>> B. Not if we are going to tell users that they may use .invalid.

>"MAY" means it's a matter of local policy.

MAY is a permissive word meaning they are permitted to do something. If 
they are not permitted to do something, then we are wrong.

>The probability that it _will have been_ crossposted to is unity.  The
>probability that it _will be_ crossposted to is less than unity.

The draft makes no such fine distinction in tenses. "Likely to be 
crossposted to" is open-ended. Even so, given the instructions we provide 
for followup agents, and observed conditions on the net, a group that HAS
BEEN crossposted to is very likely to be crossposted to again. Not just 
"likely", VERY likely.

>I disagree.  A site might well understand the risks behind violating
>that SHOULD and choose to do so.

"Understand the risks" is not sufficient to violate a SHOULD, as defined 
by some members here. According to RFC2119, it is.

>No, it doesn't.  Crossposting to a group whose name was clearly
>generated by a cat walking on the keyboard is not likely to be
>repeated.

Read USENET for awhile and see if actual practice doesn't contradict you.

>Only if _he thinks_ it's likely to happen.

The fact that it has happened makes the likelyhood of it happening unity.

>Are you likely to graduate from high school?

Since I have already, the answer is yes, I am very likely to graduate. It 
is a certainty.

>> SHOULD does not suffice. Regular people will see the RECOMMENDATION 
>> synonym for SHOULD and configure their servers to reject .invalid 
>> addresses, even though we explicitely tell people they MAY use them.

>No, we tell sites they MAY accept them.

We ought to tell them MUST, since we also tell users they MAY use them.

>> In order for them to be able to use them, they have to work. That's
>> a MUST.

>What is the global harm if a site requires posts to have verifiable
>local addresses?

What is the global harm saying they must accept the standard way of 
denoting an invalid address?

>"Here's the authorized way of doing this.  Sites will adopt their own
>policies, which may be weaker or stronger."

"Here's the standard way of doing things, but it might not be standard
where you are because we didn't actually make it part of the requirements 
for the standard." Foof.

Bruce Lilly <blilly@xxxxxxxxx>:

>But you still mixed up posting with authoring --

No, I am not. Please read what I wrote.

>an author might
>use a posting agent to prepare an article specifically for news,
>or a message prepared by one or more authors might be submitted
>by a poster to an injection agent, but one does not submit articles
>to a posting agent, at least not as that term is defined in the draft.
 
I wish I could find some significance in what you wrote there. The main 
job of the posting agent is to help post articles. They can be authored 
anywhere. See -- "posting agent", post. So "poster" ought to mean "one who 
posts", not "one who composes".

>Separate from the issue of the "poster" keyword in the
>Followup-To header field (as mentioned by Sebastian Brocks),
>which does not necessarily refer to the poster or the author(s).

If one is going to bring up the issue of "authors who write without 
intention of their material being posted to USENET", then one ought to 
recognize that those "authors" are not going to be interested in getting 
followups to those unintended articles, thus "poster" is the correct 
destination for them.

>           But if
>we are going to deal with it in this document, then we ought
>to do so adequately -- and simply waving hands and (effectively)
>saying "the message gets to the moderator via magic" isn't
>adequate.

Nobody has suggested using that terminology. Magic isn't involved. I don't
think anybody writing a new server is going to see "the submission is sent
to the moderator" and assume that he can do anything he pleases and have
it work with the existing infrastructure. On the other hand, if someone
comes up with a new way of doing things tomorrow, we ought not to be
shutting him out because his system doesn't use email.

Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx>:

>> there is no MUST about it, since a site may very well say
>> "we require all our users to use valid and working email
>>  addresses".

>Yes, that was the idea, and as a consequence TLD .invalid did
>not work on this server.  And what you've written in the draft
>addresses it perfectly.

Let's see. My ISP goes anal and tells me "you must use a valid and working
email address in your postings." (Forget for the moment that THEY are
inserting a non-working address in my articles for me at the present time.
Forget for the moment that they are also outsourcing news to another
company.) Ok. We'll even assume that they have some magic method of
determining whether or not an "@peak.org" address is valid and working.  
(Forgetting for the moment that they allow the use of procmail and +
addresses, which means an address that works today might not work
tomorrow.)

So, I use an address from one of my other domains. They say "no". And yet
I'm using a valid and working address. I'm following the stated rule, and 
yet, I am not permitted to post. Why is that? Because WE told them they 
can PRETEND to know something that is patently unknowable. We told them 
THEY can determine what is a valid address (for use in news), when the 
truth is they cannot.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Wed, 08 Dec 2004 05:24:36 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 48
Sender: [email-masked]

Charles Lindsey wrote:

> if they can detect the bad ones (they might even check with
> the DNS that the domain does not exist) then they have the
> right to refuse them (according to local site policy).

That's what news.t-online.de does (or at least did), and IIRC
they even checked the complete address if it was one of their
own user@t-online

> there is no MUST about it, since a site may very well say
> "we require all our users to use valid and working email
>  addresses".

Yes, that was the idea, and as a consequence TLD .invalid did
not work on this server.  And what you've written in the draft
addresses it perfectly.

 [Message-ID as "comment"] 
> There might be no other place to put it in the other medium.

Time to close or burn this gateway and whatever is behind it.

> that is how a loop would be prevented.

Do you have an example ?  Is this a news-irc-news scenario ?

>>> where there is no one "official" gateway, some other method
>>> of generating message identifiers has to be used to avoid
>>> collisions.

Gatebau94 tried to define that for Fido gateways among others.

> people who maintain gateways need to be aware of the way
> things actually happen within the particular group (or set
> of related groups) concerned.

In other words "don't use GiGo", now that is very old news ;-)
Add some more MUST-NOTs to this chapter, the gateway operator
MUST be terrified.

> there just is no "one size fits all" solution to the
> gatewaying problem. Every case is different.

"Don't make up Message-IDs without your lawyer" MUST be clear.

                           Bye, Frank

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Seth Breidbart <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Tue, 7 Dec 2004 21:21:58 -0500
Cc: [email-masked]
Sender: [email-masked]

On Tue December 7 2004 19:56, Seth Breidbart wrote:
> 
> John Stanley <[email-masked]> wrote:
> > "Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:
> 
> >> It says they "MAY" allow certain articles.
> >
> > No, it says "it MAY allow articles in which [...]

> No, the implication is that the agent can _sometimes_ tell [...]

> "MAY" means it's a matter of local policy.

The entire mess being discussed is policy.  The draft is supposed
to be about protocol. Policy doesn't belong in it.

> Besides, email isn't guaranteed to work, either.  What is important is
> that the article be provided to the moderator for a decision.  The
> means of providing it is irrelevant.

Perhaps.  There are many issues involved with moderation that
are inadequately specified.  One approach would be to simply
defer dealing with moderation in a separate document.  But if
we are going to deal with it in this document, then we ought
to do so adequately -- and simply waving hands and (effectively)
saying "the message gets to the moderator via magic" isn't
adequate.

The following text in the previous message pertained to
some silliness which, when tracked back a half-dozen earlier
messages turned out to have been related to just one such
moderation issue; the assumption implicit in the text that
there is exactly one "moderation status" for every newsgroup,
uniform across all sites.  We had that discussion a long
time ago, and I recall that the conclusion was that
moderation status was site policy, not something that
could be characterized as *the* moderation status.  If it's
a policy issue it doesn't belong in a protocol document.
If we can agree on a clear and complete description of how
moderation and moderation-related issues work, we should
probably defer the matter to a separate document.  A lot
of pointless bickering about cats walking on keyboards, the
probability of this and that, etc. will only delay getting our
Standards Track documents finished.

Date: Wed, 8 Dec 2004 13:21:37 +1100
From: Thorfinn <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

On Tue 07 Dec 2004 at 05:36:56PM -0800, in <[email-masked]>,
John Stanley <[email-masked]> wrote:
> Seth Breidbart <sethb@xxxxxxxxx>:
> >Then you're wrong.   I think one collision, total, on average every
> >thousand years is good enough. 
> The "error" is not whether you think that is "good enough", but on the
> meaning of "once per thousand years". It does not mean it cannot happen 
> tomorrow or the day after. It does not mean it will only happen in
> one thousand years.

You seem to not get it.

> > I also understand probability theory quite well.
> Then you would know that "once per thousand years" is 1) a pretty 
> meaningless statistic

Err, no, it's not meaningless.  It means that it's an extremely low
probability event.  Sufficiently low in probability, that the likelihood
of other untoward things happening (like the computer crashing between
pressing send and the server getting it, or the likelihood that relying
on the FQDN part being unique will generate a collision) is *much*
higher in importance.

> and 2) not in any way an implementation of "MUST 
> NOT".

"The computer crashes" isn't an implementation of MUST NOT, either.

> >> A breakdown in a piece of hardware is different than a programmed, known
> >> flaw.
> >They both cause the same violation of the same MUST.
> No, they don't, and even were that true, they are still very different 
> things. The existance of one does not mean the other is unimportant.

It's a question of scale.  Do you want to worry about the fact that it
is possible that your component atoms could suddenly decide to separate
and fly in all different directions?  That could happen you know.  It's
extremely low probability... for some reason, most of us choose to
ignore it.

Or, how about the probability that a meteor will land on your head?  Or
a plane?  A car crashing into you as you walk down the street?

A "MUST" isn't some magical stick to be implemented in airy fairy theory
land.  It butts up against the reality that is practice.  And in
practice, hardly anyone is going to give a shit about once in every
thousand years events (hell - I should know - I got an MD5 collision on
two randomly generated numbers once on an automated MD5 module install
test, shrugged and tried again), when there are far more critical issues
affecting the problem.

> >Nothing.
> Oh. Then the chances of it breaking are just as good then as at any other 
> time. In fact, hitting "send" is likely to activate the disk, which is a 
> time when failures would be more likely. And since computers fail more 
> often than "once per thousand years", on average ...

Then there are rather more important issues than worrying about once per
thousand years events.

> >So you'll have to make the effort of asking them, instead of
> >expecting the information to be thrust upon you.
> 
> And THEY will know the answer because? The magic information fairy tells 
> them?

Well, presumably, if they're running the software that generates the
Message-ID, they can find out, either by asking the vendor or inspecting
the source code.

Later,

  Thorf

-- 
<a href="http://tertius.net.au/~thorfinn/">[email-masked]</a>
"Rabbit or gerbil, rabbit or gerbil?"
    -- Benno!jeamland.net

Date: Tue, 7 Dec 2004 20:49:17 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

John Stanley <[email-masked]> wrote:
> Seth Breidbart <sethb@xxxxxxxxx>:

>>>> To whom are you attributing that error?
>>> To everyone who says that "once per thousand years" is good enough.
>>Then you're wrong.   I think one collision, total, on average every
>>thousand years is good enough. 
>
> The "error" is not whether you think that is "good enough", but on the
> meaning of "once per thousand years".

I fully understand the meaning of "once per thousand years".  I also
think it's good enough.  Hence, you are attributing the error to me,
and I'm not making it.  Therefore you're wrong.

> It does not mean it cannot happen tomorrow or the day after. It does
> not mean it will only happen in one thousand years.

I understand that.  It means it will happen, on average, one thousand
time in one million years.

>> I also understand probability theory quite well.
> Then you would know that "once per thousand years" is 1) a pretty 
> meaningless statistic

It has a clear meaning (providing the details are spelled out: how
many users posting how often, etc.)

> and 2) not in any way an implementation of "MUST NOT".

It's a flawed implementation.  So is any other, since whatever method
you use might generate a Message-ID that was previously used by a
previous owner of the domain (or might sometime in the future be used
by a subsequent owner).

>>> A breakdown in a piece of hardware is different than a programmed, known
>>> flaw.
>>They both cause the same violation of the same MUST.
> No, they don't,

They both cause a non-unique Message-ID.  That's the same violation of
the same MUST.

> and even were that true, they are still very different things.

I didn't say they were the same thing, I said they had the same
effect.

>>> Really? What is special about "finishing editing the proto-article" that
>>> protects the computer from failure until hitting "send"?
>>Nothing.
> Oh. Then the chances of it breaking are just as good then as at any other 
> time. In fact, hitting "send" is likely to activate the disk, which is a 
> time when failures would be more likely. And since computers fail more 
> often than "once per thousand years", on average ...

Yes, and spend what fraction of their time between my finishing the
proto-article and my hitting send?

>>So you'll have to make the effort of asking them, instead of
>>expecting the information to be thrust upon you.
> And THEY will know the answer because? The magic information fairy tells 
> them?

You don't think ISPs will know what software they're running?  Maybe
you should avoid such ISPs.  I do.

>>>> You don't have to consider that a problem.
>>> There are lots of things I don't HAVE to consider a problem that are still
>>> problems.
>>You are welcome to worry about them as much as you want to.  I choose
>>not to bother.
> You are choosing to bother by participating in this discussion. Make up 
> your mind.

I am not at all bothered by the expected once in the next thousand
years Message-ID collision.

Seth

Date: Tue, 7 Dec 2004 17:36:56 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

Seth Breidbart <sethb@xxxxxxxxx>:

>>> To whom are you attributing that error?
>> To everyone who says that "once per thousand years" is good enough.

>Then you're wrong.   I think one collision, total, on average every
>thousand years is good enough. 

The "error" is not whether you think that is "good enough", but on the
meaning of "once per thousand years". It does not mean it cannot happen 
tomorrow or the day after. It does not mean it will only happen in
one thousand years.

> I also understand probability theory quite well.

Then you would know that "once per thousand years" is 1) a pretty 
meaningless statistic and 2) not in any way an implementation of "MUST 
NOT".

>> A breakdown in a piece of hardware is different than a programmed, known
>> flaw.

>They both cause the same violation of the same MUST.

No, they don't, and even were that true, they are still very different 
things. The existance of one does not mean the other is unimportant.

>> Really? What is special about "finishing editing the proto-article" that
>> protects the computer from failure until hitting "send"?

>Nothing.

Oh. Then the chances of it breaking are just as good then as at any other 
time. In fact, hitting "send" is likely to activate the disk, which is a 
time when failures would be more likely. And since computers fail more 
often than "once per thousand years", on average ...

>So you'll have to make the effort of asking them, instead of
>expecting the information to be thrust upon you.

And THEY will know the answer because? The magic information fairy tells 
them?

>>> You don't have to consider that a problem.
>> There are lots of things I don't HAVE to consider a problem that are still
>> problems.

>You are welcome to worry about them as much as you want to.  I choose
>not to bother.

You are choosing to bother by participating in this discussion. Make up 
your mind.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: John Stanley <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Tue, 7 Dec 2004 20:17:57 -0500
Cc: [email-masked]
Sender: [email-masked]

On Tue December 7 2004 11:50, John Stanley wrote:

> Bruce Lilly <blilly@xxxxxxxxx>:
> 
> >An author or group of authors may compose some content
> >without specifically composing it *for submission to a posting
> >agent*. 
> 
> That's nice. Also irrelevant. Their "intent" is not the important part of 
> that sentence, their ACTION is. And the action is "composes". 

But you still mixed up posting with authoring -- an author might
use a posting agent to prepare an article specifically for news,
or a message prepared by one or more authors might be submitted
by a poster to an injection agent, but one does not submit articles
to a posting agent, at least not as that term is defined in the draft.
 
> >No, I'm not arguing.  I am confirming as fact what you said
> >you "thought" happened, and stated that "poster" (as used
> >in text, not as a keyword) corresponds to RFC 2822 "sender".
> 
> No, "poster" as used in the draft does NOT correspond to RFC2822 "sender".
> It ought to, but as it currently is written, it does not. That's why I
> brought it up.

Yes, we're agreed that the draft is in error and that the reality
(as reflected in earlier WG discussion but not reflected in the
draft text) is that the poster corresponds to the RFC 2822
sender.

Separate from the issue of the "poster" keyword in the
Followup-To header field (as mentioned by Sebastian Brocks),
which does not necessarily refer to the poster or the author(s).
We could mention that oddity as an historical artifact, and/or
we could plan a strategy for correcting the unfortunate
mistake (long-term; keeping the term for backwards
compatibility).

Date: Tue, 7 Dec 2004 19:56:34 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

John Stanley <[email-masked]> wrote:
> "Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>> It says they "MAY" allow certain articles.
>
> No, it says "it MAY allow articles in which headers contain unverified
> email addresses, that is, addresses which are not known to be valid for
> the trusted source,...". The implication is that the agent can tell which
> are valid (and thus which are not.)

No, the implication is that the agent can _sometimes_ tell that an
email address is (known to be) valid, and sometimes cannot tell.

An unverified email address is one that has not been verified.  It
might not be verified because it's bogus, or it might not be verified
because nobody got around to verifying it.

> You are defending a very general statement by using one specific
> case.  Defend the general case. Yes, SOME addresses are easily
> determined to be "not valid for the user", but that specificity does
> not imply a general solution.

There is no requirement for a general solution.  "MUST post if the
email address is valid, MUST NOT post if the email address is not
valid" requires a general solution.

>>But there is no
>>MUST about it, since a site may very well say "we require all our users 
>>to use valid and working email addresses".
>
> A. Here we are again with the agent pretending it can know what addresses
> are not valid for a poster. If the agent says "you are required to use
> a valid address" and rejects the one being presented to it, it is saying
> "this is not valid." 

No, it is saying "I do not know this to be valid so _I_ won't let you
use it."

> B. Not if we are going to tell users that they may use .invalid.

"MAY" means it's a matter of local policy.

>>>Here's this artificial limitation again. Why must it be via EMAIL?
>
>>Because, for the worldwide Usenet, that is the way the protocol has been
>>defined, 
>
> We must define it this way because it is defined this way. No, we can 
> change this. We do NOT have to define it that way. It is simply a fact 
> that other transports can work, even if they are not currently widely 
> used.

That's correct.

>> and there is no other machinery that is guaranteed to work.
>
> Excuse me? Nothing else can exist that can do the same thing? Sorry, but
> you are patently wrong.

Besides, email isn't guaranteed to work, either.  What is important is
that the article be provided to the moderator for a decision.  The
means of providing it is irrelevant.

> Of course it does. If an unknown newsgroup name appears in a
> newsgroups header (where another, known name appears), then we
> already know that there is a GOOD likelyhood (probability is ONE)
> that that group is likely to be crossposted to. I.e., it WAS
> crossposted to at least once, so the probability that it will be
> crossposted to is unity.

The probability that it _will have been_ crossposted to is unity.  The
probability that it _will be_ crossposted to is less than unity.

> Combined with the strength of that SHOULD, that means that it is as
> good as a MUST.

I disagree.  A site might well understand the risks behind violating
that SHOULD and choose to do so.

>>A group that is seen to have been crossposted to "once" is not
>>thereby a group that is "likely to be crossposted to" again.
>
> The standard does not say 'again', it simply says "all groups likely
> to be crossposted to".

Which implies in the future, not the past.  (Ask someone divorced "Are
you likely to be married to <your ex-spouse>?" and see if the
likelihood is closer to 0 or 1.)

> And yes, Charles, crossposting done once makes it very 
> likely to be done again, even if it had not been done already.

No, it doesn't.  Crossposting to a group whose name was clearly
generated by a cat walking on the keyboard is not likely to be
repeated.

>>The use of the word "likely" indicates that the
>>admin of the serving agent has to exercise some judgement here,
>
> Get real. The admin is not going to look at every report of every unknown 
> group to determine if it ought to be listed in the active list.

So he'll have code to do that.

> In fact, we tell him that he SHOULD include it in the list -- and
> that SHOULD is one of your SHOULDs.

Only if _he thinks_ it's likely to happen.

> There is no judgement necessary -- it happened once,
> that means it is likely to happen.

Are you likely to graduate from high school?

>>So if the Injection-Date is earlier than the expiry date
>>(however computed) that it implements (probably of a per group basis), 
>>then it MUST drop it.

That seems backwards; I keep posts for a week, so expiration is
today+7.  Injection had better be earlier than that.

Or do you mean that if the expiry date based on Injection-Date is
before now, I MUST drop it?  Expiry is often not based on
Injection-Date.

> I asked once, and I expected an answer. Why is this a MUST? Expiration 
> policy is the site's responsibility, not ours. 

That, too.  There's no global harm if a post expires too soon, or
never.

> SHOULD does not suffice. Regular people will see the RECOMMENDATION 
> synonym for SHOULD and configure their servers to reject .invalid 
> addresses, even though we explicitely tell people they MAY use them.

No, we tell sites they MAY accept them.

> In order for them to be able to use them, they have to work. That's
> a MUST.

What is the global harm if a site requires posts to have verifiable
local addresses?

>>AIUI, we don't want to prevent people doing that kind of munge, but we
>>don't want to force injecting agents to accept it either.
>
> We cannot have our cake and eat it, too. If we want people to use a 
> standard way of munging, then we have to make it a standard way of doing 
> it. It's patently silly to say "here's the authorized way of doing this, 
> but it might not be authorized...".

"Here's the authorized way of doing this.  Sites will adopt their own
policies, which may be weaker or stronger."

Seth

Date: Tue, 7 Dec 2004 16:13:14 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>If it is "undocumented", how does one know it is a tracing header?

>With difficulty. But people have always managed to discover that
>NNTP-Posting-Host was a tracing header.

Were we talking only about NNTP-Posting-Host, you might have a point. We 
are not. In fact, NNTP-Posting-Host is mentioned explicitely, so it is not
included in the "other undocumented tracing header", and thus we aren't 
talking about NNTP-Posting-Host at all. The question stands, if it is 
undocumented, how does one know it is a "tracing header" versus any
other kind? (Do we define 'tracing header' anywhere?)

> It says they "MAY" allow certain articles.

No, it says "it MAY allow articles in which headers contain unverified
email addresses, that is, addresses which are not known to be valid for
the trusted source,...". The implication is that the agent can tell which
are valid (and thus which are not.)

>But if they can
>detect the bad ones (they might even check with the DNS that the domain
>does not exist) 

You are defending a very general statement by using one specific case. 
Defend the general case. Yes, SOME addresses are easily determined to be
"not valid for the user", but that specificity does not imply a general 
solution.

>Essentially, it is encouraging you
>to take advantage of that "MAY" in this circumstance.

Then we ought to say that. But the "MUST" is important for a latter 
section of the draft to be meaningfull, so it ought to say that.

>But there is no
>MUST about it, since a site may very well say "we require all our users 
>to use valid and working email addresses".

A. Here we are again with the agent pretending it can know what addresses
are not valid for a poster. If the agent says "you are required to use
a valid address" and rejects the one being presented to it, it is saying
"this is not valid." 

B. Not if we are going to tell users that they may use .invalid.

>>So, either an injection agent doesn't deal just with "proto-articles", or
>>there needs to be a "reinjection agent".

>The first sentence of 7.2 indicates the normal situation in which an
>injecting agent is given a proto-article.

The first sentence mentions nothing about "normal". It tells us what kind
of articles the agent deals with: proto-articles.

Reinjected articles are NOT proto-articles; thus injecting agents do not 
deal with them. Thus spake paragraph one of the section dealing with 
injecting agents. 

>A couple of paragraphs further
>on it states that, "in exceptional circumstances" 

What is "exceptional" about a service that makes routine use of 
reinjection?

>Do you really want me to change the first sentence to read
>   "An Injecting Agent is responsible for taking a (proto-)article from a
>   posting (or other) agent ..."   ?

I'm sorry, but yes, I want the standard to accurate and correct. If that
means changing a paragraph that says what kind of articles an agent deals 
with, yes.

>>Here's this artificial limitation again. Why must it be via EMAIL?

>Because, for the worldwide Usenet, that is the way the protocol has been
>defined, 

We must define it this way because it is defined this way. No, we can 
change this. We do NOT have to define it that way. It is simply a fact 
that other transports can work, even if they are not currently widely 
used.

> and there is no other machinery that is guaranteed to work.

Excuse me? Nothing else can exist that can do the same thing? Sorry, but
you are patently wrong.

>>Knowing as I do the interesting interpretation of SHOULD that most folks 
>>here have, I have to ask whether this sentence is saying nothing less than 
>>that a serving agent must list all groups that it has ever heard of. I 
>>mean, a "strong recommendation that can be avoided only after careful 
>>study and consideration of all possible negative outcomes" combined with 
>>the guessing game of "group likely to be crossposted to" results in "all 
>>groups ever heard mentioned at least once".

> Not, it doesn't result in that. 

Of course it does. If an unknown newsgroup name appears in a newsgroups
header (where another, known name appears), then we already know that
there is a GOOD likelyhood (probability is ONE) that that group is likely
to be crossposted to. I.e., it WAS crossposted to at least once, so the
probability that it will be crossposted to is unity. Combined with the 
strength of that SHOULD, that means that it is as good as a MUST.

>A group that is seen to have been
>crossposted to "once" is not thereby a group that is "likely to be
>crossposted to" again.

The standard does not say 'again', it simply says "all groups likely to be 
crossposted to". And yes, Charles, crossposting done once makes it very 
likely to be done again, even if it had not been done already. But once is
enough. 

>The use of the word "likely" indicates that the
>admin of the serving agent has to exercise some judgement here,

Get real. The admin is not going to look at every report of every unknown 
group to determine if it ought to be listed in the active list. In fact, 
we tell him that he SHOULD include it in the list -- and that SHOULD is 
one of your SHOULDs. There is no judgement necessary -- it happened once,
that means it is likely to happen. Whether there is some VALUE when it 
happens is a different issue.

>So if the Injection-Date is earlier than the expiry date
>(however computed) that it implements (probably of a per group basis), 
>then it MUST drop it.

I asked once, and I expected an answer. Why is this a MUST? Expiration 
policy is the site's responsibility, not ours. 

>>However, we've just told it that it SHOULD include that group in the list 
>>of groups that it keeps. After all, if it appears in a Newsgroups header, 
>>it is quite likely that it will be crossposted to.

>No, we have not told it that. 

Quote from the draft:

   A serving agent MUST maintain a list of the newsgroups it stores in
   its news database showing the moderation status of each one (see
   6.2.1), and SHOULD include in that list all groups likely to be
   crossposted to from those groups (e.g. all other groups in the same
   hierarchy(ies)).

I see "SHOULD include" clearly stated there. Oh, wait, I remember, you 
think that a group that has ALREADY been crossposted to isn't likely to
be crossposted to. I remember as well that the Captain of the Titanic 
thought the Titanic wasn't likely to sink, until it did.

>>This paragraph is meaningless without an accompanying MUST for the 
>>injection agent. 

>Yes, you may have a point there. Would you suggest that I weaken the
>wording here, or strengthen the wording under injecting agent ("SHOULD"
>would probably suffice)? 

SHOULD does not suffice. Regular people will see the RECOMMENDATION 
synonym for SHOULD and configure their servers to reject .invalid 
addresses, even though we explicitely tell people they MAY use them.
In order for them to be able to use them, they have to work. That's
a MUST.

>AIUI, we don't want to prevent people doing that kind of munge, but we
>don't want to force injecting agents to accept it either.

We cannot have our cake and eat it, too. If we want people to use a 
standard way of munging, then we have to make it a standard way of doing 
it. It's patently silly to say "here's the authorized way of doing this, 
but it might not be authorized...".

>Now that I have removed the text regarding MIME-style parameters that used
>to be in there, I considered using "copied" instead of "taken". But since
>it is the header content rather than the whole header that gets copied, I
>concluded on balance that defining the special term "taken" would shorten
>the wording, on balance. 

There is no reason to redefine a standard english word when there is 
already a standard english word that conveys the correct meaning. This 
"taken" from nonsense is how SBC got such ridiculous penalties for the 
people who had copies of some of its documents. We ought not to further
that line of propoganda.

>>And once again, when we specify ONE ACTION for an agent, that is THE 
>>DEFAULT. We don't need to keep saying "by default", we already know
>>that the one action we specify IS THE DEFAULT.

>There are TWO actions for this agent. Either it uses the user's override
>(if any) or else it uses the "default".

Nonsense. We tell it what it must use. We do NOT tell the user what he 
uses. Since there is only ONE thing that the followup agent is allowed to 
do ("copy the content") that is the default. There is no other default.

>>1. The content of the Newsgroups-header SHOULD be copied from the content 
>>of the precursor's Followup-To-header, if present. Otherwise, it is copied
>>from the content of the precursor's Newsgroups-header.

>That is precisely the longer wording I have avoided by defining the term
>"taken".

It's two sentences of simple english. It could be one more complex
sentence.

>And "by default" is necessary because sometimes the outgoing
>Newsgroups-header is not the Followup-To (etc) one.

The "outgoing" header is what the user wants it to be. What WE tell the 
FOLLOWUP agent it SHOULD do is copy the content from one of two places. 
That's the DEFAULT and ONLY action for the followup agent. You are 
confusing the actions of the agent with the actions of the user.

>      If the References header of the precursor has grown to excessive
>      length, it MAY first be trimmed, but its first and last message
>      identifiers MUST NOT be removed.

No. It serves no purpose to trim the References header of the precursor. 
You want to trim the one in the followup. "If the resulting 
References-header is excessively long, it may be trimmed by removing 
message identifiers other than the first and the last."

>>A reading agent is free to show articles to the user in any manner the 
>>user so desires. This is not limited to character sets.

>It does not say it is limited to character sets. It says "such as
>availability of charsets". 

The point remains, the reading agent is under NO limitations on how it may 
display articles. The "such as" implies there are limits.  We have no 
business saying what it must or should do -- there are "reading 
agents" we discuss later which display NOTHING to the user, they are 
called "outgoing gateways".

>>Of course it may. It may also display icons or green flashing dots.

>Very pretty, especially at Christmas time. But not a reason for mentioning
>them :-) .

There is just as much reason for mentioning them as for mentioning 
anything else.

>>s/by email//

>No, see above.

Yes, see above.

>>Irrelevant and meaningless. Elide.

>The next sentence would be meaningless without it.

And the next paragraph I wrote already showed the next sentence is
meaningless and ought to be removed. "Keep this meaningless sentence
because otherwise the next meaningless sentence would still be
meaningless" doesn't convince me we ought to keep either one.

>How about:

>   .... The operation of the outgoing gateway
>   is subject to additional constraints due to the possibility of one or
>   more corresponding incoming gateways back from that medium to
>   Netnews, since this opens the possibility of loops.

We already cover the possibility of loops, so there is no need for this 
paragraph at all. Start the section titled "Duties of an Outgoing Gateway" 
with "In general, the following practices are recommended for all outgoing 
gateways," which is actually relevant to the section in which it appears.

>>I fail to see how a comment in the body of a message helps to prevent 
>>loops.

>There might be no other place to put it in the other medium. 

Oh, ok. Wait. I still don't see how a comment in the body of a message 
helps prevent loops.

>But the
>gateway that put it there should be able to find and recognize it if it
>ever comes back to the same gateway,

It is unlikely that an outgoing gateway is going to see an incoming 
message. However, if you meant that the same outgoing gateway is going to 
see the same message outgoing more than once, then it already has the 
mandatory message id (not in the body) to work with. A comment in the body 
of a messsage is not likely to prevent loops. Saying it does is silly.

>>>   The incoming gateway has the serious responsibility of ensuring that
>>>   all of the requirements of this standard are met by the articles that
>>>   it forms.

>>ALL? Or do you mean that it must meet the standards expected by the 
>>injecting agent that it will be using?

>It is responsible for ensuring that proper checks are done. If it can
>trust a local injecting agent to do the checks properly, then that is
>fine.

That's not what our draft says. It says that IT has the responsibility, 
not the injecting agent. And it talks about articles IT forms, not the 
final form as it would appear on the wire.

>>I guess you did mean "all". I disagree. An incoming gateway does NOT need 
>>to be an injecting and relaying agent too. It can function quite well 
>>below the injection agent. 

>How it organizes itself internally is none of our business, just so long
>as its duties and responsibilities are taken care of.

Then why are we saying that IT has the responsibility for ALL of the 
standard, and not just the parts for the function it is performing? 

>>What is the reason for this RFC2119 mandate? The injecting agent will stop 
>>a duplicate.

> Not necessarily.

7.2.2.  Procedure to be followed by Injecting Agents

   4. It MUST reject any article that does not have the proper mandatory
      headers for a proto-article (or, when reinjecting, all the
      mandatory headers other than Injection-Date), or which contains
      any header that does not have legal contents.  

It takes a bit of thought there to realize that a duplicated message id is 
an illegal content for the message id header. And most injecting agents 
do, indeed, reject duplicates.

>Even if it uses the services of an injecting agent, it
>cannot expect that agent to be aware of all the sneaky ways in which an
>already gatewayed article may come back to it from the other medium.

It does not need to care what "sneaky" ways an article gets to it. 

>>Cool. My gateway's method of determining a "valid equivalent" is by seeing
>>a Control header. But I must remove it, even though I am permitted to 
>>put it in. Which is it?

>I would suggest checking for and noting the presence of the cancel message
>_before_ removing it.

There is no cancel message before removing it. I am told I must remove it 
before I can inject it.

>Bruce Lilly <blilly@xxxxxxxxx> writes:
>>or automatic process).  The "poster" corresponds to
>>"sender", not to "author".

>Yes, that is more or less what we decided. And as a consequence you will
>now find the words "author or sender" in 7.6.

Yes, but the problem appears in the definitions, where it clearly says 
that the POSTER is the one COMPOSING the message. That's not true. 

Date: Tue, 7 Dec 2004 18:35:40 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

John Stanley <[email-masked]> wrote:
> Seth Breidbart <sethb@xxxxxxxxx>:

>> To whom are you attributing that error?
> To everyone who says that "once per thousand years" is good enough.

Then you're wrong.   I think one collision, total, on average every
thousand years is good enough.  I also understand probability theory
quite well.

>> Yes.  A hardware flaw could cause collisions in a "never" scheme;
> A breakdown in a piece of hardware is different than a programmed, known
> flaw.

They both cause the same violation of the same MUST.

>>Computer breakdown during the 3 seconds between finishing editing the
>>proto-article and hitting "send" is less common than breakdown in
>>general.
> Really? What is special about "finishing editing the proto-article" that
> protects the computer from failure until hitting "send"?

Nothing.  It's just not that much more likely to break in that
3-second window than any _other_ 3-second window, and there are a lot
of 3-second windows in a computer's lifetime.

>> If it's that important to you, choose your ISP based on it 
> And how many ISPs advertise their message id creation algorithm?

Advertise?  None, because there aren't enough potential customers who
care.  So you'll have to make the effort of asking them, instead of
expecting the information to be thrust upon you.

> Your pretense that one can choose not to use a broken implementation
> of a deeply-buried algorithm relies on knowledge of that broken
> implementation being used. That's not going to happen.

If you generate your own Message-ID, then you don't have to know or
care about the algorithm that would otherwise have been used.

>>> and when did I get to choose 
>>> the implementation used on your server?
>>You don't. 
> Since YOUR message ids show up in MY news stream, then I have not had the
> opportunity to choose not to be subject to the broken implementation.

That's right.  Usenet contains broken messages.  The only way we can
prevent that is by saying that there's no such thing, any string is an
acceptable message.  I don't think that's useful.

>> You don't have to consider that a problem.
> There are lots of things I don't HAVE to consider a problem that are still
> problems.

You are welcome to worry about them as much as you want to.  I choose
not to bother.

Seth

Date: Tue, 7 Dec 2004 15:01:49 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

Seth Breidbart <sethb@xxxxxxxxx>:

> To whom are you attributing that error?

To everyone who says that "once per thousand years" is good enough.

> Yes.  A hardware flaw could cause collisions in a "never" scheme;

A breakdown in a piece of hardware is different than a programmed, known
flaw.

>Computer breakdown during the 3 seconds between finishing editing the
>proto-article and hitting "send" is less common than breakdown in
>general.

Really? What is special about "finishing editing the proto-article" that
protects the computer from failure until hitting "send"? I'd like to know;
I'll get all my systems thinking that I've just finished editing a 
proto-article so they will be less likely to break down.

> If it's that important to you, choose your ISP based on it 

And how many ISPs advertise their message id creation algorithm? Your 
pretense that one can choose not to use a broken implementation of a 
deeply-buried algorithm relies on knowledge of that broken implementation
being used. That's not going to happen.

>> and when did I get to choose 
>> the implementation used on your server?

>You don't. 

Since YOUR message ids show up in MY news stream, then I have not had the
opportunity to choose not to be subject to the broken implementation.

> You don't have to consider that a problem.

There are lots of things I don't HAVE to consider a problem that are still
problems.

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>> But the id-right is NOT case-sensitive, 

>>I see nothing in RFC2822 that says it is not case-sensitive.

>Then I think we are in agreement. But I am aware of people who do not
>share that view.

Oh, good, we've started this game again. No Charles, if you say something
is NOT case sensitive, and I say that I see nothing that says it is not, 
that means we DO NOT AGREE. The fact that RFC2822 does NOT say it is 
case-insensitive, combined with the knowledge that ASCII 'a' is not the
same as ASCII 'A', means that I think it IS case sensitive.

>Indeed, injecting agents may well have to check for these funny cases (but
>then that is what I said).

No, that is not what you said. You said "I expect many injecting agents do 
little more than check there is a single '@' somewhere in it" and I'm 
telling you my experience is that they do much more than that.

Date: Tue, 7 Dec 2004 16:52:03 -0500 (EST)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: msg-id
Sender: [email-masked]

On Tue, 7 Dec 2004, Charles Lindsey wrote:
> >> But the id-right is NOT case-sensitive, 
> >I see nothing in RFC2822 that says it is not case-sensitive.
> 
> Then I think we are in agreement. But I am aware of people who do not
> share that view.

Hmm, interesting.  This changed from 822 to 2822.  RFC 2821, SMTP, is
explicit that the domain of an addr-spec is case-insensitive, but 2822 is
silent about it... and the body of a message ID is no longer an addr-spec
anyway. 

> Indeed, injecting agents may well have to check for these funny cases (but
> then that is what I said). But relaying and serving agents have no time to
> perform such checks, and so will proceed on the assumption that any broken
> cases will have been weeded out beforehand.

I think it would be more accurate to say "might not have time" and "may
proceed on the assumption".  Some will undoubtedly wish to be fussier, but
this should be optional rather than mandatory. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
Subject: Re: msg-id
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 07 Dec 2004 13:03:04 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> John Stanley <[email-masked]> writes:

>> I don't think so. When I was running a mail to news gateway, one of the
>> first steps I was forced to take was to convert the mail message id
>> into something legal for news, since a large number of them seemed to
>> contain spaces. The injecting agent didn't accept that at all. It did
>> not just compare everything from the first '<' to the next '>' or SP or
>> just look for an '@' somewhere.

> Indeed, injecting agents may well have to check for these funny cases
> (but then that is what I said). But relaying and serving agents have no
> time to perform such checks, and so will proceed on the assumption that
> any broken cases will have been weeded out beforehand.

Charles is indeed mistaken about what many relaying and serving agents do.
John is correct, and not just for injecting agents.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: LIST: [email-masked];
Xref: clerew local.usefor:20399
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Tue, 7 Dec 2004 17:13:02 GMT
Lines: 41
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>> But the id-right is NOT case-sensitive, 

>I see nothing in RFC2822 that says it is not case-sensitive.

Then I think we are in agreement. But I am aware of people who do not
share that view.

>>I don't think it matters particularly. What happens in practice is that
>>news agents just take everything from the first '<' to the next '>' or SP
>>and do a simple comparison of the resulting string. I doubt any agent,
>>except an injecting agent, every looks inside to see what is there, and I
>>expect many injecting agents do little more than check there is a single
>>'@' somewhere in it.

>I don't think so. When I was running a mail to news gateway, one of the 
>first steps I was forced to take was to convert the mail message id into 
>something legal for news, since a large number of them seemed to contain
>spaces. The injecting agent didn't accept that at all. It did not just 
>compare everything from the first '<' to the next '>' or SP or just look 
>for an '@' somewhere.

Indeed, injecting agents may well have to check for these funny cases (but
then that is what I said). But relaying and serving agents have no time to
perform such checks, and so will proceed on the assumption that any broken
cases will have been weeded out beforehand.

To: LIST: [email-masked];
Xref: clerew local.usefor:20396
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: IMC ietf-usefor archive
Date: Tue, 7 Dec 2004 14:45:21 GMT
Lines: 43
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon December 6 2004 12:47, John Stanley wrote:
>> 
>> "There is a complete archive of the mailing list." 

>The corresponding link points to a web interface which
>presents only fractions of messages.  And for the record,
>the Landfield.com web interface also presented only
>fractions of messages (in particular, there is no way in
>either case to examine message header contents).

Yes there is (was). The articles were also available in mbox format on a
month-by-month basis (and that same archive is now available on the new
site - indeed for stuff over 2 years old it is the only form available
until I get enough round tuits to copy over the remainder of the thml
archive).

>You will find on the same page the text
>"You can also access the entire archive.", and the
>corresponding link will provide you with the real archive,
>unfettered by HTML constraints and without the
>obfuscation of content that typically accompanies
>web interfaces.

But only as a monolithic file containing every message ever posted since
the list started. Fortunately, for this list that is not too big (yet).
But totally useless for a list like ietf-822.

Hence my complaint that the overall system is nowhere as useful as it was
on Landfield (when Landfield was working properly, that is).

To: LIST: [email-masked];
Xref: clerew local.usefor:20397
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Tue, 7 Dec 2004 17:03:47 GMT
Lines: 403
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>See http://www.ietf-usefor.imc.org/drafts/draft-ietf-usefor-usepro-02.txt 
>>and http://www.ietf-usefor.imc.org/drafts/draft-ietf-usefor-usepro-02.unpaged

>"www.ietf-usefor.imc.org could not be found. Please check the name and try 
>again."

Sorry:
See http://www.imc.org/ietf-usefor/drafts/draft-ietf-usefor-usepro-02.txt 
and http://www.imc.org/ietf-usefor/drafts/draft-ietf-usefor-usepro-02.unpaged

>>    A "poster" is the person or software that composes a possibly
>>    compliant article for submission to a posting agent. The poster is
>>    synonymous with [RFC 2822]'s author.

>Then we ought to call it "author" and use the term "poster" ...

See my separate response to Bruce Lilly.

>>   1. It MUST remove any Injection-Info- or Complaints-To-header already
>>      present (though it might be useful to copy them to suitable X-
>>      headers). It SHOULD likewise remove any NNTP-Posting-Host, X-Trace
>>      or other undocumented tracing header.

>If it is "undocumented", how does one know it is a tracing header?

With difficulty. But people have always managed to discover that
NNTP-Posting-Host was a tracing header.

>>   2. It SHOULD verify that the article is from a trusted source.
>>      However, it MAY allow articles in which headers contain unverified
>>      email addresses, that is, addresses which are not known to be
>>      valid for the trusted source, and notably so if they end in
>>      ".invalid".

>Since the injecting agent cannot know, in the general case, that an 
>address is not valid for the poster, then it SHOULD NOT pretend that it
>can, and we SHOULD NOT tell authors of injecting agents they should
>pretend they can. 

We don't. It says they "MAY" allow certain articles. If they can't tell
the address is bad, then they pretty well have to let them through
(otherwise they would never inject any articles at all). But if they can
detect the bad ones (they might even check with the DNS that the domain
does not exist) then they have the right to refuse them (according to
local site policy).

>I missed the section in RFC2119 which explains how "notably so" modifies
>any of the RFC2119 terms. We should either explain what "notably so" 
>means, or say "Injection agents MUST accept addresses that end in 
>.invalid." (For those who say otherwise, then please explain exactly what
>purpose .invalid service, then?)

"Notably so" is only modifying a "MAY". Essentially, it is encouraging you
to take advantage of that "MAY" in this circumstance. But there is no
MUST about it, since a site may very well say "we require all our users to
use valid and working email addresses".

>>7.2.2.  Procedure to be followed by Injecting Agents

>So, either an injection agent doesn't deal just with "proto-articles", or
>there needs to be a "reinjection agent".

The first sentence of 7.2 indicates the normal situation in which an
injecting agent is given a proto-article. A couple of paragraphs further
on it states that, "in exceptional circumstances" an injection agent may
also be given a full article (not a proto-article). So both circumstances
are covered, and I see little point in complicating the wording by being
more explicit.

Do you really want me to change the first sentence to read
   "An Injecting Agent is responsible for taking a (proto-)article from a
   posting (or other) agent ..."   ?

>>   1. It MUST forward it to the moderator of the first (leftmost)
>>      moderated group listed in the Newsgroups-header via email (see 7.8
>>      for how that moderator may forward it to further moderators).
>>      There are two possibilities for doing this:

>Here's this artificial limitation again. Why must it be via EMAIL?

Because, for the worldwide Usenet, that is the way the protocol has been
defined, and there is no other machinery that is guaranteed to work.

> Is
>there some reason those two words have to be there? I understand that 
>whatever IS done will have to "interoperate", but if that "interoperation"
>means that MY server hosting MY private newsgroups hands the moderators
>articles to approve via a web interface, how is that any skin off of YOUR 
>nose?

You are talking about a "cooperating subnet" which has chosen (and
presumably published to its membership) some alternative method. The term
"cooperating subnet" is still defined in USEPRO, though it seems to have
disappeared from USEFOR (and I will have to watch that it remains
somewhere when we finally sort out where all the various definitions are
going to go).

>>   A serving agent MUST maintain a list of the newsgroups it stores in
>>   its news database showing the moderation status of each one (see
>>   6.2.1), and SHOULD include in that list all groups likely to be
>>   crossposted to from those groups (e.g. all other groups in the same
>>   hierarchy(ies)).

>Knowing as I do the interesting interpretation of SHOULD that most folks 
>here have, I have to ask whether this sentence is saying nothing less than 
>that a serving agent must list all groups that it has ever heard of. I 
>mean, a "strong recommendation that can be avoided only after careful 
>study and consideration of all possible negative outcomes" combined with 
>the guessing game of "group likely to be crossposted to" results in "all 
>groups ever heard mentioned at least once".

Not, it doesn't result in that. A group that is seen to have been
crossposted to "once" is not thereby a group that is "likely to be
crossposted to" again. The use of the word "likely" indicates that the
admin of the serving agent has to exercise some judgement here, and if he
concludes "after careful study and consideration" that he can ignore the
possibility of crossposts between the chinese newsgroups (which he
carries) and the groups in the national hierarchy for Mozambique (of which
he has probably never heard), then I think he has fulfilled all his
obligations under that "SHOULD".

But I am sure that there nevertheless exists some chinaman somewhere in
the world who has a cousin in Mozambique.

>>   2. It MUST examine the Injection-Date-header (or, if that is absent,
>>      the Date-header) and reject the article as stale (a-5.7) if that
>>      predates the earliest articles of which it normally keeps record,
>>      or if it is more than 24 hours into the future (the margin MAY be
>>      less than that 24 hours).

>Isn't expiration policy based on arrival date, not injection date? In any 
>case, is there a reason for RFC2119 language regarding a site's expiration
>policy? E.g., if a site wants to serve articles for "three days" even if 
>they arrive two days after being posted, is that our business?

The wording is designed to cover whatever expiration policy the particular
relaying agent observes (which is indeed customarily x days after
arrival). So if the Injection-Date is earlier than the expiry date
(however computed) that it implements (probably of a per group basis), then
it MUST drop it.

>>   Serving agents MUST NOT create new newsgroups simply because an
>>   unrecognized newsgroup-name occurs in a Newsgroups-header (see a-
>>   7.2.1 for the correct method of newsgroup creation).

>However, we've just told it that it SHOULD include that group in the list 
>of groups that it keeps. After all, if it appears in a Newsgroups header, 
>it is quite likely that it will be crossposted to.

No, we have not told it that. It has decided (after careful study and
consideration) not to list groups in the Mozambiquean hierarchy. If an
article cross-posted to that hierarchy now appears, it MUST NOT create the
group just for that reason (it MAY choose to list it henceforth, but the
list is not restricted to groups that it actually keeps).

>>   Contrary to [RFC 2822], which states that the mailbox(es) in the From
>>   header is that of the poster(s), a poster who does not, for whatever
>>   reason, wish to use his own mailbox MAY use any mailbox ending in the
>>   top level domain ".invalid" [RFC 2606].

>This paragraph is meaningless without an accompanying MUST for the 
>injection agent. 

Yes, you may have a point there. Would you suggest that I weaken the
wording here, or strengthen the wording under injecting agent ("SHOULD"
would probably suffice)? And other opinions would be welcome. AIUI, we
don't want to prevent people doing that kind of munge, but we don't want to
force injecting agents to accept it either.

>>                                                        Wherever in
>>   the following it is stated that, "by default", a header is to be
>>   taken from some header in the precursor, it means that its initial
>>   content is to be copied from the content of that precursor header.

>Then why don't we just say "copied" in the relevant sections and avoid
>this redefinition and nonsense of "taken from". *NOTHING IS TAKEN FROM
>ANYTHING. It is copied.

Now that I have removed the text regarding MIME-style parameters that used
to be in there, I considered using "copied" instead of "taken". But since
it is the header content rather than the whole header that gets copied, I
concluded on balance that defining the special term "taken" would shorten
the wording, on balance.

>And once again, when we specify ONE ACTION for an agent, that is THE 
>DEFAULT. We don't need to keep saying "by default", we already know
>that the one action we specify IS THE DEFAULT.

There are TWO actions for this agent. Either it uses the user's override
(if any) or else it uses the "default".

>>   1. The Newsgroups-header (a-5.5) SHOULD by default be taken from the
>>      precursor's Followup-To-header if present, and otherwise from the
>>      precursor's Newsgroups-header. 

>No. 

>1. The content of the Newsgroups-header SHOULD be copied from the content 
>of the precursor's Followup-To-header, if present. Otherwise, it is copied
>from the content of the precursor's Newsgroups-header.

That is precisely the longer wording I have avoided by defining the term
"taken". And "by default" is necessary because sometimes the outgoing
Newsgroups-header is not the Followup-To (etc) one.

>>      Followup agents MAY trim References-headers which have grown to
>>      excessive length, but the first and last message identifiers from
>>      the precursor MUST NOT be removed.

>Is it then acceptable to remove the last message id in the 
>References-content of the followup? The last message id in the followup's
>References header is not the last message id in the precursor's References
>header.

OK. How about:

      If the References header of the precursor has grown to excessive
      length, it MAY first be trimmed, but its first and last message
      identifiers MUST NOT be removed.

>>   A reading agent downloads articles from a serving agent, as directed
>>   by the reader, and displays them (or processes them in some other
>>   manner), subject to any limitations of the reading agent, such as
>>   availability of charsets, and having first decoded any Content-
>>   Transfer-Encodings, encoded-words, etc.  It SHOULD also have the
>>   capability to show the raw article exactly as received.

>A reading agent is free to show articles to the user in any manner the 
>user so desires. This is not limited to character sets.

It does not say it is limited to character sets. It says "such as
availability of charsets". 

>"A reading agent accepts articles from a serving agent and displays them 
>to the reader."

>>   It MAY present lists of articles available for display, and MAY
>>   structure those lists so as to show the relationships between the
>>   articles, as determined by the References-, Subject-, Date- and
>>   other-headers (see [USEAGE] for some usual methods of doing this).

>Of course it may. It may also display icons or green flashing dots.

Very pretty, especially at Christmas time. But not a reason for mentioning
them :-) .

>>7.8.  Duties of a Moderator

>>   A Moderator receives news articles by email, decides whether to
>>   accept them 

>s/by email//

No, see above.

>s/accept/approve/

>He has already accepted the message -- it is in his mailbox or other input 
>queue.

OK, and several other occurrences of "accept" in that section.

>>7.9.1.  Duties of an Outgoing Gateway

>>   From the perspective of Netnews, an outgoing gateway is just a
>>   special type of reading agent.

>Irrelevant and meaningless. Elide.

The next sentence would be meaningless without it. Generally, the whole of
the section on gatewaying is in a more discursive style than the rest of
the document because it is dealing with a variety of situations which much
current software handles badly, and is setting out basic principles to be
observed rather than precise protocols.

>>                               The operation of the outgoing gateway
>>   is only subject to additional constraints in the presence of one or
>>   more corresponding incoming gateways back from that medium to
>>   Netnews, since this opens the possibility of loops.

>The operation of a gateway ought always to be based on the possibility
>of the back gateway, not on the actual existance of same. That implies
>contstraints based not on the presence, but on the ability. Elide 
>sentence.

How about:

   .... The operation of the outgoing gateway
   is subject to additional constraints due to the possibility of one or
   more corresponding incoming gateways back from that medium to
   Netnews, since this opens the possibility of loops.

>>   1. The message identifier of the news article should be preserved if
>>      at all possible, preferably as or within the corresponding unique
>>      identifier of the other medium, but if not at least as a comment
>>      in the message. This helps greatly with preventing loops.

>I fail to see how a comment in the body of a message helps to prevent 
>loops.

There might be no other place to put it in the other medium. But the
gateway that put it there should be able to find and recognize it if it
ever comes back to the same gateway, and that is how a loop would be
prevented.

>>7.9.2.  Duties of an Incoming Gateway

>>   The incoming gateway has the serious responsibility of ensuring that
>>   all of the requirements of this standard are met by the articles that
>>   it forms.

>ALL? Or do you mean that it must meet the standards expected by the 
>injecting agent that it will be using?

It is responsible for ensuring that proper checks are done. If it can
trust a local injecting agent to do the checks properly, then that is
fine.

>>              In addition to its special duties as a gateway, it bears
>>   all of the duties and responsibilities of an injecting agent as well,
>>   and additionally has the same responsibility of a relaying agent to
>>   reject articles that it has already gatewayed.

>I guess you did mean "all". I disagree. An incoming gateway does NOT need 
>to be an injecting and relaying agent too. It can function quite well 
>below the injection agent. 

How it organizes itself internally is none of our business, just so long
as its duties and responsibilities are taken care of.

>>   An incoming gateway MUST NOT gate the same message twice. 

>What is the reason for this RFC2119 mandate? The injecting agent will stop 
>a duplicate.

Not necessarily. Even if it uses the services of an injecting agent, it
cannot expect that agent to be aware of all the sneaky ways in which an
already gatewayed article may come back to it from the other medium.

>>   Incoming gateways MUST NOT pass control messages (articles containing
>>   a Control- or Supersedes-header) without removing or renaming that
>>   header. 

>Thus preventing a user on the other side of an incoming gateway from ever 
>cancelling or superceding his own articles. 

Yes, this may be a bit severe. But it is what Russ originally wrote and
nobody has queried it before now. I am open to further views (and I have
certainly had occasion to put control messages through mail2news before
now).

>>                          If a gateway receives a message that it can
>>   determine is a valid equivalent of a cancel message in the medium it
>>   is gatewaying, it SHOULD discard that message without gatewaying it,
>>   generate a corresponding cancel message of its own, and inject that
>>   cancel message.

>Cool. My gateway's method of determining a "valid equivalent" is by seeing
>a Control header. But I must remove it, even though I am permitted to 
>put it in. Which is it?

I would suggest checking for and noting the presence of the cancel message
_before_ removing it.

>>        different portions).  In these cases, where there is no one
>>        "official" gateway, some other method of generating message
>>        identifiers has to be used to avoid collisions. It would
>>        obviously be preferable for there to be only one gateway which
>>        crossposts, but this may not be possible to coordinate.

>Given the previous discussion about possiblilities and not actual 
>presence, then what we are actually saying is that gateways MUST create
>message ids for news messages and MUST NOT use existing ids found in any
>incoming message.

No, I think what it (Russ) is saying is that people who maintain gateways
need to be aware of the way things actually happen within the particular
group (or set of related groups) concerned. Because there just is no "one
size fits all" solution to the gatewaying problem. Every case is
different.

To: LIST: [email-masked];
Xref: clerew local.usefor:20398
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Tue, 7 Dec 2004 17:06:46 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon December 6 2004 14:01, John Stanley wrote:

>> >    A "poster" is the person or software that composes a possibly
>> >    compliant article for submission to a posting agent. The poster is
>> >    synonymous with [RFC 2822]'s author.
>> 
>> Then we ought to call it "author" and use the term "poster" for the person
>> who posts the messages. I thought we had a discussion about this already
>> and determined that using "poster" in that sense made no difference to 
>> anything in the draft AND removed the confusion of having a deliberately
>> different term than the overriding RFC.

>We indeed had a discussion, but it was pointed out that the
>poster and the author may very well be different (e.g. a FAQ
>written by one or more authors and posted by an individual
>or automatic process).  The "poster" corresponds to
>"sender", not to "author".

Yes, that is more or less what we decided. And as a consequence you will
now find the words "author or sender" in 7.6.

Date: Tue, 7 Dec 2004 11:51:29 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

John Stanley <[email-masked]> wrote:
> Seth Breidbart <sethb@xxxxxxxxx>:

>> True; but so what? 
> So it is an error to talk about "once per thousand years" as if it
> meant it could not happen tomorrow and the next day.

To whom are you attributing that error?

>> The _spec_ says "MUST be unique". 
> That's right. So that's why the error I mentioned above makes a "once
> per thousand years" system inappropriate. Do you disagree?

Yes.  A hardware flaw could cause collisions in a "never" scheme; is
that scheme therefore inappropriate if run on hardware?

>>If a particular
>>implementation only guarantees uniqueness with a probability greater
>>than the one that my computer won't melt in between my typing the
>>message and hitting "send", 
>
> Computers breaking down is much more probable than "once per thousand 
> years". If you work in the business you will see it happen much more often
> than that.

Computer breakdown during the 3 seconds between finishing editing the
proto-article and hitting "send" is less common than breakdown in
general.

>>I consider that level of lossage acceptable.  If you don't, feel
>>free not to use that implementation.
>
> When did I get to choose what implementation of message id is used in
> the news server installed at the ISP I use,

If it's that important to you, choose your ISP based on it or run your
own news server as a peer.  (Can't your user agent generate the
Message-Id anyway?)

> and when did I get to choose 
> the implementation used on your server?

You don't.  If I generate duplicates, then my system won't play nice
with others, and some of my articles will disappear.  You don't have
to consider that a problem.

Seth

Date: Tue, 7 Dec 2004 08:50:24 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

Bruce Lilly <blilly@xxxxxxxxx>:

>An author or group of authors may compose some content
>without specifically composing it *for submission to a posting
>agent*. 

That's nice. Also irrelevant. Their "intent" is not the important part of 
that sentence, their ACTION is. And the action is "composes". 

>Sorry, I didn't see where you stated that "poster" corresponds
>to RFC 2822 "sender".  I looked again, and still don't see it.
>Did you use invisible characters?

I said that the poster is the one who posts the article. If you read the
draft you'll notice that the draft says:

   A "sender" is the person or software (usually, but not always, the
   same as the poster) responsible for the operation of the posting
   agent or, which amounts to the same thing, for passing the article to
   the injecting agent.

That's posting.

If A=B and B=C, then A=C. 

>No, I'm not arguing.  I am confirming as fact what you said
>you "thought" happened, and stated that "poster" (as used
>in text, not as a keyword) corresponds to RFC 2822 "sender".

No, "poster" as used in the draft does NOT correspond to RFC2822 "sender".
It ought to, but as it currently is written, it does not. That's why I
brought it up.

Date: Tue, 7 Dec 2004 08:41:21 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

Seth Breidbart <sethb@xxxxxxxxx>:

> True; but so what? 

So it is an error to talk about "once per thousand years" as if it
meant it could not happen tomorrow and the next day.

> The _spec_ says "MUST be unique". 

That's right. So that's why the error I mentioned above makes a "once
per thousand years" system inappropriate. Do you disagree?

>If a particular
>implementation only guarantees uniqueness with a probability greater
>than the one that my computer won't melt in between my typing the
>message and hitting "send", 

Computers breaking down is much more probable than "once per thousand 
years". If you work in the business you will see it happen much more often
than that.

>I consider that
>level of lossage acceptable.  If you don't, feel free not to use that
>implementation.

When did I get to choose what implementation of message id is used in
the news server installed at the ISP I use, and when did I get to choose 
the implementation used on your server?

<thorfinn@xxxxxxxxxxxxxx>:

>It's a lot less of a worry than the people who don't use any kind of
>scheme at all for the LHS.

A novel excuse for doing it wrong: it's less wrong than people who do it
wronger.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: John Stanley <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Tue, 7 Dec 2004 08:06:19 -0500
Cc: [email-masked]
Sender: [email-masked]

On Mon December 6 2004 19:56, John Stanley wrote:

> Ummm, yes, that is why I said that we ought to call the person or software
> that composes a possibly compliant article for submission to a posting
> agent "author"

An author or group of authors may compose some content
without specifically composing it *for submission to a posting
agent*.  It might be composed as a document intended for
multiple means of access (ftp, http, etc.), or it may be
composed as a message for multiple means of delivery. It
might be composed specifically for delivery as email, which
is subsequently (possibly without knowledge of the author(s))
gatewayed into news [the gateway in that case, though it
may modify the message specifically "for submission", is the
poster, but is certainly not "the author"].
 
> >The "poster" corresponds to "sender", not to "author".
> 
> Yes, that's what I said.

Sorry, I didn't see where you stated that "poster" corresponds
to RFC 2822 "sender".  I looked again, and still don't see it.
Did you use invisible characters?

> What's the problem here? You seem to be arguing with me by repeating what 
> I said.

No, I'm not arguing.  I am confirming as fact what you said
you "thought" happened, and stated that "poster" (as used
in text, not as a keyword) corresponds to RFC 2822 "sender".

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Sebastian Brocks" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Tue, 7 Dec 2004 07:39:22 -0500
Cc: [email-masked]
Sender: [email-masked]

On Tue December 7 2004 02:38, Sebastian Brocks wrote:

> Except for the "Followup-To" Header, certainly in that case, the "author"
> excepts any email replies, not the "sender".

No, "certainly" if a Reply-To field is present, a response is
directed to the addresses specified there, which might
include neither authors nor the poster, the poster but
no authors, some authors but not the poster, poster
plus authors and possibly others, ...

Date: Tue, 7 Dec 2004 08:38:30 +0100 (MET)
From: "Sebastian Brocks" <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

> We indeed had a discussion, but it was pointed out that the
> poster and the author may very well be different (e.g. a FAQ
> written by one or more authors and posted by an individual
> or automatic process).  The "poster" corresponds to
> "sender", not to "author".

Except for the "Followup-To" Header, certainly in that case, the "author"
excepts any email replies, not the "sender".

Date: Tue, 7 Dec 2004 12:39:56 +1100
From: Thorfinn <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

On Mon 06 Dec 2004 at 05:15:14PM -0800, in <[email-masked]>,
John Stanley <[email-masked]> wrote:
> "Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:
> >It has been discussed peridically in this WG, and the opinion expressed
> >has always been that one failure per thousand years (whatever - I haven't
> >calculated the exact probability for the Agent scheme) could be regarded
> >as satisfying that MUST for all practical purposes. 
> It is a mistake to view the odds of "once per thousand years" as meaning
> "won't happen for 1000 years". A "once per thousand years" system is just 
> as likely to duplicate a message id tomorrow as it is 999 years from now. 
> And then once it has duplicated the id tomorrow, it is just as likely to 
> do so the day after that.
> Casinos get rich off of people who play roulette using that kind of system.

It's a lot less of a worry than the people who don't use any kind of
scheme at all for the LHS.  A *lot* of newsreaders stick in the "FQDN"
of the "host" they're on on the RHS... and all too often that's
"foo.localnet".

It's all well and good when the hostname *is* a real FQDN, but an awful
lot of people are behind IP masqueraded firewalls these days, even
"home" users, who often have broadband "routers" that do IPmasq.

Ook,

 Thorf

-- 
<a href="http://tertius.net.au/~thorfinn/">[email-masked]</a>
"New zero-employee strategy.  It's the latest way of saving on costs."
    -- sharkey*zoic.org

Date: Mon, 6 Dec 2004 20:33:15 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

John Stanley <[email-masked]> wrote:
> "Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>It has been discussed peridically in this WG, and the opinion expressed
>>has always been that one failure per thousand years (whatever - I haven't
>>calculated the exact probability for the Agent scheme) could be regarded
>>as satisfying that MUST for all practical purposes. 
>
> It is a mistake to view the odds of "once per thousand years" as meaning
> "won't happen for 1000 years". A "once per thousand years" system is just 
> as likely to duplicate a message id tomorrow as it is 999 years from now. 
> And then once it has duplicated the id tomorrow, it is just as likely to 
> do so the day after that.

True; but so what?  The _spec_ says "MUST be unique".  If a particular
implementation only guarantees uniqueness with a probability greater
than the one that my computer won't melt in between my typing the
message and hitting "send", then that implementation is going to cause
me to lose fewer articles than Murphy already does.  I consider that
level of lossage acceptable.  If you don't, feel free not to use that
implementation.

Seth

Date: Mon, 6 Dec 2004 17:15:14 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: msg-id
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>It has been discussed peridically in this WG, and the opinion expressed
>has always been that one failure per thousand years (whatever - I haven't
>calculated the exact probability for the Agent scheme) could be regarded
>as satisfying that MUST for all practical purposes. 

It is a mistake to view the odds of "once per thousand years" as meaning
"won't happen for 1000 years". A "once per thousand years" system is just 
as likely to duplicate a message id tomorrow as it is 999 years from now. 
And then once it has duplicated the id tomorrow, it is just as likely to 
do so the day after that.

Casinos get rich off of people who play roulette using that kind of system.

> But the id-right is NOT case-sensitive, 

I see nothing in RFC2822 that says it is not case-sensitive. Even if you 
assume that the id-right is obs-domain, there is nothing in RFC2822 that 
says that obs-domain is not case-sensitive. I don't believe that 
case-insensitivity is mentioned until one gets to the DNS standards. In
a message id, it is a string, and certainly one would understand that 'A'
is not the same as 'a', when working at that level.

>I don't think it matters particularly. What happens in practice is that
>news agents just take everything from the first '<' to the next '>' or SP
>and do a simple comparison of the resulting string. I doubt any agent,
>except an injecting agent, every looks inside to see what is there, and I
>expect many injecting agents do little more than check there is a single
>'@' somewhere in it.

I don't think so. When I was running a mail to news gateway, one of the 
first steps I was forced to take was to convert the mail message id into 
something legal for news, since a large number of them seemed to contain
spaces. The injecting agent didn't accept that at all. It did not just 
compare everything from the first '<' to the next '>' or SP or just look 
for an '@' somewhere.

Date: Mon, 6 Dec 2004 16:56:50 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-02
Sender: [email-masked]

Bruce Lilly <blilly@xxxxxxxxx>:

>We indeed had a discussion, but it was pointed out that the
>poster and the author may very well be different (e.g. a FAQ
>written by one or more authors and posted by an individual
>or automatic process).

Ummm, yes, that is why I said that we ought to call the person or software
that composes a possibly compliant article for submission to a posting
agent "author" and use the term "poster" for the person who posts the
messages. That is a pretty clear indication that I know the two "may very 
well be different". 

>The "poster" corresponds to "sender", not to "author".

Yes, that's what I said. The poster is NOT the person that composes the
article, that is the "author". The poster is the one who posts it. Just 
like one who reads english would expect.

If RFC2822 uses the term "author" for the person who writes something
(just like other english speakers do), why shouldn't we?

What's the problem here? You seem to be arguing with me by repeating what 
I said.

Date: Mon, 6 Dec 2004 16:49:02 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: IMC ietf-usefor archive
Sender: [email-masked]

Bruce Lilly <blilly@xxxxxxxxx>:

>The corresponding link points to a web interface which
>presents only fractions of messages. 

Gee, could that be why I'm reporting this as a problem?

>You will find on the same page the text
>"You can also access the entire archive.", and the
>corresponding link will provide you with the real archive,

"Real" archive, "complete" archive, "entire" archive. Whatever. Isn't
it nice that people have to download YEARS worth of messages just to read 
the complete, unmangled version of the last three hour's worth?

>The real archive, a.k.a. "entire archive" is not obfuscated.

Then there is no reason to mangle the "complete" archive. I mean, if
spammers are already being handed years of real email addresses, why
should MESSAGE BODIES be mangled to prevent them from getting invalid
ones?

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: John Stanley <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-02
Date: Mon, 6 Dec 2004 14:57:23 -0500
Cc: [email-masked]
Sender: [email-masked]

On Mon December 6 2004 14:01, John Stanley wrote:

> >    A "poster" is the person or software that composes a possibly
> >    compliant article for submission to a posting agent. The poster is
> >    synonymous with [RFC 2822]'s author.
> 
> Then we ought to call it "author" and use the term "poster" for the person
> who posts the messages. I thought we had a discussion about this already
> and determined that using "poster" in that sense made no difference to 
> anything in the draft AND removed the confusion of having a deliberately
> different term than the overriding RFC.

We indeed had a discussion, but it was pointed out that the
poster and the author may very well be different (e.g. a FAQ
written by one or more authors and posted by an individual
or automatic process).  The "poster" corresponds to
"sender", not to "author".

Date: Mon, 6 Dec 2004 11:01:17 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: draft-ietf-usefor-usepro-02
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>See http://www.ietf-usefor.imc.org/drafts/draft-ietf-usefor-usepro-02.txt 
>and http://www.ietf-usefor.imc.org/drafts/draft-ietf-usefor-usepro-02.unpaged

"www.ietf-usefor.imc.org could not be found. Please check the name and try 
again."

>    A "poster" is the person or software that composes a possibly
>    compliant article for submission to a posting agent. The poster is
>    synonymous with [RFC 2822]'s author.

Then we ought to call it "author" and use the term "poster" for the person
who posts the messages. I thought we had a discussion about this already
and determined that using "poster" in that sense made no difference to 
anything in the draft AND removed the confusion of having a deliberately
different term than the overriding RFC.

>   1. It MUST remove any Injection-Info- or Complaints-To-header already
>      present (though it might be useful to copy them to suitable X-
>      headers). It SHOULD likewise remove any NNTP-Posting-Host, X-Trace
>      or other undocumented tracing header.

If it is "undocumented", how does one know it is a tracing header?

>   2. It SHOULD verify that the article is from a trusted source.
>      However, it MAY allow articles in which headers contain unverified
>      email addresses, that is, addresses which are not known to be
>      valid for the trusted source, and notably so if they end in
>      ".invalid".

Since the injecting agent cannot know, in the general case, that an 
address is not valid for the poster, then it SHOULD NOT pretend that it
can, and we SHOULD NOT tell authors of injecting agents they should
pretend they can. 

I missed the section in RFC2119 which explains how "notably so" modifies
any of the RFC2119 terms. We should either explain what "notably so" 
means, or say "Injection agents MUST accept addresses that end in 
.invalid." (For those who say otherwise, then please explain exactly what
purpose .invalid service, then?)

>7.2.2.  Procedure to be followed by Injecting Agents

>   An injecting agent receives proto-articles from posting and followup
>   agents. 

Ok. Then we see in later sections talk about "reinjection". But we have
been told:

>        NOTE: An article that is offered for reinjection has, by
>        definition, already been injected once, and is not therefore to
>        be considered as a proto-article.  

So, either an injection agent doesn't deal just with "proto-articles", or
there needs to be a "reinjection agent".

>   1. It MUST forward it to the moderator of the first (leftmost)
>      moderated group listed in the Newsgroups-header via email (see 7.8
>      for how that moderator may forward it to further moderators).
>      There are two possibilities for doing this:

Here's this artificial limitation again. Why must it be via EMAIL? Is
there some reason those two words have to be there? I understand that 
whatever IS done will have to "interoperate", but if that "interoperation"
means that MY server hosting MY private newsgroups hands the moderators
articles to approve via a web interface, how is that any skin off of YOUR 
nose?

>   A serving agent MUST maintain a list of the newsgroups it stores in
>   its news database showing the moderation status of each one (see
>   6.2.1), and SHOULD include in that list all groups likely to be
>   crossposted to from those groups (e.g. all other groups in the same
>   hierarchy(ies)).

Knowing as I do the interesting interpretation of SHOULD that most folks 
here have, I have to ask whether this sentence is saying nothing less than 
that a serving agent must list all groups that it has ever heard of. I 
mean, a "strong recommendation that can be avoided only after careful 
study and consideration of all possible negative outcomes" combined with 
the guessing game of "group likely to be crossposted to" results in "all 
groups ever heard mentioned at least once".

>   2. It MUST examine the Injection-Date-header (or, if that is absent,
>      the Date-header) and reject the article as stale (a-5.7) if that
>      predates the earliest articles of which it normally keeps record,
>      or if it is more than 24 hours into the future (the margin MAY be
>      less than that 24 hours).

Isn't expiration policy based on arrival date, not injection date? In any 
case, is there a reason for RFC2119 language regarding a site's expiration
policy? E.g., if a site wants to serve articles for "three days" even if 
they arrive two days after being posted, is that our business?

>   Serving agents MUST NOT create new newsgroups simply because an
>   unrecognized newsgroup-name occurs in a Newsgroups-header (see a-
>   7.2.1 for the correct method of newsgroup creation).

However, we've just told it that it SHOULD include that group in the list 
of groups that it keeps. After all, if it appears in a Newsgroups header, 
it is quite likely that it will be crossposted to.

>   Contrary to [RFC 2822], which states that the mailbox(es) in the From
>   header is that of the poster(s), a poster who does not, for whatever
>   reason, wish to use his own mailbox MAY use any mailbox ending in the
>   top level domain ".invalid" [RFC 2606].

This paragraph is meaningless without an accompanying MUST for the 
injection agent. 

>                                                        Wherever in
>   the following it is stated that, "by default", a header is to be
>   taken from some header in the precursor, it means that its initial
>   content is to be copied from the content of that precursor header.

Then why don't we just say "copied" in the relevant sections and avoid
this redefinition and nonsense of "taken from". *NOTHING IS TAKEN FROM
ANYTHING. It is copied.

And once again, when we specify ONE ACTION for an agent, that is THE 
DEFAULT. We don't need to keep saying "by default", we already know
that the one action we specify IS THE DEFAULT.

>   However, posters MAY then override that default before posting if
>   they so wish.

Of course they can enter their own material. 

>   1. The Newsgroups-header (a-5.5) SHOULD by default be taken from the
>      precursor's Followup-To-header if present, and otherwise from the
>      precursor's Newsgroups-header. 

No. 

1. The content of the Newsgroups-header SHOULD be copied from the content 
of the precursor's Followup-To-header, if present. Otherwise, it is copied
from the content of the precursor's Newsgroups-header.

>   2. The Subject-header SHOULD by default be taken from that of the
>      precursor. The case sensitive string "Re: " MAY be prepended to
>      its Subject-Content unless it already begins with that string.

s/by default be taken from that of/be copied from/ 

Same for 3. 

4. /taken from/copied/

>      Followup agents MAY trim References-headers which have grown to
>      excessive length, but the first and last message identifiers from
>      the precursor MUST NOT be removed.

Is it then acceptable to remove the last message id in the 
References-content of the followup? The last message id in the followup's
References header is not the last message id in the precursor's References
header.

>   A reading agent downloads articles from a serving agent, as directed
>   by the reader, and displays them (or processes them in some other
>   manner), subject to any limitations of the reading agent, such as
>   availability of charsets, and having first decoded any Content-
>   Transfer-Encodings, encoded-words, etc.  It SHOULD also have the
>   capability to show the raw article exactly as received.

A reading agent is free to show articles to the user in any manner the 
user so desires. This is not limited to character sets. If we need a 
definition for "reading agent", then:

"A reading agent accepts articles from a serving agent and displays them 
to the reader."

>   It MAY present lists of articles available for display, and MAY
>   structure those lists so as to show the relationships between the
>   articles, as determined by the References-, Subject-, Date- and
>   other-headers (see [USEAGE] for some usual methods of doing this).

Of course it may. It may also display icons or green flashing dots. 

>7.8.  Duties of a Moderator

>   A Moderator receives news articles by email, decides whether to
>   accept them 

s/by email//
s/accept/approve/

He has already accepted the message -- it is in his mailbox or other input 
queue.

>7.9.1.  Duties of an Outgoing Gateway

>   From the perspective of Netnews, an outgoing gateway is just a
>   special type of reading agent.

Irrelevant and meaningless. Elide.

>                               The operation of the outgoing gateway
>   is only subject to additional constraints in the presence of one or
>   more corresponding incoming gateways back from that medium to
>   Netnews, since this opens the possibility of loops.

The operation of a gateway ought always to be based on the possibility
of the back gateway, not on the actual existance of same. That implies
contstraints based not on the presence, but on the ability. Elide 
sentence.

>   1. The message identifier of the news article should be preserved if
>      at all possible, preferably as or within the corresponding unique
>      identifier of the other medium, but if not at least as a comment
>      in the message. This helps greatly with preventing loops.

I fail to see how a comment in the body of a message helps to prevent 
loops.

>7.9.2.  Duties of an Incoming Gateway

>   The incoming gateway has the serious responsibility of ensuring that
>   all of the requirements of this standard are met by the articles that
>   it forms.

ALL? Or do you mean that it must meet the standards expected by the 
injecting agent that it will be using?

>              In addition to its special duties as a gateway, it bears
>   all of the duties and responsibilities of an injecting agent as well,
>   and additionally has the same responsibility of a relaying agent to
>   reject articles that it has already gatewayed.

I guess you did mean "all". I disagree. An incoming gateway does NOT need 
to be an injecting and relaying agent too. It can function quite well 
below the injection agent. 

>   An incoming gateway MUST NOT gate the same message twice. 

What is the reason for this RFC2119 mandate? The injecting agent will stop 
a duplicate.

>   Incoming gateways MUST NOT pass control messages (articles containing
>   a Control- or Supersedes-header) without removing or renaming that
>   header. 

Thus preventing a user on the other side of an incoming gateway from ever 
cancelling or superceding his own articles. 

>                          If a gateway receives a message that it can
>   determine is a valid equivalent of a cancel message in the medium it
>   is gatewaying, it SHOULD discard that message without gatewaying it,
>   generate a corresponding cancel message of its own, and inject that
>   cancel message.

Cool. My gateway's method of determining a "valid equivalent" is by seeing
a Control header. But I must remove it, even though I am permitted to 
put it in. Which is it?

>        different portions).  In these cases, where there is no one
>        "official" gateway, some other method of generating message
>        identifiers has to be used to avoid collisions. It would
>        obviously be preferable for there to be only one gateway which
>        crossposts, but this may not be possible to coordinate.

Given the previous discussion about possiblilities and not actual 
presence, then what we are actually saying is that gateways MUST create
message ids for news messages and MUST NOT use existing ids found in any
incoming message.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: John Stanley <[email-masked]>
Subject: IMC ietf-usefor archive
Date: Mon, 6 Dec 2004 13:53:43 -0500
Cc: [email-masked]
Sender: [email-masked]

On Mon December 6 2004 12:47, John Stanley wrote:
> 
> "There is a complete archive of the mailing list." 

The corresponding link points to a web interface which
presents only fractions of messages.  And for the record,
the Landfield.com web interface also presented only
fractions of messages (in particular, there is no way in
either case to examine message header contents).

> >If you want "the archive", I suspect that you may find a
> >link to one on the IMC site ...

You will find on the same page the text
"You can also access the entire archive.", and the
corresponding link will provide you with the real archive,
unfettered by HTML constraints and without the
obfuscation of content that typically accompanies
web interfaces.

> I like whatever "interface" it is that does not mangle the content of 
> messages it is supposed to be archiving. Archives ought not to be hiding
> the content they are intended to maintain.

The real archive, a.k.a. "entire archive" is not obfuscated.

Date: Mon, 6 Dec 2004 09:47:41 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Sender: [email-masked]

Bruce Lilly <blilly@xxxxxxxxx>:

>You're not looking at "the archive"; you're looking at a bunch of HTML
>that bears a superficial resemblance to some fraction of an archive
>message.

"There is a complete archive of the mailing list." 

>If you want "the archive", I suspect that you may find a
>link to one on the IMC site ...

Yes, that's where I found it.

>Now why on Earth would somebody who refuses to accept mail
>want working mailto URIs ;/? 

I don't refuse all email, and the issue is not email, it is the continued
mangling of messages.

Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx>:

>Web interfaces to archives of mailing lists _always_ munge
>anything with an "@" in it, this is just the 3rd millenium.

The don't "always" do this, and if this is already the 3rd millenium
you'd think programmers would be smart enough to know how not to do this.

>Maybe you like the GMaNe interfaces, it has even a "blog style",
>RDF news, search, NNTP, and other features. 

I like whatever "interface" it is that does not mangle the content of 
messages it is supposed to be archiving. Archives ought not to be hiding
the content they are intended to maintain.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: Archive mangling of record traffic...
Date: Mon, 06 Dec 2004 02:22:54 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 20
Sender: [email-masked]

Charles Lindsey wrote:

> Well that is not, of course, what I wrote.

Web interfaces to archives of mailing lists _always_ munge
anything with an "@" in it, this is just the 3rd millenium.

> The way things were archived at Landfield were much better
> (when they worked, that is :-( ).

Maybe you like the GMaNe interfaces, it has even a "blog style",
RDF news, search, NNTP, and other features.  It allows no write
access with bogus addresses for obvious reasons.  Examples:

<http://article.gmane.org/gmane.ietf.usenet.format/27911>
<http://article.gmane.org/gmane.ietf.usenet.format/27911/raw>
<http://news.gmane.org/group/gmane.ietf.usenet.format/thread=27911>

                       Bye, Frank

To: LIST: [email-masked];
Xref: clerew local.usefor:20385
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: FWS problem
Date: Sat, 4 Dec 2004 20:58:14 GMT
Lines: 78
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Hi, I still don't understand the overall syntax for all header
>fields.  The drafts say something like:

>         field = "name:" SP body CRLF
>         body  = [FWS] content [FWS]

>But it also says:

>| The header contents of every header line (including the first
>| and any that are subsequently folded) MUST contain at least
>| one non-whitespace character.

>The definition of "header content" is "field body" as in 2822.
>Therefore "name:" SP CRLF SP content CRLF is illegal, and the
>version   "name:" SP content CRLF SP CRLF is also illegal, in
>both cases there would a line without non-whitespace content.

Yes, you are right, and the wording in USEFOR needs to be fixed.

Perhaps something like:

   This specification uses the terms "header", "header name", and
   "header content" which are synonymous with the [RFC2822] terms
   "header field", "field name", and "field body" respectively. Note
   that "header content" includes all whitespace (including the
   initial SP) after the ':'.

and then:

   o  All agents MUST generate headers so that at least one space
      immediately follows the ':' separating the header name and the rest
      of the header content (for compatibility with deployed software).
      News agents MAY accept headers which do not contain the required
      space.
   o  The header content of every header line (including the first and
      any that are subsequently folded) MUST contain at least one
      non-whitespace character.
         NOTE: This means that no header content defined by or referenced
	 by this document can consist of WSP only.  As a result, this
	 document updates the <unstructured> construct from Section 3.2.6
	 of [RFC2822] as follows:

Alternatively, we might define "header content" to exclude any leading
WSP.

Ken, could you please fix this in the next draft?

>First problem, why this fuzz about a mandatory SP ?

Because much deployed software breaks without it. ANd for the same reason
it is also required by the new NNTP draft.

>Some of the then legal forms are:

>(ugly) name: TAB content CRLF
>(ugly) name: CRLF SP content TAB CRLF
>(????) name: SP CRLF TAB content CRLF
>(????) name: SP content CRLF TAB CRLF

And all of those would cause problems (often severe) for existing
software.

>(good) name: SP SP content CRLF
>(good) name: SP TAB content TAB CRLF

To: LIST: [email-masked];
Xref: clerew local.usefor:20384
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Archive mangling of record traffic...
Date: Sat, 4 Dec 2004 20:39:20 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>While reading a comment of Charles', I saw:

>>The case we were considering was where we might put, in Injection-Info:

>>... mail-complaints-to="abuse@xxxxxxxxxxxxxxxxxxx";
>>    mail-complaints-to="abuse@xxxxxxxxxxxxxxxxxxx";

Well that is not, of course, what I wrote.

>No, it is another example of the archive of this group mangling the body
>of a message to make its meaning absolutely opaque. Don't we have enough
>opaque content without the archive helping us?

Indeed. I am not at all satisfied with the archiving arrangements at
imc.org. The way things were archived at Landfield were much better (when
they worked, that is :-( ).

To: LIST: [email-masked];
Xref: clerew local.usefor:20383
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Sat, 4 Dec 2004 20:36:55 GMT
Lines: 88
Sender: [email-masked]

>> they rely on generating a random number long enough that the
>> probability of getting an accidental duplicate is reduced to
>> once per thousand years, or somesuch.

>Not good enough for the MUST in 2822, and it won't be allowed
>anymore with your drafts.  At the moment it's at least dubious.

It's a MUST in RFC 2822, and the current USEFOR draft inherits that MUST,
no more and no less.

It has been discussed peridically in this WG, and the opinion expressed
has always been that one failure per thousand years (whatever - I haven't
calculated the exact probability for the Agent scheme) could be regarded
as satisfying that MUST for all practical purposes. Nobody ever asked for
special wording to cover that case.

>> I have no problem with that as an adequate implementation
>> technique.

>But I have, it's really annoying to find these problems if an
>article is apparently lost, or if Groups.Google accepts dupes
>keeping only the newest instead of older identical Message-IDs.

If Google makes a mistake once every thousand years for this particular
cause, then that is as nothing compared to all the stupidities that Google
regularly makes every day of the week :-( .

Has anyone else anything to say on this topic? My preference is to leave
it as it is, and turn a blind eye once every thousand years (whevever).

>Some days ago somebody asked in de.admin.news.misc about his
>Message-ID <xyz@news-fe-01>, and the consensus was that it's
>wrong.  The ..!news-fe-01!.. in his Path: wasn't much better.

Well I suppose "xyz" COULD be the first in a long sequence of strings all
provably different, but it is not a convincing example, as you say.

>> the basic concept of the RFC 2822 <msg-id> is broken, but
>> there it is.

>If it's broken let's fix it, with the same strategy for both
>sides.  If the strategy for the RHS is "case sensitive", then
>it should be "keep quoted pairs literally" in the LHS.

But the id-right is NOT case-sensitive, whether it looks like it came from
a domain-name or not (I know Bruce will disagree with this, but he is
wrong). You will also see some wording on this in 7.3 of the latest
USEPRO.

>But at the moment you want a canonical form for the LHS, and
>a magical token for the RHS, I just don't understand it.  Why
>not use the same strategy for both sides, whatever it is ?

I would have no problem changing it if the WG consensus is to do so. But I
hear no other support for the idea.

>     NOTE:  News message IDs are a restricted subset of
>     MAIL message IDs.  In particular, no existing news
>     software  copes properly with MAIL quoting conven-
>     tions within the local part, so they  are  forbid-
>     den.

>If that was true 1994, why do you think that it's wrong now ?
>Is there any evidence that s-o-1036 is completely obsolete, a
>statistics about Message-ID syntax in news after 2822 maybe ?

I don't think it matters particularly. What happens in practice is that
news agents just take everything from the first '<' to the next '>' or SP
and do a simple comparison of the resulting string. I doubt any agent,
except an injecting agent, every looks inside to see what is there, and I
expect many injecting agents do little more than check there is a single
'@' somewhere in it.

To: LIST: [email-masked];
Xref: clerew local.usefor:20381
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: draft-ietf-usefor-usepro-02
Date: Fri, 3 Dec 2004 19:54:53 GMT
Lines: 798
Sender: [email-masked]

See http://www.ietf-usefor.imc.org/drafts/draft-ietf-usefor-usepro-02.txt
and http://www.ietf-usefor.imc.org/drafts/draft-ietf-usefor-usepro-02.unpaged

Also sent to the drafts editor.

The changes comprise:

A. Various items we discussed earlier, mostly regarding relaying agents
and control messages, and whose revised texts I posted here a couple of
weeks back.

B. Various bits of text formerly in draft 13 which, after discussion with
our Chair, it was decided belonged in USEPRO rather than USEFOR.
Mostly, items in this category do not change anything, although the new
wording regarding the use of the TLD ".invalid" in section 5 is a little
stronger than in the old a-5.2. If you think it is too strong, please
speak up.

All the changes are described briefly in an Appendix at the end. For those
who want to examine the changes word by word, here are the full diffs.

*** landfield/drafts/draft-ietf-usefor-usepro-01.unpaged	Mon Sep 20 17:18:31 2004
--- /tmp/usepro.02	Fri Dec  3 19:24:12 2004
***************
*** 2,9 ****
  INTERNET-DRAFT                               Charles H. Lindsey
  Usenet Format Working Group                  University of Manchester
!                                              September 2004
  
                  News Article Architecture and Protocols
!                    <draft-ietf-usefor-usepro-01.txt>
  
  Status of this Memo
--- 2,9 ----
  INTERNET-DRAFT                               Charles H. Lindsey
  Usenet Format Working Group                  University of Manchester
!                                              December 2004
  
                  News Article Architecture and Protocols
!                    <draft-ietf-usefor-usepro-02.txt>
  
  Status of this Memo
***************
*** 31,35 ****
     http://www.ietf.org/shadow.html.
  
!    This Internet-Draft will expire in March 2005.
  
  Abstract
--- 31,35 ----
     http://www.ietf.org/shadow.html.
  
!    This Internet-Draft will expire in June 2005.
  
  Abstract
***************
*** 83,87 ****
    2.1.  Definitions ...............................................    0
    2.2.  Defining the Architecture .................................    0
!   2.3.  Textual Notations .........................................    0
  3.  Changes to the existing protocols .............................    0
    3.1.  Principal Changes .........................................    0
--- 83,90 ----
    2.1.  Definitions ...............................................    0
    2.2.  Defining the Architecture .................................    0
!   2.3.  Header Properties .........................................    0
!     2.3.1.  Inheritable Headers ...................................    0
!     2.3.2.  Variant Headers .......................................    0
!   2.4.  Textual Notations .........................................    0
  3.  Changes to the existing protocols .............................    0
    3.1.  Principal Changes .........................................    0
***************
*** 252,261 ****
  
     An "article" is the unit of news, synonymous with an [RFC 2822]
!    "message".
  
-    A "proto-article" (7.2.1) is one that has been created by a "posting
-    agent" but has not yet been injected into a Netnews system system by
-    an "injecting agent". It may lack some otherwise mandatory headers.
- 
     A "message identifier" (a-5.3) is a unique identifier for an article,
     usually supplied by the posting agent which posted it or, failing
--- 255,263 ----
  
     An "article" is the unit of news, synonymous with an [RFC 2822]
!    "message".  A "proto-article" (7.2.1) is one that has been created by
!    a "posting agent" but has not yet been injected into a Netnews system
!    system by an "injecting agent". It may lack some otherwise mandatory
!    headers.
  
     A "message identifier" (a-5.3) is a unique identifier for an article,
     usually supplied by the posting agent which posted it or, failing
***************
*** 281,286 ****
     used where several initial components are shared.
  
!    A "poster" is the person or software that composes and submits a
!    possibly compliant article to a posting agent. The poster is
     synonymous with [RFC 2822]'s author.
  
--- 283,288 ----
     used where several initial components are shared.
  
!    A "poster" is the person or software that composes a possibly
!    compliant article for submission to a posting agent. The poster is
     synonymous with [RFC 2822]'s author.
  
***************
*** 287,300 ****
     A "reader" is the person or software reading news articles.
  
!    A "followup" is an article containing a response to the contents of
!    an earlier article (the followup's "precursor").
! [Alternative definition, to bu ised if similar woprding is not added to
! the description of the References-header (a-6.10):]
  
     A "followup" is an article containing a response to the contents of
!    an earlier article (its "precursor"), or which is otherwise intended
!    to be grouped with that article for purposes of display (e.g. as part
!    of a multipart posting such as a FAQ).
  
     An (email) "address" is the mailbox [RFC 2822] (or more particularly
     the addr-spec within that mailbox) which directs the delivery of an
--- 289,310 ----
     A "reader" is the person or software reading news articles.
  
! [Alternative-1.]
  
     A "followup" is an article containing a response to the contents of
!    an earlier article.
  
+    An article is a "precursor" of some later article which is a followup
+    to it, or which is otherwise intended to be grouped with it for
+    purposes of display (e.g. as a part of a multipart posting such as a
+    FAQ).
+ 
+ [Alternative-2.]
+ 
+    A "followup" to an earlier article (its "precursor") is one intended
+    to be grouped with that article for purposes of display (e.g. because
+    it is a response to its contents or is a part of a multipart posting
+    such as a FAQ).
+ [End of alternatives]
+ 
     An (email) "address" is the mailbox [RFC 2822] (or more particularly
     the addr-spec within that mailbox) which directs the delivery of an
***************
*** 309,317 ****
     agent or, which amounts to the same thing, for passing the article to
     the injecting agent.
  
     A "control message" is an article which is marked as containing
!    control information; a "serving agent" receiving such an article may
!    (subject to the policies observed at that site) take actions beyond
!    just filing and passing on the article.
  
  2.2.  Defining the Architecture
--- 319,329 ----
     agent or, which amounts to the same thing, for passing the article to
     the injecting agent.
+ [Is the definition in RFC 2822 sufficient?]
  
     A "control message" is an article which is marked as containing
!    control information; a "serving agent" (and in some cases a "relaying
!    agent") receiving such an article may (subject to the policies
!    observed at that site) take actions beyond just filing and passing on
!    the article.
  
  2.2.  Defining the Architecture
***************
*** 334,345 ****
     A "reading agent" is software which presents articles to a reader.
  
     A "followup agent" is a combination of reading agent and posting
     agent that aids in the preparation and posting of a followup.
- [Alternative definition, to be used if the alternative definition for
- "followup" is used:]
  
     A "followup agent" is a combination of reading agent and posting
!    agent that aids in the preparation and posting of a response intended
!    as a followup to a precursor.
  
     An "injecting agent" takes the finished article from the posting
--- 346,360 ----
     A "reading agent" is software which presents articles to a reader.
  
+ [Alternative-1.]
+ 
     A "followup agent" is a combination of reading agent and posting
     agent that aids in the preparation and posting of a followup.
  
+ [Alternative-2.]
+ 
     A "followup agent" is a combination of reading agent and posting
!    agent that aids in the preparation and posting of an article which is
!    a response to some precursor.
! [End of alternatives]
  
     An "injecting agent" takes the finished article from the posting
***************
*** 377,382 ****
     collectively as "user agents".
  
! 2.3.  Textual Notations
  
     This standard contains explanatory NOTEs using the following format.
     These may be skipped by persons interested solely in the content of
--- 393,447 ----
     collectively as "user agents".
  
! 2.3.  Header Properties
  
+    There are two special properties that may apply to particular
+    headers, namely: "inheritable", and "variant". When a header is
+    defined, in this (or any future) standard, as having one (or possibly
+    more) of these properties, it is subject to special treatment, as
+    indicated below.
+ 
+ 2.3.1.  Inheritable Headers
+ 
+    Subject only to the overriding ability of the poster to determine the
+    contents of the headers in a proto-article, headers with the
+    inheritable property MUST be copied by followup agents (perhaps with
+    some modification) into the followup article, and headers without
+    that property MUST NOT be so copied.
+ 
+    The following headers (see [USEFOR] for their definitions) are
+    classified as "inheritable":
+      o Newsgroups (a-5.5) - copied from the precursor, subject to any
+        Followup-To-header.
+      o Subject (a-5.4) - often modified by prefixing with "Re: ", but
+        otherwise copied from the precursor.
+      o References (a-6.10) - copied from the precursor, with the
+        addition of the precursor's Message-ID.
+      o Distribution (a-6.6) - copied from the precursor.
+ 
+         NOTE: The Keywords-header is not inheritable, though some older
+         newsreaders treated it as such.
+ 
+ 2.3.2.  Variant Headers
+ 
+    Headers with the variant property may differ between (or even be
+    completely absent from) copies of the same article as stored or
+    relayed throughout a Netnews system. The manner of the difference (or
+    absence) MUST be as specified in this (or some future) standard.
+    Typically, these headers are modified as articles are propagated, or
+    they reflect the status of the article on a particular serving agent,
+    or cooperating group of such agents. The variant header MAY be placed
+    anywhere within the headers (though placing it first is recommended).
+ 
+ 
+    The following headers (see [USEFOR] for their definitions) are
+    classified as "variant":
+      o Path (a-5.6) - augmented at each relaying agent that an article
+        passes through.
+      o Xref (a-6.16) - used to keep track of the article locators of
+        crossposted articles so that reading agents serviced by a
+        particular serving agent can mark such articles as read.
+ 
+ 2.4.  Textual Notations
+ 
     This standard contains explanatory NOTEs using the following format.
     These may be skipped by persons interested solely in the content of
***************
*** 454,460 ****
         Complaints-To, Injection-Info, Mail-Copies-To, Posted-And-Mailed
         and User-Agent, leading to increased functionality.
-      o Provision has been made for almost all headers to have MIME-style
-        parameters (to be ignored if not recognized), thus facilitating
-        extension of those headers in future standards.
       o Certain headers and Control messages (a-Appendix A.3 and Appendix
         A.3) have been made obsolete.
--- 520,523 ----
***************
*** 728,731 ****
--- 789,829 ----
     otherwise.
  
+    Each control message comprises a verb, which indicates the action to
+    be taken, and argument(s), which supply the details. In some cases
+    the body of the article may also supply details. The following
+    sections contain syntactic definitions for the verb, arguments, and
+    possibly the body, for each type of control message.
+ 
+    The Newsgroups-header of each control message SHOULD include the
+    newsgroup-name(s) for the group(s) affected (i.e. groups to be
+    created, modified or removed, or containing articles to be canceled).
+    This is to ensure that the message propagates to all sites which
+    receive (or would receive) that group(s). It MAY include other
+    newsgroup-names so as to improve propagation (but this practice may
+    cause the control message to propagate also to places where it is
+    unwanted, or even cause it not to propagate where it should, so it
+    should not be used without good reason).
+ 
+         NOTE: Propagation is controlled by relaying agents, and it may
+         be necessary for relaying agents to take special steps to ensure
+         that control messages such as newgroup messages for not-yet-
+         existent newsgroups are propagated correctly (see 7.3).
+ [The first of those paragraphs was originally scheduled to be moved to
+ USEFOR (and it may yet be so moved). It has been reinstated here (at
+ least temporarily) to make sure that it does not get overlooked.]
+ 
+    The presence of a Subject-content starting with the string "cmsg "
+    and followed by a <control-message> was construed under [RFC 1036] as
+    a request to perform that control action (even if no genuine Control
+    header was present). Indeed, some implementations went further and
+    added the implied Control header before injecting. Likewise, the
+    presence of a newsgroup-name ending in ".ctl" in the Newsgroups
+    header caused the Subject header content (not starting with "cmsg" in
+    this case) to be interpreted as a <control-message>.
+ 
+    All these practices are now declared to be Obsolete, and Subject
+    headers MUST NOT now be interpreted as <control-message>s under any
+    circumstances.
+ 
     The descriptions below set out REQUIREMENTS to be followed by sites
     that receive control messages and choose to honour them. However,
***************
*** 736,751 ****
     basis).
  
-    Relaying Agents MUST propagate all control messages regardless of
-    whether or not they are recognized or processed locally.
- 
-    In the following sections, each type of control message is defined
-    syntactically by defining its verb, its arguments, and possibly its
-    body.
- 
  6.1.  Digital Signature of Headers
  
! [Removed to [USEFOR] (unless it remains in this document as the Primary
! document, in which case it would not be placed here.]
  
  6.2.  Group Control Messages
  
--- 834,855 ----
     basis).
  
  6.1.  Digital Signature of Headers
  
!    It is most desirable that group control messages (6.2) in particular
!    be authenticated by incorporating them within some digital signature
!    scheme that encompasses other headers closely associated with them
!    (including at least the Approved-, Message-ID- and Date-headers). At
!    the time of writing, this is usually done by means of a protocol
!    known as "PGPverify" ([PGPVERIFY]), and continued usage of this is
!    encouraged at least as an interim measure.
  
+    However, PGPverify is not considered suitable for standardization in
+    its present form, for various technical reasons. It is therefore
+    expected that an early extension to this standard will provide a
+    robust and general purpose digital authentication mechanism with
+    applicability to all situations requiring protection against
+    malicious use of, or interference with, headers.  That extension
+    would also address other Netnews security issues.
+ 
  6.2.  Group Control Messages
  
***************
*** 1120,1129 ****
  
          NOTE: It is expected that the security extension envisaged in
!         section a-7.1 will make more detailed provisions for
!         establishing whether honouring a particular cancel message is in
!         order. In particular, it is likely that there will be provision
!         for the digital signature of 3rd party cancels (i.e. those
!         issued other than by the sender, the moderator, or the
!         injector).
  
          NOTE: A cancel submitted by the poster for an article in a
--- 1221,1229 ----
  
          NOTE: It is expected that the security extension envisaged in
!         section 6.1 will make more detailed provisions for establishing
!         whether honouring a particular cancel message is in order. In
!         particular, it is likely that there will be provision for the
!         digital signature of 3rd party cancels (i.e. those issued other
!         than by the sender, the moderator, or the injector).
  
          NOTE: A cancel submitted by the poster for an article in a
***************
*** 1368,1385 ****
        (e.g. more than 72 hours into the past).
  
!    4. It MUST reject any article that does not have the correct
!       mandatory headers for a proto-article (or, when reinjecting, all
!       the mandatory headers other than Injection-Date), or which
!       contains any header that does not have legal contents.  It SHOULD
!       reject any article which contains any header deprecated for
!       Netnews (a-4.2.1).  It SHOULD reject any article whose
!       Newsgroups-header does not contain at least one newsgroup-name for
!       an existing group (as listed by its associated serving agent) and
!       it MAY reject any newsgroup-name which, although syntactically
!       correct, violates a policy restriction established, for some
!       (sub-)hierarchy, by an agency with the appropriate authority
!       (1.2).  Observe that crossposting to unknown newsgroups is not
!       precluded provided at least one of those in the Newsgroups-header
!       is listed.
  
          NOTE: This ability to reject newsgroup-names in breach of
--- 1466,1482 ----
        (e.g. more than 72 hours into the past).
  
!    4. It MUST reject any article that does not have the proper mandatory
!       headers for a proto-article (or, when reinjecting, all the
!       mandatory headers other than Injection-Date), or which contains
!       any header that does not have legal contents.  It SHOULD reject
!       any article which contains any header deprecated for Netnews (a-
!       4.2.1).  It SHOULD reject any article whose Newsgroups-header does
!       not contain at least one newsgroup-name for an existing group (as
!       listed by its associated serving agent) and it MAY reject any
!       newsgroup-name which, although syntactically correct, violates a
!       policy restriction established, for some (sub-)hierarchy, by an
!       agency with the appropriate authority (1.2).  Observe that
!       crossposting to unknown newsgroups is not precluded provided at
!       least one of those in the Newsgroups-header is listed.
  
          NOTE: This ability to reject newsgroup-names in breach of
***************
*** 1514,1519 ****
     distribution "world", and NO relaying agent should supply or accept
     the distribution "local".
- [That SHOULD has been demoted from a MUST in draft-13. Any objections?]
  
          NOTE: Although it would seem redundant to filter out unwanted
          distributions at both ends of a relaying link (and it is clearly
--- 1610,1620 ----
     distribution "world", and NO relaying agent should supply or accept
     the distribution "local".
  
+    However, if the particular implementation does not relay non-existent
+    newsgroups, even when included in the Newsgroups-header and implied
+    (e.g. by some "wild card" notation) in the configuration tables, then
+    the agent MUST examine all group control messages (6.2) in order to
+    ensure that relaying of those messages proceeds normally.
+ 
          NOTE: Although it would seem redundant to filter out unwanted
          distributions at both ends of a relaying link (and it is clearly
***************
*** 1538,1547 ****
          distributions has been provided (7.7) for the same reason.
  
!    An article SHOULD NOT be relayed if the path-identity of the
!    receiving agent (or some known alias thereof) appears in its Path-
!    header, and the receiving agent MAY detect whether its own path-
!    identity is already present in the Path-content so as to avoid
!    further unnecessary relaying.
! [See related remarks under serving agents.]
  
     A relaying agent processes articles as follows:
--- 1639,1645 ----
          distributions has been provided (7.7) for the same reason.
  
!    In order to avoid unnecessary relaying, an article SHOULD NOT be
!    relayed if the path-identity of the receiving agent (or some known
!    alias thereof) appears in its Path-header.
  
     A relaying agent processes articles as follows:
***************
*** 1572,1583 ****
        less than that 24 hours).
  
!    3. It MUST reject any article that does not have the correct
!       mandatory headers (section a-5) present with legal contents.
  
!    4. It SHOULD reject any article whose optional headers (section a-6)
!       do not have legal contents.
! [Is that too strong? Are relaying agents really expected to check
! headers in that detail? I suggest s/SHOULD/MAY/. Even the MUST in Step 4
! for mandatory headers might be demoted to SHOULD.]
  
     5. It SHOULD reject any article that has already been sent to it (a
--- 1670,1678 ----
        less than that 24 hours).
  
!    3. It SHOULD reject any article that does not include all the
!       mandatory headers (section a-5).
  
!    4. It MAY reject any article whose headers do not have legal
!       contents.
  
     5. It SHOULD reject any article that has already been sent to it (a
***************
*** 1585,1588 ****
--- 1680,1696 ----
        and matched against).
  
+         NOTE: Even though commonly derived from the domain name of the
+         originating site (and domain names are case-insensitive), a
+         message identifier MUST NOT be altered in any way during
+         transport, or when copied (as into a References-header), and
+         thus a simple (case-sensitive) comparison of octets will always
+         suffice to recognize that same message identifier wherever it
+         subsequently reappears.
+ 
+         NOTE: These requirements are to be contrasted with those of the
+         un-normalized msg-ids defined by [RFC 2822], which may perfectly
+         legitimately become normalized (or vice versa) during transport
+         or copying in email systems.
+ 
     6. It SHOULD reject any article that matches an already received
        cancel message (or an equivalent Supersedes-header) issued by its
***************
*** 1610,1614 ****
  
     Relaying agents MUST NOT alter, delete or rearrange any part of an
!    article expect for headers designated as variant (a-4.2.5.3).  In
     particular
  
--- 1716,1720 ----
  
     Relaying agents MUST NOT alter, delete or rearrange any part of an
!    article expect for headers designated as variant (2.3.2).  In
     particular
  
***************
*** 1689,1702 ****
          hierarchy under "control." (e.g. "control.cancel").
  
!    A serving agent MAY decline to accept an article if its own path-
!    identity is already present in the Path-content or if the Path-
!    content contains some path-identity whose articles the serving agent
!    does not want, as a matter of local policy.
! [That has been changed from a-5.6.1 in previous drafts, where it said "A
! relaying agent MAY decline...". That seemed plain wrong to me. It is
! fine for a relaying agent to ignore articles which have apparently
! already passed through it (and I still say that), but surely it is for
! serving agents to "reject" for policy reasons, and to which the NOTE
! below would apply.  Comments?]
  
          NOTE: This last facility is sometimes used to detect and decline
--- 1795,1801 ----
          hierarchy under "control." (e.g. "control.cancel").
  
!    A serving agent MAY decline to accept an article if the Path-content
!    contains some path-identity whose articles the serving agent does not
!    want, as a matter of local policy.
  
          NOTE: This last facility is sometimes used to detect and decline
***************
*** 1704,1710 ****
          deliberately seeded with a path-identity to be "aliased out" by
          sites not wishing to act upon them.
! [Again, is this "aliasing out" usually detected by the serving agent, or
! does it more usually work because the previous relaying agent will never
! have sent it in the first place?]
  
     A serving agent processes articles as follows:
--- 1803,1808 ----
          deliberately seeded with a path-identity to be "aliased out" by
          sites not wishing to act upon them.
! [INN at least does this. It might be argued that it is not necessary to
! mention it here.]
  
     A serving agent processes articles as follows:
***************
*** 1719,1725 ****
        less than that 24 hours).
  
!    3. It MUST reject any article that does not have the correct
!       mandatory headers (section a-5) present, or which contains any
!       header that does not have legal contents.
  
     4. It SHOULD reject any article that has already been sent to it (a
--- 1817,1823 ----
        less than that 24 hours).
  
!    3. It MUST reject any article that does not include all the mandatory
!       headers (section a-5), or which contains any header that does not
!       have legal contents.
  
     4. It SHOULD reject any article that has already been sent to it (a
***************
*** 1757,1768 ****
     or Complaints-To-header.
  
     Posting agents meant for use by ordinary posters SHOULD reject any
     attempt to post an article which cancels or Supersedes another
!    article of which the poster is not the author.
  
  7.6.  Duties of a Followup Agent
  
--- 1856,1868 ----
     or Complaints-To-header.
  
+    Contrary to [RFC 2822], which states that the mailbox(es) in the From
+    header is that of the poster(s), a poster who does not, for whatever
+    reason, wish to use his own mailbox MAY use any mailbox ending in the
+    top level domain ".invalid" [RFC 2606].
+ 
     Posting agents meant for use by ordinary posters SHOULD reject any
     attempt to post an article which cancels or Supersedes another
!    article of which the poster is not the author or sender.
  
  7.6.  Duties of a Followup Agent
  
***************
*** 1770,1779 ****
     bound by all the posting agent's requirements. Followup agents MUST
     create valid followups and are subject to special requirements
!    involving certain inheritable (a-4.2.5.2) and other headers.
!    Wherever in the following it is stated that, "by default", a header
!    is to be taken from some header in the precursor, it means that its
!    initial content (plus its extension-parameters, if any) are to be
!    copied from those of that precursor header.  However, posters MAY
!    then override that default before posting if they so wish.
  
          NOTE: There is no provision in this standard for a followup to
--- 1870,1879 ----
     bound by all the posting agent's requirements. Followup agents MUST
     create valid followups and are subject to special requirements
!    involving certain inheritable (2.3.1) and other headers. Wherever in
!    the following it is stated that, "by default", a header is to be
!    taken from some header in the precursor, it means that its initial
!    content is to be copied from the content of that precursor header.
!    However, posters MAY then override that default before posting if
!    they so wish.
  
          NOTE: There is no provision in this standard for a followup to
***************
*** 1796,1805 ****
  
     4. If the precursor did not have a References-header (a-6.10), the
!       followup's References-content MUST be formed by the message
!       identifier of the precursor. A followup to an article which had a
!       References-header MUST have a References-header containing the
!       precursor's References-content (subject to trimming as described
!       below) plus the precursor's message identifier appended to the end
!       of the list (separated from it by CFWS).
  
        Followup agents MAY trim References-headers which have grown to
--- 1896,1905 ----
  
     4. If the precursor did not have a References-header (a-6.10), the
!       followup's References-content MUST be taken from the Message-ID-
!       content of the precursor. A followup to an article which already
!       had a References-header MUST have a References-header comprising
!       the precursor's References-content (subject to trimming as
!       described below) followed by CFWS and the Message-ID-content of
!       the precursor.
  
        Followup agents MAY trim References-headers which have grown to
***************
*** 1825,1829 ****
        a copy-addr
              The followup agent SHOULD likewise email a copy of a posted
!             followup, which MUST then be sent to the copy-addr.
  
        When emailing a copy, the followup agent SHOULD also include a
--- 1925,1929 ----
        a copy-addr
              The followup agent SHOULD likewise email a copy of a posted
!             followup, which SHOULD then be sent to the copy-addr.
  
        When emailing a copy, the followup agent SHOULD also include a
***************
*** 1837,1846 ****
     A reading agent downloads articles from a serving agent, as directed
     by the reader, and displays them (or processes them in some other
!    manner).  The article as displayed MUST be identical to the article
!    as originally posted, subject only to limitations of the display
!    device (such as availability of charsets, etc.). It MUST provide
!    facilities for decoding any Content-Transfer-Encodings, encoded-
!    words, etc., but SHOULD also have the capability to show the article
!    exactly as received.
  
     It MAY present lists of articles available for display, and MAY
--- 1937,1944 ----
     A reading agent downloads articles from a serving agent, as directed
     by the reader, and displays them (or processes them in some other
!    manner), subject to any limitations of the reading agent, such as
!    availability of charsets, and having first decoded any Content-
!    Transfer-Encodings, encoded-words, etc.  It SHOULD also have the
!    capability to show the raw article exactly as received.
  
     It MAY present lists of articles available for display, and MAY
***************
*** 1847,1855 ****
     structure those lists so as to show the relationships between the
     articles, as determined by the References-, Subject-, Date- and
!    other-headers (see [USEAGE] for some usual methods of doing this). It
!    MAY also be configured so that unwanted distributions (a-6.6) are
!    ignored.
  
- 
  7.8.  Duties of a Moderator
  
--- 1945,1951 ----
     structure those lists so as to show the relationships between the
     articles, as determined by the References-, Subject-, Date- and
!    other-headers (see [USEAGE] for some usual methods of doing this).
! [This whole section may yet get omitted]
  
  7.8.  Duties of a Moderator
  
***************
*** 1934,1939 ****
          any case, be caught by the injecting agent if it is not).
  
!    5. Any variant headers (a-4.2.5.3) MUST be removed, except that a
!       Path-header MAY be truncated to only its pre-injection region (a-
        5.6.3).  Any Injection-Info-header (a-6.19) or Complaints-To-
        header (a-6.20) SHOULD be removed (and if they are not, the
--- 2030,2035 ----
          any case, be caught by the injecting agent if it is not).
  
!    5. Any variant headers (2.3.2) MUST be removed, except that a Path-
!       header MAY be truncated to only its pre-injection region (a-
        5.6.3).  Any Injection-Info-header (a-6.19) or Complaints-To-
        header (a-6.20) SHOULD be removed (and if they are not, the
***************
*** 2265,2269 ****
     care to take the additional precautions of using more sophisticated
     anonymity measures, should avoid that violation by the use of
!    addresses ending in the ".invalid" top-level-domain (see a-5.2).
  
     A malicious poster may also prevent his article being seen at a
--- 2357,2361 ----
     care to take the additional precautions of using more sophisticated
     anonymity measures, should avoid that violation by the use of
!    addresses ending in the ".invalid" top-level-domain (see 7.5).
  
     A malicious poster may also prevent his article being seen at a
***************
*** 2421,2424 ****
--- 2512,2518 ----
          ietf-nntpext-base-*.txt.
  
+    [PGPVERIFY] David Lawrence,
+         <ftp://ftp.isc.org/pub/pgpcontrol/README.html>.
+ 
     [RFC 1036] M. Horton and R. Adams, "Standard for Interchange of
          USENET Messages", RFC 1036, December 1987.
***************
*** 2672,2673 ****
--- 2766,2814 ----
  
     5    Some renumbering of sections and minor textual clarifications.
+ 
+    For version 02
+ 
+    1    2nd para. of a-7 temporarily reinstated in section 6.
+ 
+    2    Para. in section 6 relating to propagation of control messages
+         and local policy removed to [USEAGE].]
+ 
+    3    Requirement for some relaying agents to examine control messages
+         for non-existent groups
+         6
+         7.3
+ 
+    4    Text regarding "aliasing out" brought into line with actual
+         practice.
+         7.3
+ 
+    5    More realistic wording regarding the expectations of reading
+         agents
+         7.7
+         7.4
+ 
+    6    "Precursor" is now defined for all cases in which a References
+         header may be used (even though "followup" is not always defined
+         under Alternative-1).
+         2.1
+ 
+    7    Provision is made for a poster to use a mailbox ending in
+         ".invalid" in a From header (formerly in a-5.2).
+         7.5
+ 
+    8    "Inheritable" and "Variant" headers defined (formerly in a-
+         4.2.5).
+         2.3
+ 
+    9    Additional wording regarding function of verb/arguments/body in
+         control messages (formerly in a-6.13).
+         6
+ 
+    10   NOTE regarding not altering message indentifiers during
+         transport or copying added (formerly in a-5.3).
+         7.3
+ 
+    11   All mention of MIME-style parameters and extension-parameters
+         removed.
+         3.1
+         7.6

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: John Stanley <[email-masked]>
Subject: Re: Archive mangling of record traffic...
Date: Fri, 3 Dec 2004 07:48:30 -0500
Cc: [email-masked], [email-masked]
Sender: [email-masked]

On Thu December 2 2004 18:22, John Stanley wrote:
> 
> While reading a comment of Charles', I saw:
> 
> >The case we were considering was where we might put, in Injection-Info:
> 
> >... mail-complaints-to="abuse@xxxxxxxxxxxxxxxxxxx";
> >    mail-complaints-to="abuse@xxxxxxxxxxxxxxxxxxx";

Obviously you're looking at the web interface, not at actual mailing
list messages.

[...]
> No, it is another example of the archive of this group mangling the body
> of a message to make its meaning absolutely opaque. Don't we have enough
> opaque content without the archive helping us?

You're not looking at "the archive"; you're looking at a bunch of HTML
that bears a superficial resemblance to some fraction of an archive
message.  If you want "the archive", I suspect that you may find a
link to one on the IMC site (I'm offline at the moment, so can't be
more specific), however that's likely to be mangled as well, though
in less severe ways (beware mangled Date fields).
 
> [email-masked]?
> [email-masked]?

Now why on Earth would somebody who refuses to accept mail
want working mailto URIs ;/? 

> And wouldn't all you archive lurkers love to know what I just said?

It's almost certainly *in* the *archive*.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: FWS problem
Date: Fri, 03 Dec 2004 03:10:12 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 57
Sender: [email-masked]

Hi, I still don't understand the overall syntax for all header
fields.  The drafts say something like:

         field = "name:" SP body CRLF
         body  = [FWS] content [FWS]

But it also says:

| The header contents of every header line (including the first
| and any that are subsequently folded) MUST contain at least
| one non-whitespace character.

The definition of "header content" is "field body" as in 2822.
Therefore "name:" SP CRLF SP content CRLF is illegal, and the
version   "name:" SP content CRLF SP CRLF is also illegal, in
both cases there would a line without non-whitespace content.

First problem, why this fuzz about a mandatory SP ?  Isn't it
enough to say that there must be "something" between the colon
and the content, and then define "something" to be FWS ?  This
would result in either

         field = "name:" FWS body CRLF
         body  = content *WSP
or
         field = "name:" body CRLF
         body  = FWS content *WSP

Some of the then legal forms are:

(ugly) name: TAB content CRLF
(ugly) name: CRLF SP content TAB CRLF
(????) name: SP CRLF TAB content CRLF
(????) name: SP content CRLF TAB CRLF
(good) name: SP SP content CRLF
(good) name: SP TAB content TAB CRLF

ugly = allowed in 2822, but not by the syntax in usefor-02.
???? = allowed in 2822 and the usefor-02 syntax, but not by
       its additional "MUST contain at least one non-WS" rule
good = allowed in 2822 and in usefor-02.

Still excluded (bad) are name:content CRLF and versions like
name: SP content CRLF SP CRLF or name: SP content CRLF TAB CRLF

This would reflect the CFWS rule in 2822:

| However, where CFWS occurs in this standard, it MUST NOT be
| inserted in such a way that any line of a folded header field
| is made up entirely of WSP characters and nothing else.

For 2822 it was hell with numerous obs-rules, but for the news
drafts it's possible to integrate this rule firmly in the ABNF
of all header fields.  Maybe "they" even copy this idea later
in 2822bis, it's obvious (?).
                              Bye, Frank

Date: Thu, 2 Dec 2004 15:22:03 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
cc: [email-masked]
Subject: Archive mangling of record traffic...
Sender: [email-masked]

While reading a comment of Charles', I saw:

>The case we were considering was where we might put, in Injection-Info:

>... mail-complaints-to="abuse@xxxxxxxxxxxxxxxxxxx";
>    mail-complaints-to="abuse@xxxxxxxxxxxxxxxxxxx";

and I wondered why there was all this fuss about two of the same thing
appearing twiceas a parameter in a header. Perhaps there are differing
numbers of x's? Perhaps one 'x' isn't an 'x', it's something that only
looks like an 'x'?

No, it is another example of the archive of this group mangling the body
of a message to make its meaning absolutely opaque. Don't we have enough
opaque content without the archive helping us?

[email-masked]?
[email-masked]?

And wouldn't all you archive lurkers love to know what I just said?

Date: Thu, 02 Dec 2004 09:30:45 -0800 (PST)
From: [email-masked]
Subject: Re: Does 2231 break MIME 1.0?
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

> >... That's why Ned wrote:

> >"It may not be explicitly stated that duplicate parameters aren't allowed, but
> >given that parameter order is required not to be significant (RFC 2046 section
> >1) is is hard to come up with any other result that makes sense."

> So should I understand that the present standards do not actually prohibit
> duplicate parameters anywhere, but that they do not make any sense in a
> lot of cases.

There is no case I am aware of where a parameter has been defined that allows
for duplication. If such a case has made it through, it was done in error and
if pointed out will be rectified ASAP.

I can assure you that any media type application that calls for parameter
duplication would be rejected by the media types reviewer (me) or by the IESG
in the case of a standards tree registration. Any attempt to define a
content-disposition parameter of this sort would be treated similarly.

The bottom line is that duplicate parameters don't make sense, the intent from
the get-go was that they not be allowed, the fact that this didn't get stated
explicitly is no reason it should be assumed to be allowed. Stuff like this
happens from time to time. And more generally, to assume that every concievable
form of bad behavior has been anticipated and explicitly prohibited in the
relevant standards documents is nothing short of pure folly.

> The case we were considering was where we might put, in Injection-Info:

> ... mail-complaints-to="[email-masked]";
>     mail-complaints-to="[email-masked]";

> OTOH, there are easier ways to do the same thing, such as by saying that
> the syntax of that parameter is:

>     mail-complaints-parameter = "mail-complaints-to" "=" address-list

> which also happens to be the syntax in the Complaints-To header as
> currently proposed.

It is axiomatic that a completely new field which defines a completely new set
of parameters gets to choose its own set of rules for how those parameters will
behave. As such, reasoning on the basis of what is or isn't allowed by
content-type or content-disposition is simply silly. The point was, is, and
always will be that choices such as whether or not duplication is or isn't
allowed in some new set of things has implications in regards to syntax,
semantics, and implementation, and therefore must be carefully considered.

				Ned

To: LIST: [email-masked];
Xref: clerew local.usefor:20374
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Does 2231 break MIME 1.0?
Date: Thu, 2 Dec 2004 15:47:39 GMT
Lines: 39
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>... That's why Ned wrote:

>"It may not be explicitly stated that duplicate parameters aren't allowed, but
>given that parameter order is required not to be significant (RFC 2046 section
>1) is is hard to come up with any other result that makes sense."

So should I understand that the present standards do not actually prohibit
duplicate parameters anywhere, but that they do not make any sense in a
lot of cases.

In that case, it depends on the purpose of the "foo" parameter, and
whether it makes sense to say "foo might be x; alternatively foo might be
y".

The case we were considering was where we might put, in Injection-Info:

... mail-complaints-to="[email-masked]";
    mail-complaints-to="[email-masked]";

OTOH, there are easier ways to do the same thing, such as by saying that
the syntax of that parameter is:

    mail-complaints-parameter = "mail-complaints-to" "=" address-list

which also happens to be the syntax in the Complaints-To header as
currently proposed.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: Does 2231 break MIME 1.0?
Date: Thu, 02 Dec 2004 06:18:06 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 37
Sender: [email-masked]

Bruce Lilly wrote:

> Content-type: multipart/mixed;
>   boundary*0* =''sim ;
>   boundary*1* = ple%20
>  ;
>   boundary*2* = boundary

Nice... ;-)  Fortunately my stupid script tries to guess where
a MIME part might begin ("--"), then it accepts any Content-*
incl. folding, a blank line, and a base64 line starting with
TV or UE.  Anything else is irrelevant for its purposes.
  
OTOH it scans only TOP 50 body lines, and then tries to fix the
outer boundary after 50 lines (for an abuse complaint, that's a
truncated message/rfc822).  It won't add a --simple boundary--
in your example, and abuse@WannaSpew would be very unhappy. :-(

>| It may not be explicitly stated that duplicate parameters
>| aren't allowed, but given that parameter order is required
>| not to be significant (RFC 2046 section 1) is is hard to
>| come up with any other result that makes sense.

Yes, that was wrong, it was not hard to find counter-examples.
In fact we stumbled over an example without looking for it, our
complaint-url= case.  Alexey only said "I want more than one".
 
> I've never claimed that there's a "bug" associated with
> continuation parameters; that's your claim, but I don't think
> you've convinced anybody.

My original claim was "okay, let's simply have zero or more of
these complaint-url= beasts".  I checked that it should be okay
for MIME-2049.  And then you demonstrated that it's _not_ okay
for MIME-2231.  I was immediately convinced by your proof, and
changed the subject of this thread.  JFTR, bye, Frank  <beg>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: Does 2231 break MIME 1.0?
Date: Thu, 02 Dec 2004 05:32:55 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 18
Sender: [email-masked]

Claus Färber wrote:
 
> But why, if you just label your content with the most common
> charset; e.g. ISO-8859-1.

It was a hypothetical example, Bruce said that all unordered
lists are sets, and I needed something where that's obviously
not the case.

>> I'm ready to accept replies with windows-1252".
> No. That would be a very bad idea.

With my old MUA it would be an excellent idea, it allows me to
discuss with Hažek about “€-crats” ;-)

                         Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: msg-id
Date: Thu, 02 Dec 2004 05:17:17 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 66
Sender: [email-masked]

Charles Lindsey wrote:

> they rely on generating a random number long enough that the
> probability of getting an accidental duplicate is reduced to
> once per thousand years, or somesuch.

Not good enough for the MUST in 2822, and it won't be allowed
anymore with your drafts.  At the moment it's at least dubious.

> I have no problem with that as an adequate implementation
> technique.

But I have, it's really annoying to find these problems if an
article is apparently lost, or if Groups.Google accepts dupes
keeping only the newest instead of older identical Message-IDs.

Okay, in that particular case it's possible to say "fix your
newsreader" or "ignore deja.news", but that's not the general
idea of a Message-ID:

>> Message-IDs are essential, we're supposed to get this right

 [2822bis] 
> It never got beyond some preliminary discussions on the
> ietf-822 mailing list.

Okay, 2822bis is a fata morgana, but your drafts are real, and
they should be convincing for s-o-1036 fundamentalists.  The
Message-ID doesn't convince me.
  
Some days ago somebody asked in de.admin.news.misc about his
Message-ID <xyz@news-fe-01>, and the consensus was that it's
wrong.  The ..!news-fe-01!.. in his Path: wasn't much better.

> the basic concept of the RFC 2822 <msg-id> is broken, but
> there it is.

If it's broken let's fix it, with the same strategy for both
sides.  If the strategy for the RHS is "case sensitive", then
it should be "keep quoted pairs literally" in the LHS.

If the strategy is to get rid of any quoted pairs in the LHS,
then it's also okay to treat the RHS as "case insensitive".

But at the moment you want a canonical form for the LHS, and
a magical token for the RHS, I just don't understand it.  Why
not use the same strategy for both sides, whatever it is ?

> it now seems Russ doesn't want to do it.

He said something in the general direction of "if it ain't
broken don't fix it", but a NO-WS-CTL in Message-IDs is not
only broken = forbidden by RfC 1036, it's IMNSHO ridiculous.

     NOTE:  News message IDs are a restricted subset of
     MAIL message IDs.  In particular, no existing news
     software  copes properly with MAIL quoting conven-
     tions within the local part, so they  are  forbid-
     den.

If that was true 1994, why do you think that it's wrong now ?
Is there any evidence that s-o-1036 is completely obsolete, a
statistics about Message-ID syntax in news after 2822 maybe ?

                            Bye, Frank

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: Does 2231 break MIME 1.0?
Date: Wed, 1 Dec 2004 12:39:48 -0500
Cc: [email-masked]
Sender: [email-masked]

On Wed December 1 2004 00:13, Frank Ellermann wrote:

> Sorry, I don't have the C source for this patch, but it's a
> bad sign that you had to change something.  The patch does
> not show how you handle any duplicated resp. missing pieces.

The change was solely to print the (reassembled) parameters;
normally a parameter is simply used for its defined purpose.

> It's a PITA if all you want is to find the next MIME part,
> because that's not the precise moment where you want any:
> 
> boundary*1="pa\\rt " ; dup*1="two" ; mia*0="missing" ;
> dup*0=one ; mia*3=action ; dup*1=dup ; boundary*0=multi ;
> 
> Just in case, maybe you want more weird test cases ;-)

Do you mean something like the example that I posted to
this WG mailing list on 19 Mar 2002? :-) That contained:

Content-type: multipart/mixed;
  boundary*0* =''sim ;
  boundary*1* = ple%20
 ;
  boundary*2* = boundary
 
> > Insignificance of order, as Ned explained in the ietf-822
> > message that I cited earlier.
> 
> "Insignificance of order" and "unique" are quite different.
> 
> The latter is a "set" (any item is either IN or OUT), the
> former is a "bag" (IIRC, please correct me if I'm wrong).

Repetition ("foo=x ;foo=x") is not likely to be a problem.
Because order is insignificant (and always has been),
"foo=x ; foo=y" is a problem, because there is no way
to  determine the value associated with the attribute
foo.  One cannot depend on order, because order is
insignificant (changing order shouldn't change the
semantics).  That's why Ned wrote:

"It may not be explicitly stated that duplicate parameters aren't allowed, but
given that parameter order is required not to be significant (RFC 2046 section
1) is is hard to come up with any other result that makes sense."

> That's too substantial for the "errata".  Now give the poor
> authors a break, it's less than two weeks that you found
> this bug.

I've never claimed that there's a "bug" associated
with continuation parameters; that's your claim,
but I don't think you've convinced anybody.

To: LIST: [email-masked];
Xref: clerew local.usefor:20369
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: msg-id
Date: Wed, 1 Dec 2004 12:58:01 GMT
Lines: 68
Sender: [email-masked]

> [id-right]
>> there are other ways that can be done. Look at Agent, which
>> routinely uses the same id-right for everybody.

>AFAIK they use a domain reserved for this purpose, within the
>syntax of 2821, no problem for other domains.

Yes, but in order to ensure that the id-left is unique, they rely on
generating a random number long enough that the probability of getting an
accidental duplicate is reduced to once per thousand years, or somesuch. I
have no problem with that as an adequate implementation technique.

>Message-IDs are essential, we're supposed to get this right,
>it's no fight with RfC 2822bis, let them fix their own texts.

>BTW, where is this mythical 2822bis ?  Google doesn't find a
>2822bis at ietf.org, there's no active WG listed at ietf.org,
>any WG even remotely related to mail or MIME is apparently
>closed, we're the last survivors.

It never got beyond some preliminary discussions on the ietf-822 mailing
list.

>> The IETF is too packed with people who believe that "The
>> Internet Message Format" is some inviolable edifice that we
>> "News" people are refusing to bow down before

>Now you're confusing Bruce with "the IETF", and he finds all
>bugs, we're lucky to have him.  He finds even bugs he doesn't
>want to find. ;-)

>And you're listed in the RfC 2822 credits, together with Russ,
>Alexej, Paul, and Graham, no conspiracy to screw-up net news.

Hey! I never noticed that!

Yes, we found some problems with <msg-id) which they fixed. We found some
further problems just as they were going to last call, and we found some
more after that. As you say, the basic concept of the RFC 2822 <msg-id> is
broken, but there it is.

>> But if the WG really wants to follow that route ..........

>...as long as you restrict 2822 constructs to a proper subset
>you're free to do "the right thing".  Back to Message-IDs, you
>have:

Yes, there are lots of ways it might be fixed, but I don't want to do it,
and it now seems Russ doesn't want to do it. And without some signs of a
consensus to do it it isn't going to happen.

But if people have other views to express, then let us hear them.

To: LIST: [email-masked];
Xref: clerew local.usefor:20368
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Does 2231 break MIME 1.0?
Date: Wed, 1 Dec 2004 12:37:39 GMT
Lines: 60
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>We could, provided we provide a clear and complete specification.
>We should consider liklihood of implementation.  It has been
>claimed (unconvincingly) that because there may exist parsing
>of MIME parameters for Content-Type and Content-Disposition
>fields in some software, parsing of such parameters in other
>fields will magically happen as well.

You weren't convinced. I don't recall anyone else agreeing with you.

But in any case, there is never any need for any regular news agent to
parse the Injection-Info header to that level of detail. For injection
agents, a cursory examination to ensure that it does no gross violation of
the RFC 2045 syntax is quite sufficient.

The Injection-Info header is provided for sysadmins, geeks, spam fighters
and the like who will want to examine it only when something appears to
have gone wrong (e.g. so that they can generate abuse complaints in the
particular cases under discussion).

Usually, they will just read it. Maybe they will cut and paste bits out of
it (IP addresses, URLS, etc). If they are really keen, they will write
scripts to assist their tasks, though I doubt few of them will bother to
build full RFC 2231 capability into their scripts - simpler to disentangle
them by hand in the rare cases they will appear.

>RFC 2369 handles URIs (per RFC 1738 syntax, compatible
>with 2396 syntax), but is incompletely specified (no ABNF!).
>If we wish to use multiple URIs in a separate field similar to
>the List- fields described in RFC 2369, we could use that
>mechanism, but we should completely specify it including
>ABNF.  That's complicated by incompatible changes in the
>URI syntax draft which will probably be approved before
>we get around to a specification.

>> They simply say "internal whitespace being ignored" in 2369.

>RFC 2369 also says "MUST NOT insert whitespace within the
>brackets".  Ignoring whitespace when parsing is apparently
>intended to cope with mangling during transit, not as a
>mechanism for deliberately folding long URIs when generating
>a field.  RFC 2369 completely ignores (the common case of)
>long URIs.

Ah! So implementations that "ignore internal whitespace" like the RFC says
must first check whether the mangling occurred in transit and not do it
otherwise. How clever and sensible! NOT.

Date: Wed, 01 Dec 2004 07:22:42 -0800 (PST)
From: [email-masked]
Subject: Re: Does 2231 break MIME 1.0?
To: [email-masked]
Cc: [email-masked]
Sender: [email-masked]

> Frank Ellermann <[email-masked]> schrieb/wrote:
> > What "other considerations" ?  The author saying that he had something
> > like an environment in mind explains why he forgot to mention it
> > explicitly.  But we've already seen that it could make sense, e.g. if
> > I'd say...

> > charset=windows-1252 ; charset=iso-8859-1

> > ...it could stand for "I didn't use any 128..159, it works
> > with whatever you have,...

> You could still do this:

> Content-Type: text/plain; charset=iso-8859-1; alt-charsets*1=" iso-8859"
>   alt-charsets*0*=''iso-8859-1; alt-charsets*2="-15 iso-8859-2 windows-"
>   alt-charsets*3=1252

> But why, if you just label your content with the most common charset;
> e.g. ISO-8859-1.

> > and if you really insist on it I'm ready to accept replies with
> > windows-1252".

> No. That would be a very bad idea.

The understatement of the year arrives just in time for Christmas ;-)

				Ned

To: [email-masked]
From: [email-masked] (=?ISO-8859-1?Q?Claus_F=E4rber?=)
Subject: Re: Does 2231 break MIME 1.0?
Date: 01 Dec 2004 12:41:00 +0100
Lines: 29
Sender: [email-masked]

Frank Ellermann <[email-masked]> schrieb/wrote:
> What "other considerations" ?  The author saying that he had something
> like an environment in mind explains why he forgot to mention it
> explicitly.  But we've already seen that it could make sense, e.g. if
> I'd say...

> charset=windows-1252 ; charset=iso-8859-1

> ...it could stand for "I didn't use any 128..159, it works
> with whatever you have,...

You could still do this:

Content-Type: text/plain; charset=iso-8859-1; alt-charsets*1=" iso-8859"
  alt-charsets*0*=''iso-8859-1; alt-charsets*2="-15 iso-8859-2 windows-"
  alt-charsets*3=1252

But why, if you just label your content with the most common charset;  
e.g. ISO-8859-1.

> and if you really insist on it I'm ready to accept replies with
> windows-1252".

No. That would be a very bad idea.

Claus
-- 
http://www.faerber.muc.de