Subject: #1047 permitted constructs - a list

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 25 Jan 2006 18:08:17 -0800

At the moment, here's what I have as what "we" want as a description of 
Path - in English, because Frank's far better at writing (and 
understanding!) ABNF than I am:

- A path is a sequence of path-elements ending in a tail-element
- A path-element can be:

* path-identity followed by !
* path-identity followed by !!
* path-identity followed by path-diagnostic, followed by !

- A path-identity can be a domainname (a list of things with dots between 
them, where the last one is not all-numeric), or a bareword (a thing that 
doesn't contain dots). Details matter.

- A path-diagnostic can be:
* a "." followed by a keyword, followed by a separator (another "."?), 
followed by more information - structure defined by keyword, and not 
described in the USEFOR document. ":" permitted, to allow for IPv6 addresses
* An IPv4 address (for backwards compatibility) (????not sure????)

(the nice thing about !.diagnostic is that you only have to examine the 
first character of the diagnostic to know whether it is one or not...)

Does that capture roughly where people have been going, without introducing 
any ambiguous parses that it doesn't need to?

                   Harald

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 26 Jan 2006 12:10:32 +0100

> - A path is a sequence of path-elements ending in a

Okay, s/path-entry/path-element/ 

> * path-identity followed by path-diagnostic, followed by !

Okay.  IIRC Richard mentioned legacy cases of "!" *WSP "!",
but we can probably ignore this.  Some months ago Charles
proposed to have one or more identities in a <path-element>
for servers wishing to note more than one path-identity.

With the concept "path-element" = "everything one server
adds" that's syntactically possible.  But in practice a
server could just add more than one <path-element>, and
bypass all artificial limits for a single <path-element>.

> - A path-identity can be a domainname (a list of things
> with dots between them, where the last one is not
> all-numeric),

Yes, <toplabel> excludes <IPv4address>

> or a bareword (a thing that doesn't contain dots).
> Details matter.

At this point you've no other legacy contructs.  Things that
do contain dots like <IPv4address>, under_score.example, or
bare_word.MISMATCH (= <old-path-diagnostic> for <bareword> ).

> * a "." followed by a keyword, followed by a separator
> (another "."?), followed by more information

Yes, and apparently "we" want "." <keyword> "." resulting
in 'invisible' diagnostics from the POV of s-o-1036 servers.

Add usual 1036-blurb about .MISMATCH.::dead:feed in USEPRO,
with a hint about the colon in appendix B (missing in -06).

> - structure defined by keyword, and not described in the
> USEFOR document.

I'm not convinced that that's a good idea.  The diagnostical
value is already small, and a clear list POSTED / MISMATCH /
SEEN near to "!!" (for the MATCH case) is a good thing.

Stuff that I don't want to see as "syntactically valid": 

.LOCATION.cafe.jail.airport      (see geopriv last call)
.MOTD.there.is.no.such.thing.as.a.free.server
.INFO.processed.by.super-trooper
.URL.www.super-trooper.com
.ADV.get.a.super-trooper.for.your.inn

Such crap just doesn't belong into the path, and we're not
planning to create a <path-keyword> IANA registry.  

Admittedly a fixed list of <path-keyword>s could never be
extended, or rather not in a backwards compatible way. 

But in that case I'd say "so what", some already doubt that
the fixed list does much more than waste band-width.  Maybe
POSTED is less controversial.

> ":" permitted, to allow for IPv6 addresses

Yes, see above.  And maybe also "_" for a <path-diagnostic>
about <bareword> identities.

> * An IPv4 address (for backwards compatibility) (????not

Somewhere you have to cover this case.  Apparently you want
IPv4 only as <old-path-diagnostic> and never as <path-legacy>
identity.
  
If we add bare_word.MISMATCH or under_score.example here as
<o-p-d> catch-all we're not forced to mention IPv4 explicitly:
IPv4 just "happens" to match <o-p-d>.

                       Bye, Frank

P.S.:  An attempt to catch your list keeping fixed keywords.

       <path-legacy> renamed to <path-nodot>, <o-p-d> renamed
       to <diag-deprecated>, <letdig> renamed to <alphanum>,
       <tail-entry> allowing dots.  Bill's parser says okay.

; ------------------------------------------------------------------#

path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF
path-list       =  1*( path-identity [ path-diagnostic ] "!" [FWS] )

path-diagnostic =  diag-match / diag-posted / diag-seen /
                   diag-mismatch / diag-deprecated

diag-match      =  "!"                           ; an additional "!"
diag-posted     =  "!.POSTED"
diag-seen       =  "!.SEEN."     ( path-identity / diag-literal )
diag-mismatch   =  "!.MISMATCH." ( path-identity / diag-literal )
diag-deprecated =  "!" *( path-nodot "." ) path-nodot

diag-literal    =  IPv4address / IPv6address

path-identity   =  ( 1*( label "." ) toplabel ) / path-nodot
path-nodot      =  1*( alphanum / "-" / "_" )

tail-entry      =  *( path-nodot "." ) path-nodot

label           =  alphanum [ *( alphanum / "-" ) alphanum ]
alphanum        =  ALPHA / DIGIT                 ; compare RFC 3696

toplabel        =  ( [ label *( "-" ) ] ALPHA   *( "-" ) label   ) /
                   (   label *( "-" )   ALPHA [ *( "-" ) label ] ) /
                   (   label          1*( "-" )          label   )

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 26 Jan 2006 22:27:46 +0100

Sorry...

> diag-deprecated =  "!" *( path-nodot "." ) path-nodot

...that matches !demon, of course at least one dot here:

| diag-deprecated =  "!" 1*( path-nodot "." ) path-nodot

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 30 Jan 2006 12:39:01 GMT

>.ADV.get.a.super-trooper.for.your.inn

Stupid, but not particularly harmful.

I could live with either a hard-coded list of keywords in USEFOR, or a
generic keyword plus detailsm in USEPRO, as Harald suggests. A mild
preference for the latter, perhaps.

>       <tail-entry> allowing dots.  Bill's parser says okay.

Yes, but you have still got the [FWS] on the wrong side of the "!".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 30 Jan 2006 12:26:05 GMT

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>At the moment, here's what I have as what "we" want as a description of=20
>Path - in English, because Frank's far better at writing (and=20

>* path-identity followed by path-diagnostic, followed by !

You may also want
 * path-identity followed by !! followed by path-diagnostic, followed by !
That is for cases like ...!news.injector.net!!.POSTED!xyzzy!not-for-mail
(assuming .POSTED is a path-diagnostic and not a separate syntactic
opbject is its own right).

In that example, news.injector.net asserts that he was the injector, and
had verified that it was received from a host 'xyzzy'. That (or its
equivalent) is permitted by the current draft. OTOH, if you don't want
that feature, then it is easily left out. I gather Frank doesn't want it.

>- A path-identity can be a domainname (a list of things with dots between=20
>them, where the last one is not all-numeric), or a bareword (a thing that=20

Agreed. In particular, a <bareword> is _not_ a legacy thing to be
deprecated.

>* a "." followed by a keyword, followed by a separator (another "."?),=20
>followed by more information - structure defined by keyword, and not=20
>described in the USEFOR document. ":" permitted, to allow for IPv6 =
>addresses

I think it needs to be made clear that ":" is _only_ to be allowed for
IPv6 addresses (otherwise the dead:beef problem becomes much worse).

And probably simpler to classify a naked IPv4 address as a deprecated
obs-path-identity.

Also, the stuff following the keyword needs to be optional, so as to allow
...!.POSTED!... . With a sufficiently complicated syntax, you could make
it apply _only_ to that case, but such details might be better left to
USEPRO. Look at is this way. A path-diagnostic starts with a <keyword>,
which makes some assertion, followed by an optional argument which
provides evidence (where needed) for that assertion. Gory details in
USEPRO.

>any ambiguous parses that it doesn't need to?

More or less. Main thing left to decide is whether the full list of
keywords is hard-coded into USEFOR, or whether you just provide a generic
keyword syntax and let USEPRO introduce what is currently required (easier
for future extensions, perhaps, and less for hard-nosed parsers to worry
about).

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 30 Jan 2006 19:35:13 +0100

Charles Lindsey wrote:
 
> you have still got the [FWS] on the wrong side of the "!".

As far as I'm concerned it's the "right side" (pun intended),
otherwise it does horrible things with the "!!" syntax.  Bye

---

From: Russ Allbery <[email-masked]>
Date: Mon, 30 Jan 2006 10:47:18 -0800

> path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF

Should tail-entry really be mandatory?  It makes the grammar easier, but
that's not existing practice.

> path-list       =  1*( path-identity [ path-diagnostic ] "!" [FWS] )

path-list       = 1*( path-identity [FWS] path-separator )
path-separator  = [ path-diagnostic ] "!"

gets the FWS on the other side of the "!", which I think makes the Path
header a bit more readable.  Not a big deal, though.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 30 Jan 2006 23:03:21 +0100

Russ Allbery wrote:
 
> Should tail-entry really be mandatory?

What do you reckon ?  Historically it was in s-o-1036 and in
bang paths before 1036, but in 1036 it was only "permissible".

> It makes the grammar easier

Not much, to fix it we'd have to move "!" in front of the
<tail-entry> or any <path-element> except from the first.

> gets the FWS on the other side of the "!", which I think
> makes the Path header a bit more readable.  Not a big deal

A very big deal:  Charles and I quibble about this detail for
months.  Almost "heureka".  Could we have something like this:

 tail-entry       = path-identity [ "!" tail-local ]
 tail-local       = *( path-nodot "." ) path-nodot

In other words it's either an identity, or an identity plus a
local part, no [FWS] at this point please, [FWS] makes me mad.

Adding your path-list after removing the then unnecesseary "1":

  path-list       = *( path-identity [FWS] path-separator )
  path-separator  = [ path-diagnostic ] "!"

IMO not really "better readable" if the complete diagnostics
is on another line.  But that's only a matter of taste, if the
"!!" comes in one piece anywhere I'm happy with it.

Harald should call out "rough consensus" on this part of the
path before we screw it up again.  

Piecemail, 8 of up to 15 path productions clear modulo #1179.
  
For a 9th production only the name <path-nodot> with MUSTard
vs. <path-legacy> without MUSTard is unclear, the remaining 6
are <path-diagnostic> and <diag-*> including <diag-deprecated>.

                          Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 30 Jan 2006 23:19:05 +0100

> Piecemail

Piecemeal.

---

From: Russ Allbery <[email-masked]>
Date: Mon, 30 Jan 2006 14:43:14 -0800

> local part, no [FWS] at this point please, [FWS] makes me mad.

That works for me, and makes a lot of sense to me.

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 31 Jan 2006 11:55:29 GMT

>otherwise it does horrible things with the "!!" syntax.  Bye

I have already exhibited a syntax which puts the [FWS] on the left of the
"!" *without* doing "horrible things" such as allowing [FWS] in the middle
of the "!!".

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 31 Jan 2006 12:12:16 GMT

>that's not existing practice.

My understanding is that existing practice is for relayers to ignore
whatever comes after the last "!" (typically "not-for-mail") when deciding
whether not to send the article back to some site listed in the Path. The
reason being, of course, that historically that last entry respresented a
person rather than a site (still does, sometimes).

Therefore, a <tail-entry> is needed in order to be able to ignore it. OK,
I have no problem with a Path ending with a "!"
   Path: site3!site2!site1!
but that just means the <tail-entry> would be allowed to be <empty>.
However, common practice is to put _something_ there, so we must continue
to allow that.

>header a bit more readable.  Not a big deal, though.

Eh? You told us once that you preferred continuation lines to begin with a
"!", as in

    Path: news.site4.exmaple!news.site3.exmaple!news.site2.example
          !news.site1.example!not-for-mail

rather than splitting the line _after_ a "!" at the end of the previous
line. On the basis of that, I have always tried to make my syntax that way
round, and resisted Frank's attempts to make it the other way.

Please conmfirm which you prefer. Granted it is no huge deal, but if there
is a preferred way to do it, then we might as well do so, as is is
perfectly easy to write the syntax either way.

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 31 Jan 2006 15:11:46 +0100

Charles Lindsey wrote:

> I have already exhibited a syntax which puts the [FWS] on the
> left of the "!" *without* doing "horrible things" such as
> allowing [FWS] in the middle of the "!!".

The other horrors were "!!.POSTED!" instead of "!.POSTED!" etc.

Russ fixed it yesterday, fold _before_ the <path-diagnostics>
and use a 1036-compatible <tail-entry>, modulo #1179 that is:

1 path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF
2 path-list       = *( path-identity [FWS] path-separator )
3 path-separator  = [ path-diagnostic ] "!"

4 path-identity   =  ( 1*( label "." ) toplabel ) / path-nodot
5 path-nodot      =  1*( alphanum / "-" / "_" )

6 tail-entry       = path-identity [ "!" tail-local ]
7 tail-local       = *( path-nodot "." ) path-nodot

8 label           =  alphanum [ *( alphanum / "-" ) alphanum ]
9 alphanum        =  ALPHA / DIGIT                 ; compare RFC 3696

A toplabel        =  <TBD, see other articles>
B path-diagnostic =  <TBD, see other articles>

Ad 1: *WSP discussed in #1179, otherwise clear.
Ad 2: Russ' idea moving [FWS] before the optional (3) plus "!".

Ad 5: <path-legacy> without additional MUSTard might be better.

Ad 6: Strict 1036-compatibility, "!not-for-mail" permissible.
Ad 7: No "!" at the right end.  Some dots in local part okay.

Ad A: Almost clear, needed to reject IPv4 as <path-identity>.
Ad B: Discussed elsewhere, JFTR I've added my proposal below.

Are we clear so far with #1047 ?  How about using "!!" for the
"!.POSTED!" case instead of the dubious "!.MATCH!" case ?  Bye

diag-match      =  "!"                           ; an additional "!"
diag-posted     =  "!.POSTED"
diag-seen       =  "!.SEEN."     ( path-identity / diag-literal )
diag-mismatch   =  "!.MISMATCH." ( path-identity / diag-literal )
diag-deprecated =  "!" 1*( path-nodot "." ) path-nodot

diag-literal    =  IPv4address / IPv6address     ; see STD 66

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 31 Jan 2006 12:30:41 GMT

> tail-local       = *( path-nodot "." ) path-nodot

No that is quite wrong. The <tail-entry> is the bit that relayers are
supposed to ignore when deciding where not to forward the article to. So
the one thing you must not have in the <tail-entry> is a "!" (or anything
that some system might treat as a delimiter, such as a ":"), for otherwise
the relayer cannot recognize which bit it is suppoed to be ignoring.

>local part, no [FWS] at this point please, [FWS] makes me mad.

I don't really care what is in it, provided there is nothing that might be
taken as a delimiter. So we write the simplest syntax possible that does
not preclude any existing practice. Does one see those dots in existing
practice?

S-o-1036 uses <local-part>, but that would allow
   ...!site2!site1!"JoeDoe!yadda"
which would surely confuse the hell out of some relayers. So do not allow
that unless you can point to real-world examples of it.

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 31 Jan 2006 18:34:21 +0100

> No that is quite wrong. The <tail-entry> is the bit that
> relayers are supposed to ignore when deciding where not
> to forward the article to.

Right or wrong, it's strict 1036, and Russ said that paths
without local-part at the end are common practice.  If that
works somehow today, it will continue to work tomorrow.

Besides we want  "!.POSTED!" <tail-entry>  If it's only a
question of the _name_ <tail-entry> (because you have prose
about it in USEPRO) we could simply rename these contructs:

s/tail-entry/path-injector/ 
s/tail-local/not-for-mail/

But a <path-injector> as first <path-identity> makes sense
if you look into 1036.  And if you insist on a non-optional
<not-for-mail> that would be another point for appendix B,
and Russ said it's wrong.  

Fight it out, removing the square brackets for <not-for-mail>
or <tail-local> or whatever if you and s-o-1036 "win" is easy.

                            Bye, Frank

---

From: Russ Allbery <[email-masked]>
Date: Tue, 31 Jan 2006 10:33:19 -0800

Charles Lindsey <[email-masked]> writes:

> whatever comes after the last "!" (typically "not-for-mail") when
> deciding whether not to send the article back to some site listed in the
> Path. The reason being, of course, that historically that last entry
> respresented a person rather than a site (still does, sometimes).

INN certainly doesn't do this.  I don't know what other servers you may
have been looking at to judge existing practice.

> line. On the basis of that, I have always tried to make my syntax that
> way round, and resisted Frank's attempts to make it the other way.

Yes, that's what the above does.  That's the whole point of the change.

---

From: Russ Allbery <[email-masked]>
Date: Tue, 31 Jan 2006 10:34:33 -0800

> works somehow today, it will continue to work tomorrow.

I wouldn't say that it's *common*, just that it happens and in practice a
tail part is not required currently.

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 01 Feb 2006 00:10:37 +0100

Russ Allbery wrote:

> INN certainly doesn't do this.

So you do look at the 1036-permissible <tail-local> (in want
of a better name) treating it like the first <path-identity>.

That could cause havoc for some permissible local-parts, e.g.
for the legendary "demon", but actually for everything that
matches any existing <path-identity>.

How do you avoid that ?  Should we mention that using a common
"not-for-mail" is a SHOULD on the border to MUST, best excuse
to violate it would be no <tail-local> at all ?  Mention it in
USEFOR ?  Ordinary users and leaf-nodes have no reason to look
into USEPRO when they configure whatever <tail-local>.

---

Not exactly related rant:  This path-issue puzzles me.  A Path
is almost as essential as Message-ID, Newsgroups, and Control
for NetNews.  What did the earlier incarnations of this WG do,
if some of these essential UseNet concepts still aren't clear ?

I know when "we" talked about "Re: the note between Do and Mi"
for at least one year, with its "Subject: cmsg" part surviving
in #1032, and it took us months to fix the 2822 <msg-id> to a
certain degree.  And now we found that "we" didn't have a clear
concept for the <tail-entry> until today, astonishing :-(  Bye

---

From: Russ Allbery <[email-masked]>
Date: Tue, 31 Jan 2006 15:31:22 -0800

> of a better name) treating it like the first <path-identity>.

Yeah, INN just breaks the Path header apart on ! and uses all of them.

> into USEPRO when they configure whatever <tail-local>.

I think it's probably at least worth a warning to not put stuff into the
tail-local that you don't want to be used for the Path algorithm.

---

From: Richard Clayton <[email-masked]>
Date: Wed, 1 Feb 2006 00:15:39 +0000

In message <[email-masked]>, Frank Ellermann
<[email-masked]> writes

>into USEPRO when they configure whatever <tail-local>.

systems will still create tails like

        !highwayman.com!richard

since that was the way you could write back to [email-masked]
(back when you needed more text to the left of this bang path as well to
reach the site)  [AIUI, I didn't use Usenet that long ago]

hence "not-for-mail" to stress not using it this way :)

There's no need for a SHOULD here (making existing working clients only
"conditionally compliant") merely for specious tidyness.

Usenet works really well with people putting email local parts at the
end of Paths ... they don't in practice clash with either UUCP names or
machine/domain names -- leastwise if they do, no-one notices

>if some of these essential UseNet concepts still aren't clear ?

invent things that no-one wants to implement :(

- -- 
richard                                                   Richard Clayton

Those who would give up essential Liberty, to purchase a little temporary 
Safety, deserve neither Liberty nor Safety. Benjamin Franklin 11 Nov 1755

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 01 Feb 2006 05:03:19 +0100

> since that was the way you could write back to

Yes.  But it's a case of "no news is bad news" for any system
styling itself with <path-legacy> name "richard".  Can we now
please stop to pretend that all legacy names are completely
harmless ?

> I didn't use Usenet that long ago

Nor me, I learned about s-o-1036 and gateway issues in an FTN
echo (Fido Technology Network, zone 21, not the core FidoNet).

Before that all I knew was "Emily Postnews" (IMHO still better
than all other "netiquette" texts including the obscure RfC ;-)

> There's no need for a SHOULD here (making existing working
> clients only "conditionally compliant") merely for specious
> tidyness.

IBTD, as Russ explained it a "local part" news.clara.net would
be a horrible idea, and users need to know that it's horrible.

> Usenet works really well with people putting email local
> parts at the end of Paths ... they don't in practice clash
> with either UUCP names or machine/domain names -- leastwise
> if they do, no-one notices

Exactly the latter is the problem:  noone notices.  If they'd
do it _intentionally_ to exclude news.clara.net let them play.

But if they don't know what they are doing it's a trap.  It's
the purpose of a standard to prevent foreseeable harm as far as
possible, and "SHOULD NOT use <tail-local>, or where required
use a dummy not-for-mail" could help.  

Those insisting on doing it their way hopefully find the fine
print somewhere deep within USEPRO or NNTP or an INN manual.

That's one meaning of SHOULD, "know what you're doing if you
ignore it, it may cause harm".
                               Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 permitted constructs - a list
Date:  Wed, 01 Feb 2006 05:03:19 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 49
Sender: [email-masked]

 
 
> since that was the way you could write back to

Yes.  But it's a case of "no news is bad news" for any system
styling itself with <path-legacy> name "richard".  Can we now
please stop to pretend that all legacy names are completely
harmless ?

> I didn't use Usenet that long ago

Nor me, I learned about s-o-1036 and gateway issues in an FTN
echo (Fido Technology Network, zone 21, not the core FidoNet).

Before that all I knew was "Emily Postnews" (IMHO still better
than all other "netiquette" texts including the obscure RfC ;-)

> There's no need for a SHOULD here (making existing working
> clients only "conditionally compliant") merely for specious
> tidyness.

IBTD, as Russ explained it a "local part" news.clara.net would
be a horrible idea, and users need to know that it's horrible.

> Usenet works really well with people putting email local
> parts at the end of Paths ... they don't in practice clash
> with either UUCP names or machine/domain names -- leastwise
> if they do, no-one notices

Exactly the latter is the problem:  noone notices.  If they'd
do it _intentionally_ to exclude news.clara.net let them play.

But if they don't know what they are doing it's a trap.  It's
the purpose of a standard to prevent foreseeable harm as far as
possible, and "SHOULD NOT use <tail-local>, or where required
use a dummy not-for-mail" could help.  

Those insisting on doing it their way hopefully find the fine
print somewhere deep within USEPRO or NNTP or an INN manual.

That's one meaning of SHOULD, "know what you're doing if you
ignore it, it may cause harm".
                               Bye, Frank

Date: Wed, 1 Feb 2006 00:15:39 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1047 permitted constructs - a list
Sender: [email-masked]

        !highwayman.com!richard

- -- 
richard                                                   Richard Clayton

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1047 permitted constructs - a list
Organization: The Eyrie
Date: Tue, 31 Jan 2006 15:31:22 -0800
Sender: [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 permitted constructs - a list
Date:  Wed, 01 Feb 2006 00:10:37 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 31
Sender: [email-masked]

---

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: POSTED-1 vs. POSTED-2
Date:  Tue, 31 Jan 2006 19:47:44 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 116
Sender: [email-masked]

Charles Lindsey wrote:

> on reading more carefully what is written in USEPRO, the case
> we actually need (and which has been there all along) is

> ...!news.injector.example!.POSTED!!xyzzy.claranet.de!frank

Okay, ignoring the question why and since when "we" need this,
let's say that's the 1st case ...!.POSTED-1!xyzzy.claranet.de

> there is still some existing usage of the form:
>     ...!news.injector.example!123.234.345.456!not-for-mail

Yes, any IPv4 is covered by <diag-deprecated> in my ABNF for
Harald's idea.  In an older proposal for Russ' idea it would
be simply a <path-legacy> construct with dots, but you hated
this.  That's the <o-p-d> (Harald) vs. <o-p-i> (you) conflict.

With <o-p-d> and my proposal we'd end up with ...!not-for-mail
as a <path-nodot> case of <path-identity>.  Very misleading if
we don't rename <path-nodot> to <path-legacy>.

> Under our new system, that would logically be rendered as
> ...!news.injector.example!.POSTED!.SEEN.123.234.345.456!not-for-mail

So that's your 2nd case of POSTED or POSTED-2, resulting in
...!.POSTED-2.123.234.345.456!not-for-mail for this bogus IPv4.

In other words POSTED-1 is a standalone keyword, and POSTED-2
comes with an argument <diagnostic-identity>.

Therefore we don't need two different POSTED-1 and POSTED-2,
one name POSTED with an optional <diagnostic-identity> will do:

diag-match      =  "!"                           ; an additional "!"
diag-posted     =  "!.POSTED" [ "." diag-identity ]
diag-seen       =  "!.SEEN."        diag-identity
diag-mismatch   =  "!.MISMATCH."    diag-identity
diag-deprecated =  "!" 1*( path-nodot "." ) path-nodot

diag-identity   =  path-identity / IPv4address / IPv6adress

Killing my <diag-literal>, your <diag-identity> is IMO clearer.

Ideas about swapping "!.POSTED!" and "!.MATCH!" for "!!" also
obsolete, we need some miracles to get the <path> right without
any "!!" optimization.

> path = "Path" ":" SP [FWS]
>           *( path-identity [FWS] "!" *( path-diagnostic ) )
>           tail-entry [FWS] CRLF

> path-diagnostic = "." keyword [ "." diagnostic-identity ] [FWS] "!"
>                      / "!"

One [FWS] too many.  No "!.POSTED!.SEEN.ip".  No "!.POSTED!!y".

Apparently you want "!.POSTED.ip!" and "!.POSTED!y" for these
two purposes, that's okay.  As far as anything is still okay -
we can always give up and kill the complete <path-diagnostic>.

> now we need to decide whether we really need to allow two
> diagnostics in succession for this particular usage.

No, see above, optional <diag-identity> would be good enough.

>> We had some "no IP" rough consensus, therefore you can't
>> match it as ordinary <path-identity> together with FQDNs.

> The only agents that will ever "parse" the Path header in
> anger are relaying agents,

Yes, with your ABNF they match IPv4 as <o-p-i>, with my latest
ABNF they get <diag-deprecated> (= Harald's <o-p-d>), and for
Russ' _old_ proposal they would get a <path-legacy> with dots.

Harald's <o-p-d> tries to guarantee that they'd ignore IPv4.

> all they are ever going to do is to ignore any folding and
> whitespace, and then identify the chunks that come between
> the "!"s. If any such chunk appears in their 'sys' file, then
> they will not forward the article to that site. So if some
> admin is stupid enough to put ".POSTED" in his 'sys' file,
> he will never forward any articles anywhere :-( .

Educating stupid admins is the job of USEPRO, let's ignore this
for the moment.  Implementing a parser that won't consider any
<path-diagnostic> (incl. <o-p-d-> matching IPv4) as "chunk" is
the job of Russ' and others.  It's possible to get this right.

Otherwise we shouldn't waste time with this dubious diagnostics
concept.  Or as Richard put it "YARITWCIFD" = "Yet Another Rfc
Invention That Will Confuse Implementors For Decades."

> the question of whether some IPv4address that appears without
> any keyword in front of it is to be classed syntactically as
> an obsolete <path-identity> or as a <path-diagnostic> is
> purely academic. Nobody cares.

We had a poll about it, some of us do care.  They don't want to
allow IPv4.  Maybe because they'd then get in trouble trying to
rule out IPv6.  And I fear (or is that hope ?) that somebody in
the IESG won't let us get away with any "for IPv4 only" stunts.

> We don't particularly want any <obs-path-identity> in our
> <diagnostic-identity>s.

That "we" is dubious, it's the conflict between <o-p-d> and
<o-p-i>.  I've no problem with any legacy "demon" or "IPv4" as
<diag-identity>.

Well, tons of new issues, it's starting to get an interesting
job for Harald, Ken, and Alexey.

                         Bye, Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1047 permitted constructs - a list
Organization: The Eyrie
Date: Tue, 31 Jan 2006 10:34:33 -0800
Sender: [email-masked]

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1047 permitted constructs - a list
Organization: The Eyrie
Date: Tue, 31 Jan 2006 10:33:19 -0800
Sender: [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 permitted constructs - a list
Date:  Tue, 31 Jan 2006 18:34:21 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 30
Sender: [email-masked]

 

                            Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:23063
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: POSTED-1 vs. POSTED-2 (was: path-legacy (bareword) and toplabel in A ~ B)
Date: Tue, 31 Jan 2006 15:34:44 GMT
Lines: 88
Sender: [email-masked]

>> the "y!!.POSTED!x" was intended

>What's the difference from "y!.POSTED!x" ?  AFAIK that's
>the first time that you mention this as an _intentional_
>feature, no surprise if we arrive at different ABNF.

OK, on reading more carefully what is written in USEPRO, the case we
actually need (and which has been there all along) is

    ...!news.injector.example!.POSTED!!xyzzy.claranet.de!frank

The injector there is asserting
   a) that it is the injecting agent, and
   b) that it indeed received the article from the site xyzzy.claranet.de.

So I was trying to put the "!!" on the wrong side of the ".POSTED", but it
still amounts to two diagnostics in succession. And there is still some
existing usage of the form:

    ...!news.injector.example!123.234.345.456!not-for-mail

where the "123.234.345.456" is what the injector SAW (and not what the
posting site put there). Under our new system, that would logically be
rendered as

    ...!news.injector.example!.POSTED!.SEEN.123.234.345.456!not-for-mail

On that basis, I have now modified my syntax to:

   path = "Path" ":" SP [FWS]
             *( path-identity [FWS] "!" *( path-diagnostic ) )
             tail-entry [FWS] CRLF

   path-diagnostic = "." keyword [ "." diagnostic-identity ] [FWS] "!"
                     / "!"

which is also somewhat cleaner to follow.

But now we need to decide whether we really need to allow two diagnostics
in succession for this particular usage. One might argue that an injecting
site should be using Injection-Info if it wants to say where it got the
article from. In that case, 
   s/*( path-diagnostic )/[ path-diagnostic ]/
in the above.

>> I would much rather just allow the syntax to produce
>> 123.234.345.456 and then deprecate it with MUSTard

>We had some "no IP" rough consensus, therefore you can't
>match it as ordinary <path-identity> together with FQDNs.

The only agents that will ever "parse" the Path header in anger are
relaying agents, and all they are ever going to do is to ignore any
folding and whitespace, and then identify the chunks that come between the
"!"s. If any such chunk appears in their 'sys' file, then they will not
forward the article to that site. So if some admin is stupid enough to
put ".POSTED" in his 'sys' file, he will never forward any articles
anywhere :-( .

So the question of whether some IPv4address that appears without any
keyword in front of it is to be classed syntactically as an obsolete
<path-identity> or as a <path-diagnostic> is purely academic. Nobody
cares.

>  {diagnostic-identity = fqdn / IP-address / bareword]
>>> That won't allow <obs-path-identity> unless it's
>>> <IP-address>.

Indeed. We don't particularly want any <obs-path-identity> in our
<diagnostic-identity>s.

To: [email-masked]
Xref: clerew local.usefor:23062
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 permitted constructs - a list
Date: Tue, 31 Jan 2006 12:30:41 GMT
Lines: 37
Sender: [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 permitted constructs - a list
Date:  Tue, 31 Jan 2006 15:11:46 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 54
Sender: [email-masked]

To: [email-masked]
Xref: clerew local.usefor:23061
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 permitted constructs - a list
Date: Tue, 31 Jan 2006 12:12:16 GMT
Lines: 54
Sender: [email-masked]

To: [email-masked]
Xref: clerew local.usefor:23060
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 permitted constructs - a list
Date: Tue, 31 Jan 2006 11:55:29 GMT
Lines: 23
Sender: [email-masked]

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1047 permitted constructs - a list
Organization: The Eyrie
Date: Mon, 30 Jan 2006 14:43:14 -0800
Sender: [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  OT (was: #1047 permitted constructs - a list)
Date:  Mon, 30 Jan 2006 23:19:05 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 4
Sender: [email-masked]

Piecemeal.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 permitted constructs - a list
Date:  Mon, 30 Jan 2006 23:03:21 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 47
Sender: [email-masked]

  

                          Bye, Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1047 permitted constructs - a list
Organization: The Eyrie
Date: Mon, 30 Jan 2006 10:47:18 -0800
Sender: [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 permitted constructs - a list
Date:  Mon, 30 Jan 2006 19:35:13 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 7
Sender: [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  POSTED-1 vs. POSTED-2 (was: path-legacy (bareword) and toplabel in A ~ B)
Date:  Mon, 30 Jan 2006 19:21:46 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 75
Sender: [email-masked]

Charles Lindsey wrote:

> leave the <tail-entry> until the rest of the syntax is
> agreed, and then just build it using whatever tools are
> then available. Its only _essential_ property is that
> there is no "!" in it.

Yes, other VCHARs including ":" might be also "essential".
Not in my s-o-1036 parallel universe, but maybe for Bruce.

> the "y!!.POSTED!x" was intended

What's the difference from "y!.POSTED!x" ?  AFAIK that's
the first time that you mention this as an _intentional_
feature, no surprise if we arrive at different ABNF.

For an identical list that would be almost impossible as
soon as the list of intended Path:-features is complete.

 [limit [FWS] in the path]
> is preventing a [FWS] between a <path-identity> and a
> <path-diagnostic> really worth the complication?

It's pretty simple, not complex...

> Harald seems to be pressing for a simpler syntax

...not sure about this.  Anything where I see at first
glance what it's supposed to do would be "simple" from
my POV.  One earlier 2005 attempt with something like...

   path-sep = [FWS] 1*2( "!" ) [FWS]

...or similar was horrible, nobody liked it.  IMHO the
concept "!!" as shorthand for a non-existing "!.MATCH!"
made sense.  We could also swap !.MATCH! and !.POSTED!
if the match-case is irrelevant, and then "!!" would be
a shorthand for a non-existing "!.POSTED!".  <shrug />

> I would much rather just allow the syntax to produce
> 123.234.345.456 and then deprecate it with MUSTard

We had some "no IP" rough consensus, therefore you can't
match it as ordinary <path-identity> together with FQDNs.

In theory you could match it before FQDN like STD 66, and
then deprecate it.  Curious minds would immediately start
to [Discuss] or ask questions about IPv6 in this case.

  {diagnostic-identity = fqdn / IP-address / bareword]
>> That won't allow <obs-path-identity> unless it's
>> <IP-address>.
[...]
> Eh? The only thing in my <obs-path-identity> was
> <IPv4address>.

| obs-path-identity = IPv4address ; and any other cases thought necessary
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I wanted to point out that it won't work for any other cases
like legacy names with dots.  No issue if we agree that other
cases either don't exist or are irrelevant.

 [general]
> It depends whether ...!!.POSTED!... is to be allowed

Yes.  I vaguely recall that Bruce and I didn't like it, but if
it has a meaning different from !.POSTED! that would be a new
argument.

We could use "!.POSTED-1!" and "!.POSTED-2!" for two versions
of "!.POSTED!", but I've no clue what this difference is, and
therefore I can't propose better names for POSTED-1 / POSTED-2.

                          Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:23050
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Mon, 30 Jan 2006 14:21:03 GMT
Lines: 99
Sender: [email-masked]

>> tail-entry = path-identity

>My last proposal - <path-nodot> is a liberal <bareword> - was:

>| path-nodot      =  1*( alphanum / "-" / "_" )
>|
>| tail-entry      =  *( path-nodot "." ) path-nodot

I think we leave the <tail-entry> until the rest of the syntax is agreed,
and then just build it using whatever tools are then available. Its only
_essential_ property is that there is no "!" in it.

>>    path-diagnostic = [ "!" ]
>>                        [ "." keyword [ diagnostic-identity ] [FWS] "!" ]

>That allows  y!!.SEEN.u!x  or  y!!.POSTED!x  with a bogus "!!".

Yes, the "y!!.POSTED!x" was intended. If we decide we do not want it, it
is easily removed. And a more complicated syntax could clearly remove
other unintended funnies, but would not be so KISSable.

>And "we" want at most one [FWS] per <path-element>, you have
>two.  IIRC "limit [FWS]" was originally your idea.

Yes, but is preventing a [FWS] between a <path-identity> and a
<path-diagnostic> really worth the complication? Harald seems to be
pressing for a simpler syntax, which is why I removed that "feature". If
the WG really wants it tying down that hard, then it can be put back.

>>    keyword = ( "SEEN" / "MATCH" / "MISMATCH" / "POSTED" )

>That allows  y!.MATCH.x!x  (and  y!!.MATCH.x!x ), that's wrong,
>"we" want  y!!x  for this purpose.

If we are to have a fixed list of keywords, then I am happy to remove
"MATCH" from it (so long as "!!" remains possible).	

>>    label = alphanum / alphanum *( alphanum / "-" ) alphanum

>That doesn't have the 4234 recommended grouping.  My proposal:

>| label           =  alphanum [ *( alphanum / "-" ) alphanum

OK

>>    toplabel = [ label *( "-" ) ] ALPHA [ label *( "-" ) ]
>>               / label 1*( "-" ) label    ; at least one non-digit

>The same 4234 grouping issue for the hyphen-alternative.  And
>it's the "old" toplabel idea allowing a singleton ALPHA as TLD.

OK

>Maybe it's ridiculous, but I fear that whatever we pick here
>could sooner or later end up as 2606bis / ICANN policy for the
>whole world, and in that case I'd prefer "at least two char.s".

Indeed. It's not really our job to define what domain names look like. So
I would much rather just allow the syntax to produce 123.234.345.456 and
then deprecate it with MUSTard (all we are trying to say is "don't use an
IPv4address in here, but just accept it for legacy purposes", so why not
just say that?).

>> diagnostic-identity = fqdn / IP-address / bareword

>That won't allow <obs-path-identity> unless it's <IP-address>.
>In my proposal any <path-identity> or <IP-address> is allowed:

Eh? The only thing in my <obs-path-identity> was <IPv4address>. If <o-p-i>
acquires some more stuff, then <diagnostic-identity> might have to be
reviewed.

>> If you look at the <path-diagnostic>, you can't have more
>> than one of them at a time

>Almost, you still have "!!" together with other disagnostics,
>and you have <diagnostic-identity> where you don't want them,
>i.e. for MATCH (fixed by simply removing it) and POSTED (bug).

It depends whether ...!!.POSTED!... is to be allowed, and on whether you
really want every last detail to be hard coded in (KISS and all that).

To: [email-masked]
Xref: clerew local.usefor:23048
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 permitted constructs - a list
Date: Mon, 30 Jan 2006 12:26:05 GMT
Lines: 71
Sender: [email-masked]

You may also want

To: [email-masked]
Xref: clerew local.usefor:23051
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1177 Missing [CFWS] (was: ...ranting and maybe four missing [CFWS] ?!?)
Date: Mon, 30 Jan 2006 14:22:19 GMT
Lines: 26
Sender: [email-masked]

In <3BE23ACBCFC2FAE7714CC628@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>Ticket created.

>> JFTR:
>>
>>>> archive =3D  "Archive:" SP [CFWS] ("no" / "yes")
>>>>            *( [CFWS] ";" [CFWS] archive-param ) [CFWS] CRLF
>> {...]
>>> injection-info  =3D  "Injection-Info:" SP [CFWS] path-identity
>>>                    *([CFWS] ';' [CFWS] parameter) [CFWS] CRLF
>>
>> Plus Charles' proposed modification of the corresponding Note.

Fine by me.

To: [email-masked]
Xref: clerew local.usefor:23052
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1179 Re: ISSUE: replace twenty [FWS] by *WSP (was: Open issues)
Date: Mon, 30 Jan 2006 14:23:28 GMT
Lines: 26
Sender: [email-masked]

In <571CDBDC5B804B7501710EC0@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>Assigned ticket #1179.

>--On 23. januar 2006 21:39 +0100 Frank Ellermann <[email-masked]> =

>wrote:

>>
>>> Last states seen:
>> [...]
>>
>> I forgot to nominate the erroneous [FWS] as issue, see also:

I favour "No change needed".

To: [email-masked]
Xref: clerew local.usefor:23049
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 permitted constructs - a list
Date: Mon, 30 Jan 2006 12:39:01 GMT
Lines: 42
Sender: [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Open issues
Date:  Mon, 30 Jan 2006 16:12:02 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 72
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> Using your message to check off.....

Thanks.  I'm not sure if that list was complete.  One missing
minor point might be the <article-locator> ABNF different
from the <locator> in s-o-1036.

>> ISSUE 3.2.14 <sender> (no ticket yet ?  Under discussion)
[...]
> #1159

One solution was apparently to get rid of at least the SHOULD,
and another to get rid of sender= completely.

>> ISSUE Newsgroups and Distribution WSP
{...]
> #1178 (I hope I identified a reasonable piece from your
> message)

That was about moving a statement aboout this from USEPRO 3.2
to USEFOR 3.1.5 (Newsgroups), and about adding a note to 3.2.7
(Distribution) essentially saying "same here".

 [#1047]
> ABNF copied into ticket. It does not produce "!!".

That was a proposal for the '[FWS] before terminating "!"'
strategy.  Apparently some (most ?) of "us" prefer "!!".

The proposal also allowed "legacy identity with dots", and
that doesn't fly with your idea "IPv4 is always a legacy
diagnostics".

For a proposal reflecting your ideas about dots and "!!" see
<http://permalink.gmane.org/gmane.ietf.usenet.format/30547>
with an unnecessary ambiguity for ...!demon!...  fixed in
<http://permalink.gmane.org/gmane.ietf.usenet.format/30551>

I've added the fixed version below.  A version for the "old"
<toplabel> "allowing" a singleton ALPHA as TLD would be one
line shorter.
                        Bye, Frank

diag-literal    =  IPv4address / IPv6address     ; compare STD 66

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: fields
Date:  Sat, 28 Jan 2006 12:41:24 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 30
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> the practice of ABNF in the IETF has been that ambiguous
> parses are permitted, with no guidance on how to indicate
> how they are resolved.

The <fields> in -06 are apparently the only case in USEFOR
plus RfC 2822, where a "depth first left to right" parser
won't work.

And STD 66 (one of our sources) has some guidance in 3.2.2:

|    host        = IP-literal / IPv4address / reg-name
|
| The syntax rule for host is ambiguous because it does not
| completely distinguish between an IPv4address and a reg-name.
| In order to disambiguate the syntax, we apply the
| "first-match-wins" algorithm:
| If host matches the rule for IPv4address, then it should be
| considered an IPv4 address literal and not a reg-name.

With that trick they got rid of the dubious 2396 <toplabel>
and could allow other (non-DNS) "name registries".

With  news-fields = <our stuff> / fields  instead of the -06
notation  fields =/ <our stuff>  we'd also make sure that new
header fields for mail can't simply overwrite <our stuff> in
a "first-match-wins" parser.
                             Bye, Frank

Date: Fri, 27 Jan 2006 10:55:26 -0800
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: #1179 Re: ISSUE: replace twenty [FWS] by *WSP (was: Open issues)
Sender: [email-masked]

--==========54F93D8BBF5D50D3D396==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Assigned ticket #1179.

--On 23. januar 2006 21:39 +0100 Frank Ellermann <[email-masked]> =

wrote:

>
>> Last states seen:
> [...]
>
> I forgot to nominate the erroneous [FWS] as issue, see also:
>
> <http://article.gmane.org/gmane.ietf.usenet.format/30413>
>
> That's a list of 10 erroneous trailing [FWS] CRLF cases with
> a pointer to the submitted 2822-erratum about <unstructured>.
>
> Complete patch covering the 10 leading and 10 trailing [FWS]
> cases with a placeholder for #1047 <path-list>:
>
> unstructured    =3D  *WSP utext *( [FWS] utext ) *WSP
>
> message-id      =3D  "Message-Id:" SP *WSP msg-id *WSP
>
> newsgroup-list  =3D  *WSP newsgroup-name
>                    *( [FWS] "," [FWS] newsgroup-name ) *WSP
>
> path-list       =3D  <see #1047 discussion>
>
> poster-text     =3D  *WSP %d112.111.115.116.101.114 *WSP
>                    ; "poster" in lower-case
>
> control         =3D  "Control:" SP *WSP control-command *WSP CRLF
>
> supersedes      =3D  "Supersedes:" SP *WSP msg-id *WSP CRLF
>
> dist-list       =3D  *WSP dist-name
>                    *( [FWS] "," [FWS] dist-name ) *WSP
>
> xref            =3D  "Xref:" SP *WSP server-name
>                    1*( FWS location ) *WSP CRLF
>
> lines           =3D  "Lines" ":" SP *WSP 1*DIGIT *WSP CRLF
>
> The problematic [FWS] _within_ <newsgroup-list> and _within_
> <dist-list> is a separate ISSUE (got no ticket yet), compare
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30395>
>
> The problematic {FWS] _within_ <control-command> is a part
> of #1157 (control cancel bug).
>                                Bye, Frank
>
>
>
>

--==========54F93D8BBF5D50D3D396==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========54F93D8BBF5D50D3D396==========--

Date: Fri, 27 Jan 2006 10:53:17 -0800
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: Open issues
Sender: [email-masked]

--==========B9556F4CA90C7E480B07==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Using your message to check off.....

--On 21. januar 2006 20:04 +0100 Frank Ellermann <[email-masked]> =

wrote:

>
> Hi, I'm starting to get lost again with too many open issues
> where I've no clue if they're recorded in the ticket system,
> already fixed in the editor's copy, under discussion, or in
> some other state.
>
> Last states seen:
>
># 1155 USEFOR ABNF imports (should be "text proposed" ?)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30341>
>
># 1156 IANA considerations (NNTP-Posting-* not yet ready ?)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30342>
>
># 1157 Control cancel bug (should be "text proposed" ?)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30345>
>
> ISSUE 3.2.14 <sender> (no ticket yet ?  Under discussion)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30351>
#1159
>
># 1158 host-value (should be "text proposed" ?)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30391>
>
> ISSUE Newsgroups and Distribution WSP (no ticket yet ?)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30395>
#1178 (I hope I identified a reasonable piece from your message)

> ISSUE Missing [CFWS] (no ticket yet ?  "Text proposed")
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30427>
#1177
>
> ISSUE fields (no ticket yet ?  "Text proposed")
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30451>
Suggest to ignore.

>
># 1032 Appendix B (three missing items nominated)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30495>
>
> Maybe it's possible to get rid of all minor issues (more like
> nits) in a fresh draft.
>
> A complete #1047 ABNF proposal exists, apparently it has all
> required "features":
>
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30512>

ABNF copied into ticket. It does not produce "!!".

>                             Bye, Frank

--==========B9556F4CA90C7E480B07==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========B9556F4CA90C7E480B07==========--

Date: Fri, 27 Jan 2006 10:44:36 -0800
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: ISSUE: fields (was: path-legacy (bareword) and toplabel in A ~ B)
Sender: [email-masked]

--==========87999DACA8338DD74A4D==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

I think I'm going to not file this issue.... the practice of ABNF in the=20
IETF has been that ambiguous parses are permitted, with no guidance on how=20
to indicate how they are resolved.

                 Harald

--On 16. januar 2006 14:53 +0100 Frank Ellermann <[email-masked]> =

wrote:

>> fields          =3D       *(trace
> [...]
>>                         optional-field)
> Ugh, and we inherit this using fields =3D/ <our-stuff>.
>
>> All the headers will match "optional-field"; it's a matter of giving
>> priority in parsing (ugh) to parse "from" as a from-field and not as
>> an optional-field.
>
> You just killed all our header fields, because they come after
> the <optional-field> catch-all.
>
>> Messy, but not novel.
>
> Okay, proposed fix, we rename our <fields> to <news-fields>,
> and replace the "fields =3D/ <our-stuff>" in draft -06 by a
>
>             news-fields =3D <our-stuff> / fields
>
> With that trick <optional-field> is again the last catch-all.
>
> Besides Bill's parser never liked the old fields =3D/ approach,
> it always proposed to join the two parts of the <fields>.  Bye
>
>
>
>

--==========87999DACA8338DD74A4D==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========87999DACA8338DD74A4D==========--

Date: Fri, 27 Jan 2006 09:12:00 -0800
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: #1177 Missing [CFWS] (was: ...ranting and maybe four missing [CFWS] ?!?)
Sender: [email-masked]

--==========35CC5A2C8DC205CC7F1E==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Ticket created.

--On 12. januar 2006 21:26 +0100 Frank Ellermann <[email-masked]> =

wrote:

>
> JFTR:
>
>>> archive =3D  "Archive:" SP [CFWS] ("no" / "yes")
>>>            *( [CFWS] ";" [CFWS] archive-param ) [CFWS] CRLF
> {...]
>> injection-info  =3D  "Injection-Info:" SP [CFWS] path-identity
>>                    *([CFWS] ';' [CFWS] parameter) [CFWS] CRLF
>
> Plus Charles' proposed modification of the corresponding Note.
>
>
>
>

--==========35CC5A2C8DC205CC7F1E==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========35CC5A2C8DC205CC7F1E==========--

Date: Thu, 26 Jan 2006 14:07:06 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1047 permitted constructs - a list
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

I'll just remind that I expressed an opinion for option B that didn't seem 
to make it to the count.

>followed by more information ...

> (the nice thing about !.diagnostic ...

You didn't specify that syntax. Your syntax would result in a path entry
that looks like:

 	example.com.MISMATCH!

Which is completely undifferentiable from a normal path-identity.

How about:

- A path-element can be:

* path-identity followed by !
* path-diagnostic followed by !

And then the rest about what a path-diagnostic can be, since the path 
diagnostic is defined to contain the leading dot.

But just what IS a path-diagnostic anyway?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 permitted constructs - a list
Date:  Thu, 26 Jan 2006 22:27:46 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 8
Sender: [email-masked]

Sorry...

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: path-legacy (bareword) and toplabel in A ~ B
Date:  Thu, 26 Jan 2006 21:30:28 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 119
Sender: [email-masked]

Charles Lindsey wrote:

> There is a balance to be struck between putting extra
> complexity into the syntax, or having a simpler syntax
> with MUSTard to remove the unwanted cases.

Yes, I prefer KISS without MUSTard where possible.

> the syntax of the tail-entry is pretty unimportant, but it
> is a minor detail compared to the rest of the syntax.

[... quote rearranged ...]

> tail-entry = path-identity

My last proposal - <path-nodot> is a liberal <bareword> - was:

| path-nodot      =  1*( alphanum / "-" / "_" )
|
| tail-entry      =  *( path-nodot "." ) path-nodot

Pretty similar, your version matches less obscure tail-entries.

> [how about tail-entry = 1*( alphanum / "-" / "_" / "." ) ?]

That would match more than the <path-nodot> construct, it can
have adjacent dots.  Allowing "!..." might be a dubious idea.
An unquoted "..." is even no 2822 <dot-atom-text> or <id-left>.

>    path = "Path" ":" SP [FWS]
>              *( path-identity [FWS] "!" path-diagnostic )
>              tail-entry [FWS] CRLF
[...]
>    path-diagnostic = [ "!" ]
>                        [ "." keyword [ diagnostic-identity ] [FWS] "!" ]

That allows  y!!.SEEN.u!x  or  y!!.POSTED!x  with a bogus "!!".

And "we" want at most one [FWS] per <path-element>, you have
two.  IIRC "limit [FWS]" was originally your idea.

>    keyword = ( "SEEN" / "MATCH" / "MISMATCH" / "POSTED" )

That allows  y!.MATCH.x!x  (and  y!!.MATCH.x!x ), that's wrong,
"we" want  y!!x  for this purpose.

Or at most  y!.MATCH!x  in a scheme for "[FWS] before "!" where
"!!" doesn't work.

It also allows  y!.POSTED.u!x  (and  Y!!.POSTED.u!x ), but "we"
don't want a <diagnostic-identity> with .POSTED or .MATCH

>    path-identity = fqdn / bareword / obs-path-identity
>
>    obs-path-identity = IPv4address ; and any other cases thought necessary

Apparently Harald wants the IPv4 case firmly buried in <o-p-d>
and removed from <path-identity>.  I renamed his <o-p-d> to
<diag-deprecated> and used the same trick as for <tail-entry>:

| diag-deprecated =  "!" *( path-nodot "." ) path-nodot

That matches almost any crap (incl. IPv4) as <diag-deprecated>.

>    label = alphanum / alphanum *( alphanum / "-" ) alphanum

That doesn't have the 4234 recommended grouping.  My proposal:

| label           =  alphanum [ *( alphanum / "-" ) alphanum

>    toplabel = [ label *( "-" ) ] ALPHA [ label *( "-" ) ]
>               / label 1*( "-" ) label    ; at least one non-digit

The same 4234 grouping issue for the hyphen-alternative.  And
it's the "old" toplabel idea allowing a singleton ALPHA as TLD.

Maybe it's ridiculous, but I fear that whatever we pick here
could sooner or later end up as 2606bis / ICANN policy for the
whole world, and in that case I'd prefer "at least two char.s".

>    bareword = ALPHA / ALPHA *( alphanum / [ "-" / "_" ] ) alphanum )

More restrictive than <path-nodot> (and missing 4234 grouping).
Can the legacy names really never begin with DIGIT / "-" / "_"
or end with "-" / "_" ?

> diagnostic-identity = fqdn / IP-address / bareword

That won't allow <obs-path-identity> unless it's <IP-address>.
In my proposal any <path-identity> or <IP-address> is allowed:

| diag-seen       =  "!.SEEN."     ( path-identity / diag-literal )
| diag-mismatch   =  "!.MISMATCH." ( path-identity / diag-literal )

> If you look at the <path-diagnostic>, you can't have more
> than one of them at a time

Almost, you still have "!!" together with other disagnostics,
and you have <diagnostic-identity> where you don't want them,
i.e. for MATCH (fixed by simply removing it) and POSTED (bug).

> the keyword may or may not be followed by an identity (that
> is for the benefit of ".POSTED").

ACK, what you call "benefit" is what I call "bug".

> I have introduced <obs-path-identity>, which can grow to
> accomodate any further cases deemed necessary.

Yes, but apparently not allowed in <diagnostic-identity>.

> We can argue as to whether <bareword> belongs in
> <obs-path-identity> or not.

This would also affect !.MISMATCH.demon! (allowed as you have
it, also in my version), but not more if we move it to <o-p-i>.

                         Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  #1047 path-legacy vs. SHOULD NOT path-nodot / bareword (was: path-legacy (bareword) and toplabel in A ~ B)
Date:  Thu, 26 Jan 2006 20:02:45 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 16
Sender: [email-masked]

Charles Lindsey wrote:

> by all means review our previous policy if you want to, but
> do not bring in changes by the back door by introducing words
> such as "legacy" for the names of syntactic objects.

If you really think that an _explicit_ SHOULD NOT in the prose
about the <bareword> in
<http://permalink.gmane.org/gmane.ietf.usenet.format/30548>
or about the <path-nodot> in
<http://permalink.gmane.org/gmane.ietf.usenet.format/30547>
is "better" than simply saying <path-legacy> without any
2119 prose, then that's a bit odd, but otherwise acceptable.

                              Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:23037
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Thu, 26 Jan 2006 13:12:47 GMT
Lines: 128
Sender: [email-masked]

>> That is just a matter of writing the correct syntax,
>> so that [FWS] just never happens in between the two '!'s.

>Nobody managed to propose acceptable syntax for this detail.

I published such a syntax six or so weeks ago. I give another such syntax
below.

>There's no third option to get this right if "!" [FWS] "!"
>is undesirable, and if "!!" must not be used in conjunction
>with other diagnostics.

There is a balance to be struck between putting extra complexity into the
syntax, or having a simpler syntax with MUSTard to remove the unwanted
cases.

>We don't want ...!x!.POSTED!!y!not-for-mail (or similar).
>We don't want ...!x!.MISMATCH.z!!y!... (dito for any SEEN)
>We don't want ...!x!.POSTED!.POSTED!y!not-for-mail
>We don't want ..,!x!.MISMATCH.a!.MISMATCH.b!y...
>We don't want ...!x!.MISMATCH.a!!.POSTED!y!not-formail

All of those examples are stupid, but none of them would actually do any
harm if it slipped through. The instructions in USEPRO telling you how to
construct the Path header would not produce any of them.

>>>> Why on earth do you want to permit a <path-legacy> with
>>>> with both a "." and a "_" in it?

>>> Russ wants this.

><http://article.gmane.org/gmane.ietf.usenet.format/30442>
><http://article.gmane.org/gmane.ietf.usenet.format/30459>

I have reread both of those, and they do not seem to say what you imply.
The first just seems to be saying that if you allowed dots into a
bareword, then that would permit some non-valid FQDNs. It does not say
that such things need to be accomodated.

The second is commenting on your syntax which introduced such dots, and
seems to be saying that at least it includes all the things currently seem
out there.

However, Russ can easily clarify what he meant.

Russ: Habe you ever seen things like !news.foo_bar.example! out there in
the real world, and often enough that we need to take account of them?

>As far as dots in the <tail-entry> are concerned all I can
>offer is "I use addresses with dots in their local part".

In your tail entries? I agree that the syntax of the tail-entry is pretty
unimportant, but it is a minor detail compared to the rest of the syntax.

Anyway, if you want a syntax that leaves as little as possible to
verbiage, then here it is. Personally, I would prefer something shorter
that corresponded to the MUST accept stuff, and left what was to be
generated to MUSTard, either here or in USEPRO.

   path = "Path" ":" SP [FWS]
             *( path-identity [FWS] "!" path-diagnostic )
             tail-entry [FWS] CRLF 

[The 1st & last [FWS] might become *WSP if the Issue on that topic
succeeds] 
 
   path-diagnostic = [ "!" ]
                       [ "." keyword [ diagnostic-identity ] [FWS] "!" ]

   keyword = ( "SEEN" / "MATCH" / "MISMATCH" / "POSTED" ) 

[or maybe keyword = 1*ALPHA and let USEPRO introduce those that are
needed] 
 
   path-identity = fqdn / bareword / obs-path-identity 

   obs-path-identity = IPv4address ; and any other cases thought necessary

   fqdn = 1*( label "." ) toplabel

   label = alphanum / alphanum *( alphanum / "-" ) alphanum

   toplabel = [ label *( "-" ) ] ALPHA [ label *( "-" ) ]
              / label 1*( "-" ) label    ; at least one non-digit

   alphanum = ALPHA / DIGIT

   bareword = ALPHA / ALPHA *( alphanum / [ "-" / "_" ] ) alphanum )

   tail-entry = path-identity

[how about tail-entry = 1*( alphanum / "-" / "_" / "." ) ?]

   diagnostic-identity = fqdn / IP-address / bareword

   IP-address = IPv4address / IPv6address ;  see [RFC 3986]

Some notes on that:

If you look at the <path-diagnostic>, you can't have more than one of them
at a time, and the syntax allows them to be completely empty (the usual
case).

They may start with a "!" (giving you a "!!" with no FWS inside it). The
notation is to use a leading "." before the <keyword> (which seems now to
be the consensus), and the keyword may or may not be followed by an
identity (that is for the benefit of ".POSTED").

I have introduced <obs-path-identity>, which can grow to accomodate any
further cases deemed necessary. We can argue as to whether <bareword>
belongs in <obs-path-identity> or not.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 permitted constructs - a list
Date:  Thu, 26 Jan 2006 12:10:32 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 123
Sender: [email-masked]

 

  

                       Bye, Frank

Date: Wed, 25 Jan 2006 18:08:17 -0800
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1047 permitted constructs - a list
Sender: [email-masked]

--==========037366EBAF1A5F9F5231==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

(the nice thing about !.diagnostic is that you only have to examine the=20
first character of the diagnostic to know whether it is one or not...)

Does that capture roughly where people have been going, without introducing =

                   Harald

--==========037366EBAF1A5F9F5231==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========037366EBAF1A5F9F5231==========--

To: [email-masked]
Xref: clerew local.usefor:23031
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Wed, 25 Jan 2006 11:14:48 GMT
Lines: 51
Sender: [email-masked]

In <[email-masked]> Seth Breidbart <[email-masked]> writes:

>>>> Seems to me that "legacy" is a word we should avoid using,
>>
>>>IBTD, it's perfectly clear.
>>
>> Not if it implies "SHOULD NOT", because Richard has specifically asked
>> that Demon should not be in any violation of a SHOULD NOT.

>I think "legacy" means "It's OK to continue if you're already doing
>it, but you SHOULD NOT start doing it."  So Demon isn't violating a
>SHOULD NOT because they've been doing it long enough.

>So I think "legacy" is the correct term.

There is a bunch of stuff out there that we MUST continue to accept, so
things don't suddenly stop working, and so that parsers need not feel
obliged to spend time checking for every conceivable syntactic oddity.

Out of that, there is a sub-bunch of stuff that we are happy to see
generated in the future, both for old sites and for new sites (FQDNs are
the obvious example, but not if they have "_"s in them).

The question at issue is whether "barewords" are in this latter category,
subject to recommendations not to use them without good cause, and
warnings to choose them carefully because there is no global register to
ensure uniqueness.

IMO they _should_ be in that latter category and until this last week,
when the word "legacy" suddenly started to be used to describe them, that
was the position we had agreed in this WG. In particular, the wording
containing those recommendations and warnings, and which is now in USEPRO
(as quoted in full a couple of days ago in response to Russ) was discussed
in this WG months ago and seemed to be agreed (for example, it was still
present in a proposed text submitted by Harald, even though an issue
remained with related wording regarding FQDNs).

So by all means review our previous policy if you want to, but do not
bring in changes by the back door by introducing words such as "legacy"
for the names of syntactic objects.

To: [email-masked]
Xref: clerew local.usefor:23032
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Wed, 25 Jan 2006 11:18:54 GMT
Lines: 24
Sender: [email-masked]

>> it is made clear that people use these barewords at their
>> own risk.

>This is NOT clear in your ABNF.

It was not meant to be clear in the ABNF (nor is there any way to do so).
The wording is in USEPRO, because this is a matter of "how" to use the
facilities provided, not a matter of what can legitimately be present
(which is what USEFOR is supposed to be about).

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: path-legacy (bareword) and toplabel in A ~ B
Date:  Wed, 25 Jan 2006 02:01:07 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 103
Sender: [email-masked]

Charles Lindsey wrote:

>>> more likely just B!!A!AA.

>> The latter won't fly for your "[FWS] before last !".

> That is just a matter of writing the correct syntax,
> so that [FWS] just never happens in between the two '!'s.

Nobody managed to propose acceptable syntax for this detail.

You either arrive at "!!" [FWS] with a folding always at the
end of an "identity plus optional diagnostics" <path-entry>,

Or you arrive at "!.MATCH" {FWS] "!" with a folding always
before the terminating "!" of this <path-entry> record.

There's no third option to get this right if "!" [FWS] "!"
is undesirable, and if "!!" must not be used in conjunction
with other diagnostics.

We don't want ...!x!.POSTED!!y!not-for-mail (or similar).
We don't want ...!x!.MISMATCH.z!!y!... (dito for any SEEN)
We don't want ...!x!.POSTED!.POSTED!y!not-for-mail
We don't want ..,!x!.MISMATCH.a!.MISMATCH.b!y...
We don't want ...!x!.MISMATCH.a!!.POSTED!y!not-formail

Your proposals make all these cases syntactically correct.
Not only Bruce already didn't like this months ago.

>>> Why on earth do you want to permit a <path-legacy> with
>>> with both a "." and a "_" in it?

>> Russ wants this.

> I do not recall that. Can you point me to where he said it?

<http://article.gmane.org/gmane.ietf.usenet.format/30442>
<http://article.gmane.org/gmane.ietf.usenet.format/30459>

> If it is common in present usage, then indeed we have to
> accept it (but not generate, of course). But some Real
> World examples would settle the matter.

As far as dots in the <tail-entry> are concerned all I can
offer is "I use addresses with dots in their local part".

Somebody found cases of IPv4 in the path, mostly near the
<tail-entry>, or probably some case of "old-path-diagnostic".

Any "old-path-diagnostic" is a proper subset of the catch
all <path-legacy> construct.  Harald has just confirmed that
we don't want IPs in "regular" path-identities, i.e. neither
diagnostics nor legacy.

> we don't want agents to be refusing articles just because
> they mistakenly have two diagnostics in succession

I certainly want this to be a clear syntax error for a "new"
<path-diagnostic>.  That's already dubious as it is, relax
the syntax, and it's garbage.  Everybody insisting on adding
garbage to the path can use <path-legacy> for this purpose.

If a relaying agent anywhere on the world checks the syntax
of the path before relaying articles it missed some details
in USEPRO - IIRC relaying agents are not supposed to do this.

> Yes, but you have carefully snipped my wording further down,
> where I said that an IPv4address there was a "MUST accept,
> MUST/SHOULD NOT generate".

I don't care much about most MUSTard in the prose, I want the
ABNF to be clear.  Or as somebody else put it:

 <http://permalink.gmane.org/gmane.org.w3c.public-iri/122>
| I have sufficient experience with "Grammar says this but spec
| says that"-style discussions that I avoid them as much as
| possible.

This is our best chance to get it right and avoid such useless
discussions in the future.

>> Your <bareword> also cannot start with a DIGIT.

> I think that is what we agreed a few weeks back when
> <bareword>s were last discussed.

I don't recall that, and I found nothing about it in an old
grizzly book.  With <path-legacy> as catch-all it's no issue,
why should "123abc" be worse than say "demon" or "a__123" ?

BTW, it would be simple to integrate your idea of "multiple
path identities" into the proposed _unfoldable_ <path-entry>:

~~~ old ~~~
path-entry      =  path-identity / path-verified / path-unverified /
                   path-mismatch / path-posted
~~~ new ~~~
path-entry      =  *( path-identity "!" ) ( path-identity /
                      path-verified / path-unverified /
                      path-mismatch / path-posted )
~~~ bye ~~~

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: path-legacy (bareword) and toplabel in A ~ B
Date:  Tue, 24 Jan 2006 23:13:22 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 27
Sender: [email-masked]

Charles Lindsey wrote:

>>> Seems to me that "legacy" is a word we should avoid using,

>> IBTD, it's perfectly clear.

> Not if it implies "SHOULD NOT", because Richard has
> specifically asked that Demon should not be in any
> violation of a SHOULD NOT.

Then propose some text to establish a IANA registry replacing
the defunct UUCP maps for "demon", and propose ABNF reflecting
this concept.  Otherwise it's in conflict with a MUST about
relayer names in son-of-1036.

> it is made clear that people use these barewords at their
> own risk.

This is NOT clear in your ABNF.  Please accept that this is
"legacy" that won't work as expected in the case of conflicts
or trivial tasks like "try to add news@ to get an address".

With a path ...!what.ever example!demon!...  it's dubious that
an address <[email-masked]> works.  This is now
ancient history.  We all love it, that's why we are here, but
we should now also admit that it's "legacy" and move on.  Bye

Date: Tue, 24 Jan 2006 16:43:15 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Sender: [email-masked]

Seth Breidbart wrote:

>>>>Seems to me that "legacy" is a word we should avoid using,
>>
>>>IBTD, it's perfectly clear.
>>
>>Not if it implies "SHOULD NOT", because Richard has specifically asked
>>that Demon should not be in any violation of a SHOULD NOT.
> 
> 
> I think "legacy" means "It's OK to continue if you're already doing
> it, but you SHOULD NOT start doing it."  So Demon isn't violating a
> SHOULD NOT because they've been doing it long enough.
> 
> So I think "legacy" is the correct term.

+1

And 'SHOULD NOT' and 'MUST NOT' are not the only words we should use
to communicate reality to a cooperative implementor.

We've already been through all the flame wars here about that.  EVERY 
implementor that reads an IETF document is a voluntary, cooperative
implementor....

Especially with Usenet, which apparently exists for the express purpose
of lawless symbiosis, IETF standards are not effective cluebats.

No imperatives needed here for legacy, if plain words will do.

Date: Tue, 24 Jan 2006 16:27:44 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Sender: [email-masked]

>>> Seems to me that "legacy" is a word we should avoid using,
>
>>IBTD, it's perfectly clear.
>
> Not if it implies "SHOULD NOT", because Richard has specifically asked
> that Demon should not be in any violation of a SHOULD NOT.

I think "legacy" means "It's OK to continue if you're already doing
it, but you SHOULD NOT start doing it."  So Demon isn't violating a
SHOULD NOT because they've been doing it long enough.

So I think "legacy" is the correct term.

Seth

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: replace twenty [FWS] by *WSP
Date:  Tue, 24 Jan 2006 22:10:47 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 67
Sender: [email-masked]

Charles Lindsey wrote:

> Yes, you have been flogging this for months

We needed about a year to determine that it's unnecessary to
do something with leading SP [CFWS] and trailing [CFWS] CRLF,
because the latter is clear in 2822.

After that red herring we checked why 2822 <unstructured> is
utter dubious, here and on the rfc822 list (summer 2005).

With Bruce's help we finally arrived at the conclusion that
the 2822 <unstructured> [FWS] cannot simply be replaced by
*WSP, because that would lose the important <obs-FWS> case.

That's now (2006) submitted as 2822-erratum, RfC-editor and
2822-author can fight it out when they find the time for it.

USEFOR -06 has it clear (1.4 and 2.1) that <obs-FWS> isn't
allowed in conformant articles, and that the ABNF describes
header fields in conformant articles.

> not a single person has supported your position

You supported that 2822 <unstructured> is wrong, and while
Bruce didn't go that far he also said that an "empty" header
field line is clearly NOT what RfC 2822 wants, including all
unstructured header fields.

> If this is a problem, then it is a problem in RFC 2822, and
> RFC 2822bis is the place to fix it

It is a problem, and 2822bis cannot fix errors in the ABNF of
NetNews header fields Newsgropus, Distribution, Followup-To,
Xref, Lines, Supersedes, Control, and Path.

> any such fix will not fix all of the cases where folding is
> disallowed

It addresses obvious problems for the mentioned header fields.
Dito for Message-Id and all <unstructured> header fields like
Summary, Comments, and Subject.

As far as I'm concerned Message-Id is a NetNews header field.
We've seen that folding it at the first [FWS] does not work.

Summary is also something that can't be "fixed" by a 2822bis,
unless they adopt it.  Besides they'd be completely unable to
get this right for NetNews:

- they have no "magic SP" like NetNews
- they have to consider their own legacy <obs-FWS>
- their only leading + trailing [FWS] is in <unstructured>
- their only additional leading [FWS] is in <day-of-week>
- improving the leading [FWS] in <unstructured> could affect
  2047-encoded unstructured text wrt to line length limits.

=> our job to fix this for NetNews, 2822bis cannot solve it.

We got separate tickets or pending issues for three cases of
"embedded" [FWS] in Newsgroups, Distribution, and Control.

This is no nonsense or lillygilding, it's a serious issue:
Russ proposed to send a folded Control directly to /dev/null.

                        Bye, Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Injection-Info and <sender>
Organization: The Eyrie
Date: Tue, 24 Jan 2006 12:52:06 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> The important bit is providing a place to put the authorization
>> parameter, encrypted, if the site wants to.  Providing an encrypted
>> place for the authentication parameter as well is possibly useful but
>> not horribly important to me.

> I think we agree that goes in posting-account, if the situation arises.

The authorization parameter, you mean.  Yes?

Date: Tue, 24 Jan 2006 20:22:24 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes
>
>In <[email-masked]> Russ Allbery <[email-masked]> 
>writes:
>
>>I really don't think that !.MISMATCH.news.example.com! is going to stress
>>anyone's brain to figure out.  I think it's a lot cleaner to put the token
>>in front rather than doubling the period; people are too likely to omit
>>the second period because it looks like a typo.
>
>If that is the majority view, then OK. But I still think a single "." at
>the beginning is easily overlooked by the human reader.

no-one reads this stuff unless they are paranoid about forgery...

... if so then they will pay attention

> How about a double
>dot, just to make it more conspicuous as in
>       "!..MISMATCH.news.example.com!"?

people pay money for their bandwidth -- it adds nothing :(

- -- 
richard @ highwayman . com                       "Nothing seems the same
                          Still you never see the change from day to day
                                And no-one notices the customs slip away"

Date: Tue, 24 Jan 2006 13:10:22 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Sender: [email-masked]

Charles Lindsey wrote:

> In <[email-masked]> Russ Allbery <[email-masked]> writes:
> 
> 
>>I really don't think that !.MISMATCH.news.example.com! is going to stress
>>anyone's brain to figure out.  I think it's a lot cleaner to put the token
>>in front rather than doubling the period; people are too likely to omit
>>the second period because it looks like a typo.
> 
> 
> If that is the majority view, then OK. But I still think a single "." at
> the beginning is easily overlooked by the human reader. How about a double
> dot, just to make it more conspicuous as in
>        "!..MISMATCH.news.example.com!"?
> 

I think two dots is easily overlooked, especially in a proportional font
set to show all 900 characters on one line.  3 dots
might be good, but might be confusing because it looks like an ellipsis,
and some gateway might do inappropriate Unicoding of that into some other code 
point entirely.

Plus if there is a data channel bit error, the dot might turn into something 
else entirely.

So,

If we make it 10 dots, there is a sufficient level of redundancy that bit-error
rates on lossy channels will not confuse anyone, and future I18N of
domain names is extremely unlikely to use a sequence of 10 dots for
overloading.  And it isn't a multiple of 3, so that avoids some ellipsis
recoding problems.  Hmm, maybe 11 is better because that is 3n+2, so there
are at least 2 dots leftover.

I think we need a poll with choices of 1 to 10, and then take
the average number.  That would be most democratic, and a loud 10 is weighted
in the average 5 times better than a vote for 2.

Date: Tue, 24 Jan 2006 13:03:26 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
CC: Charles Lindsey <[email-masked]>
Subject: Re: ISSUE: replace twenty [FWS] by *WSP
Sender: [email-masked]

Charles Lindsey wrote:

> In <[email-masked]> Frank Ellermann <[email-masked]> writes:
> 
> 
>>>Last states seen:
>>
>>[...]
> 
> 
>>I forgot to nominate the erroneous [FWS] as issue, see also:
> 
> 
>><http://article.gmane.org/gmane.ietf.usenet.format/30413>
> 
> 
> Yes, you have been flogging this for months, but not a single person has
> supported your position AFAICT.

I support Frank's position.  Posting "me too" is bad form, and as
far as I remember, no one has offered a reasonable counter to
Frank's position, except that it isn't what RFC2822 says, (as if
that argument carries all the water in other header fields discussions.)

'*WSP' is even one less character than '[FWS]' and I'm all for
shortening documents (although USEFOR isn't offensive.)

> 
> If this is a problem, then it is a problem in RFC 2822, and RFC 2822bis is
> the place to fix it. But, since any such fix will not fix all of the cases
> where folding is disallowed, there is little to be gained by making the
> change for these few.

We said that USEFOR is more restrictive than RFC 2822.

*WSP is a strict subset of [FWS]

To: [email-masked]
Xref: clerew local.usefor:23023
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: replace twenty [FWS] by *WSP (was: Open issues)
Date: Tue, 24 Jan 2006 16:22:31 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>> Last states seen:
>[...]

>I forgot to nominate the erroneous [FWS] as issue, see also:

><http://article.gmane.org/gmane.ietf.usenet.format/30413>

Yes, you have been flogging this for months, but not a single person has
supported your position AFAICT.

If this is a problem, then it is a problem in RFC 2822, and RFC 2822bis is
the place to fix it. But, since any such fix will not fix all of the cases
where folding is disallowed, there is little to be gained by making the
change for these few.

>The problematic [FWS] _within_ <newsgroup-list> and _within_
><dist-list> is a separate ISSUE (got no ticket yet),

[FWS] within newsgroups list was discussed and wording agreed long ago. I
agree that similar wording is meeded in <dist-list> but there is no
controversy there. Ken should just get on and do it (maybe Harald should
confirm that).

To: [email-masked]
Xref: clerew local.usefor:23022
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Tue, 24 Jan 2006 16:12:02 GMT
Lines: 106
Sender: [email-masked]

>> B!A..MISMATCH!AA and B!..POSTED!A!AA

>I like Harald's _leading_ dot better, because that's
>more obvious (in conjunction with a leading keyword),
>but maybe that's a matter of taste.

Indeed a matter of taste. If the majority wants it at the beginning, it is
easily done. But I would still prefer two dots, even at the beginning,
just to make it stand out more for human readers (anyone foolish enough to
read headers in a proportional font will have trouble spotting it).

>> B!A..MATCH!A!AA, or more likely just B!!A!AA.

>The latter won't fly for your "[FWS] before last !".

That is just a matter of writing the correct syntax, so that [FWS] just
never happens in between the two '!'s.

>> Why on earth do you want to permit a <path-legacy>
>> with both a "." and a "_" in it?

>Russ wants this.

I do not recall that. Can you point me to where he said it? If it is
common in present usage, then indeed we have to accept it (but not
generate, of course). But some Real World examples would settle the
matter.

>  And if <tail-entry> is some kind
>of syntactically restricted local part, we might
>need it anyway.

Yes, I have no problem if people want to allow more complicated things in
the <tail-entry>, such as a full <path-identity>.

>> path = "Path" ":" SP [FWS] *( path-entry [FWS] ( "!" / "!!" ) )
>>                            tail-entry [FWS] CRLF

>The first {FWS] violates MUSTard, it should be *WSP.
>Dito the last [FWS].

No, that is a separate issue for which there is a separate thread.

>> path-entry = path-identity / path-diagnostic

>You've split "MATCH" from the rest of the diagnostic
>zoo, that doesn't reflect the "!!" semantic.

Eh? "MATCH" (if we have it) is a keyword. "!!" needs to appear in the syntax
in its own right (to ensure there is no [FWS] inside it). That means it is
probably not a keyword syntactically speaking, even though its semantics
maybe the the same as a keyword called "MATCH". But USEPRO is the place to
tell you when to use or not use it.

>A <path-entry> should be what a site adds to the path,
>one record consisting of identity and maybe dianostic.

Again, that can be enforced in USEPRO. Our Chair seems to want to keep the
syntax here as simple as we can get away with (and we don't want agents to
be refusing articles just because they mistakenly have two diagnostics in
succession). That is why I removed that minor complication from what I had
in my earlier syntaxes. It can go back if people want it so.

>It does not add arbitrary sequences of identity and
>diagnostic, it adds precisely one identity followed
>by zero or one diagnostic.  And if the record ends
>with "!!" it's a MATCH.  You can't have "!!" behind
>any other diagnostic like MISMATCH / POSTED / SEEN.

As I said, if something turns up that USEPRO says should NEVER have been
generated, then nothing in the protocol actually breaks, though someone
who inspects the line manually might well be confused.
> 
>>    path-identity = fqdn / bareword
>>    fqdn = 1*( label "." ) label ; subject to restriction below

>Without <toplabel> that matches <IPv4address>, but
>we have some kind of consensus for "no IP".

Yes, but you have carefully snipped my wording further down, where I said
that an IPv4address there was a "MUST accept, MUST/SHOULD NOT generate".

>>    label = alphanum / alphanum *( alphanum / "-" ) alphanum
>>    bareword = ALPHA / ALPHA *( alphanum / [ "-" / "_" ] ) alphanum )

>Your <bareword> also cannot start with a DIGIT.

I think that is what we agreed a few weeks back when <bareword>s were last
discussed. 

To: [email-masked]
Xref: clerew local.usefor:23020
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Info and <sender>
Date: Tue, 24 Jan 2006 15:32:04 GMT
Lines: 46
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Oh, I see, you meant something else by "authentication is the one we
>really want" than what I thought.  I thought you meant that in the sense
>that it was the more important value, which isn't the case; you instead
>meant that it was the one that should be a separate parameter.  That I
>agree with, although really, the naming of the parameters is unimportant
>to me as long as it's not horrible and as long as the document clearly
>defines it.

OK, does that mean we now have agreement to go ahead and include it? You
and I appear to accept it. Any other comments?

And there is also your proposal to remove the semder parameter. I am not
enthusiastic about that, but if it is the general desire, then so be it.

>The important bit is providing a place to put the authorization parameter,
>encrypted, if the site wants to.  Providing an encrypted place for the
>authentication parameter as well is possibly useful but not horribly
>important to me.

I think we agree that goes in posting-account, if the situation arises.

>I am considerably less interested in the use of Injection-Info for
>killfiles (something that I frankly think is unlikely given the hideous
>syntax that won't work with existing killfile tools) than I am in its use
>for trace information, which should be the *primary* purpose.

I think people will want to try and use killfiles here, if their user
agents make it possible. So long as injectors don't make their
Injection-Infor headers needlessly complex, then it will propably be
possible. But I agree that it is not the primary purpose of this header -
just a useful fallout possibility.

To: [email-masked]
Xref: clerew local.usefor:23021
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Tue, 24 Jan 2006 15:39:29 GMT
Lines: 22
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>I really don't think that !.MISMATCH.news.example.com! is going to stress
>anyone's brain to figure out.  I think it's a lot cleaner to put the token
>in front rather than doubling the period; people are too likely to omit
>the second period because it looks like a typo.

If that is the majority view, then OK. But I still think a single "." at
the beginning is easily overlooked by the human reader. How about a double
dot, just to make it more conspicuous as in
       "!..MISMATCH.news.example.com!"?

To: [email-masked]
Xref: clerew local.usefor:23024
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Tue, 24 Jan 2006 16:27:15 GMT
Lines: 35
Sender: [email-masked]

>New browsers SHOULD NOT only support [see above], only old
>browsers have an excuse.

>> Seems to me that "legacy" is a word we should avoid using,

>IBTD, it's perfectly clear.

Not if it implies "SHOULD NOT", because Richard has specifically asked
that Demon should not be in any violation of a SHOULD NOT.

>Therefore it's our duty to say that this might not work as
>expected, and while we're at it it's also clear that "demon"
>is not protected from other systems claiming to be "demon".

Yes, we do say that, and it is made clear that people use these barewords
at their own risk. But they have been doing just that for years, and the
sky has not fallen in yet.

Date: Mon, 23 Jan 2006 23:50:07 -0800
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: Open issues
Sender: [email-masked]

--==========D5F850BE04D324E0B93E==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Hi, apologies for falling behind.... on the road again.

Expect to get back to USEFOR issue tracking on Wednesday - not today.

            Harald

--On 21. januar 2006 20:04 +0100 Frank Ellermann <[email-masked]> =

wrote:

>
> Hi, I'm starting to get lost again with too many open issues
> where I've no clue if they're recorded in the ticket system,
> already fixed in the editor's copy, under discussion, or in
> some other state.
>
> Last states seen:
>
># 1155 USEFOR ABNF imports (should be "text proposed" ?)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30341>
>
># 1156 IANA considerations (NNTP-Posting-* not yet ready ?)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30342>
>
># 1157 Control cancel bug (should be "text proposed" ?)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30345>
>
> ISSUE 3.2.14 <sender> (no ticket yet ?  Under discussion)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30351>
>
># 1158 host-value (should be "text proposed" ?)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30391>
>
> ISSUE Newsgroups and Distribution WSP (no ticket yet ?)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30395>
>
> ISSUE Missing [CFWS] (no ticket yet ?  "Text proposed")
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30427>
>
> ISSUE fields (no ticket yet ?  "Text proposed")
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30451>
>
># 1032 Appendix B (three missing items nominated)
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30495>
>
> Maybe it's possible to get rid of all minor issues (more like
> nits) in a fresh draft.
>
> A complete #1047 ABNF proposal exists, apparently it has all
> required "features":
>
> <http://permalink.gmane.org/gmane.ietf.usenet.format/30512>
>
>                             Bye, Frank
>
>
>
>

--==========D5F850BE04D324E0B93E==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========D5F850BE04D324E0B93E==========--

Date: Mon, 23 Jan 2006 23:53:00 -0800
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: #1047 POLL RESULT: Path-diagnostics
Sender: [email-masked]

--==========1A0C1D31BF98E7A0F5C5==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Given that 5 days have passed with no comments on this poll result=20
announcement, I'm declaring that the group has rough consensus (where Russ=20
Allbery and I are part of the rough) on Option A:

A: A path-diagnostic must be syntactically distinguishable from a=20
path-identity. Stuff that isn't a path-identity or a syntactically marked=20
path-diagnostic (such as an IP address) should be syntacitcally illegal.

So be it.

--On 18. januar 2006 16:02 +0100 Harald Tveit Alvestrand=20
<[email-masked]> wrote:

>
> I note a lot of discussion following this thread. Still, these are the
> messages I picked up that clearly respond to the poll:
>
> A: A path-diagnostic must be syntactically distinguishable from a
> path-identity. Stuff that isn't a path-identity or a syntactically marked
> path-diagnostic (such as an IP address) should be syntacitcally illegal.
>
> - Seth Breidbart "Preference"
> - Frank Ellermann "I could support"
> - Richard Clayton "would be closest to my viewpoint"
> - Charles Lindsey "Prefer"
>
> B: A path-diagnostic must be syntactically distinguishable from a
> path-identity. Stuff that isn't a path-identity or a syntactically marked
> path-diagnostic (such as an IP address) should be syntacitcally legal,
> but have no standardized meaning.
>
> - Russ Allbery "most realistic"
> - Seth Breidbart "also acceptable"
> - ??Charles Lindsey "might be moving a little closer to" (unclear)
>
> C: A path-diagnostic need not be syntactically distinguishable from a
> path-identity.
>
> - Russ Allbery "I can live with"
>
> D: None of the above alternatives is close to my opinion.
>
> - Frank Ellermann "I could support"
>
> 5 people, 9 opinions. I didn't count myself.
>
> Based on this, C seems to be definitely off the table. And there seems to
> be a strong opinion among the 5 of us that we should make stuff that
> doesn't match our syntax for diagnostics illegal.
>
> Are there further comments or people who want to be listed differently?
>
>
>                      Harald
>
>
>

--==========1A0C1D31BF98E7A0F5C5==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========1A0C1D31BF98E7A0F5C5==========--

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  ISSUE: replace twenty [FWS] by *WSP (was: Open issues)
Date:  Mon, 23 Jan 2006 21:39:58 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 45
Sender: [email-masked]

> Last states seen:
[...]

I forgot to nominate the erroneous [FWS] as issue, see also:

<http://article.gmane.org/gmane.ietf.usenet.format/30413>

That's a list of 10 erroneous trailing [FWS] CRLF cases with
a pointer to the submitted 2822-erratum about <unstructured>.

Complete patch covering the 10 leading and 10 trailing [FWS]
cases with a placeholder for #1047 <path-list>:

unstructured    =  *WSP utext *( [FWS] utext ) *WSP

message-id      =  "Message-Id:" SP *WSP msg-id *WSP

newsgroup-list  =  *WSP newsgroup-name
                   *( [FWS] "," [FWS] newsgroup-name ) *WSP

path-list       =  <see #1047 discussion>

poster-text     =  *WSP %d112.111.115.116.101.114 *WSP
                   ; "poster" in lower-case

control         =  "Control:" SP *WSP control-command *WSP CRLF

supersedes      =  "Supersedes:" SP *WSP msg-id *WSP CRLF

dist-list       =  *WSP dist-name
                   *( [FWS] "," [FWS] dist-name ) *WSP

xref            =  "Xref:" SP *WSP server-name
                   1*( FWS location ) *WSP CRLF

lines           =  "Lines" ":" SP *WSP 1*DIGIT *WSP CRLF

The problematic [FWS] _within_ <newsgroup-list> and _within_
<dist-list> is a separate ISSUE (got no ticket yet), compare
<http://permalink.gmane.org/gmane.ietf.usenet.format/30395>

The problematic {FWS] _within_ <control-command> is a part
of #1157 (control cancel bug).
                               Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: path-legacy (bareword) and toplabel in A ~ B
Date:  Mon, 23 Jan 2006 20:21:49 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 87
Sender: [email-masked]

Charles Lindsey wrote:
 
> I think we are converging on a possible solution

No, I don't think so, quite the contrary.

> your syntactic sugar is all wrong

It's my way to express what I think the concepts are.

> We already have some usage of
> !news.example.com.MISMATCH!

Yes, that's flat out, no 2606bis only for USEFOR and
obscure keyword pseudo-TLDs.

> B!A..MISMATCH!AA and B!..POSTED!A!AA

I like Harald's _leading_ dot better, because that's
more obvious (in conjunction with a leading keyword),
but maybe that's a matter of taste.

A quick reality check nslookup .a.de / nslookup a..de
has apparently the same (error) result.

> B!A..MATCH!A!AA, or more likely just B!!A!AA.

The latter won't fly for your "[FWS] before last !".
 
> several things I don't like about your syntax,
> including the lack of "!!".

That lack is a side-effect of your [FWS] preference.
With "! like comma" the "!!" is of course elegant.

If we ignore that IIRC Richard saw a few bogon "!!",
where the unclear meaning was definitely not "MATCH".

> Why on earth do you want to permit a <path-legacy>
> with both a "." and a "_" in it?

Russ wants this.  And if <tail-entry> is some kind
of syntactically restricted local part, we might
need it anyway.  <path-legacy> is a legacy catch-all.

> path = "Path" ":" SP [FWS] *( path-entry [FWS] ( "!" / "!!" ) )
>                            tail-entry [FWS] CRLF

The first {FWS] violates MUSTard, it should be *WSP.
Dito the last [FWS].  

> path-entry = path-identity / path-diagnostic

You've split "MATCH" from the rest of the diagnostic
zoo, that doesn't reflect the "!!" semantic.

A <path-entry> should be what a site adds to the path,
one record consisting of identity and maybe dianostic.  

It does not add arbitrary sequences of identity and
diagnostic, it adds precisely one identity followed
by zero or one diagnostic.  And if the record ends
with "!!" it's a MATCH.  You can't have "!!" behind
any other diagnostic like MISMATCH / POSTED / SEEN.
 
>    path-identity = fqdn / bareword
>    fqdn = 1*( label "." ) label ; subject to restriction below

Without <toplabel> that matches <IPv4address>, but
we have some kind of consensus for "no IP".

>    label = alphanum / alphanum *( alphanum / "-" ) alphanum
>    bareword = ALPHA / ALPHA *( alphanum / [ "-" / "_" ] ) alphanum )

Russ wanted "-" and "_" everywhere in <path-legacy>,
Your <bareword> also cannot start with a DIGIT.

>    path-diagnostic = [ path-identity / IP-address ] ".." keyword
>    IP-address = IPv4address / IPv6address ;  see [RFC 3986]
>    keyword = ; see below
>    tail-entry = bareword

That's a <tail-entry> without dots.  This does
nowhere reflect what we talked about for weeks.

                     Bye, Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Organization: The Eyrie
Date: Mon, 23 Jan 2006 09:19:57 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> OK, I think we are converging on a possible solution, but your syntactic
> sugar is all wrong. We already have some usage of
> !news.example.com.MISMATCH!, so we need a notation that is immediately
> recognisable to people who are used to that. Hence my proposal for

> B!A..MISMATCH!AA and B!..POSTED!A!AA

I really don't think that !.MISMATCH.news.example.com! is going to stress
anyone's brain to figure out.  I think it's a lot cleaner to put the token
in front rather than doubling the period; people are too likely to omit
the second period because it looks like a typo.

To: [email-masked]
Xref: clerew local.usefor:23010
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Date: Mon, 23 Jan 2006 14:42:03 GMT
Lines: 79
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>In message <[email-masked]>, Charles Lindsey
><[email-masked]> writes

>>In the cases I looked at, the IP address had clearly been added by the
>>site to the left (since the site to the left always appeared with some IP
>>address to its right - which is essentially what you are reporting).

>I see IPv4 addresses a lot as well...  but only right at the edge of
>network (at the far right of the path)  Usually there is something else
>nearby in the Path header field which is equivalent to POSTED.

Well I posted my evidence in
http://www.imc.org/ietf-usefor/mail-archive/msg01772.html, and indeed
there was one site (uni-berline.de) that regularly did it when injecting,
but there were others (over 1% of my sample) where it was in the middle.

Here is a bunch of Paths with IP addresses in the middle. I have truncated
the irelevant part of the Path up to the site of interest, and I have
folded manually so you can see them.

Path: ...!worldnet.att.net!207.14.113.17!news.alt.net!usenet
Path: ...!worldnet.att.net!216.196.98.144!border2.nntp.dca.giganews.com!
    nntp.giganews.com!local1.nntp.dca.giganews.com!nntp.nildram.net!
    news.nildram.net.POSTED!not-for-mail
Path: ...!worldnet.att.net!198.6.0.85!ash.uu.net!sl-news.
Path: ...!worldnet.att.net!198.6.0.86!ash.uu.net!sl-news.
Path: ...!worldnet.att.net!216.196.98.144!border2.nntp.dca.giganews.com!
    border1.nntp.dca.giganews.com!nntp.giganews.com!
    local1.nntp.dca.giganews.com!nntp.nildram.net!news.nildram.net.POSTED!
    not-for-mail
Path: ...!worldnet.att.net!216.196.98.144!border2.nntp.dca.giganews.com!
    border1.nntp.dca.giganews.com!nntp.giganews.com!
    local1.nntp.dca.giganews.com!nntp.pipex.net!news.pipex.net.POSTED!not-for-mail
Path: ...!worldnet.att.net!216.196.98.144!border2.nntp.dca.giganews.com!
    border1.nntp.dca.giganews.com!nntp.giganews.com!
    local1.nntp.dca.giganews.com!nntp.nildram.net!news.nildram.net.POSTED!
    not-for-mail
Path: ...!worldnet.att.net!207.14.113.17!news.alt.net!usenet
Path: ...!worldnet.att.net!207.14.113.17!news.alt.net!usenet
Path: ...!cyclone-sf.pbi.net!216.218.192.242!news.he.net!news.kjsl.com!
    news-peer-lilac.gradwell.net!not-for-mail
Path: ...!worldnet.att.net!207.14.113.17!news.alt.net!usenet
Path: ...!cyclone-sf.pbi.net!216.218.192.242!news.he.net!news.kjsl.com!
    news-peer-lilac.gradwell.net!news-peer.gradwell.net!news.markshouse.net
Path: ...!cyclone-sf.pbi.net!216.218.192.242!news.he.net!
    news.kjsl.com!news-peer-lilac.gradwell.net!not-for-mail
Path: ...!worldnet.att.net!207.14.113.17!news.alt.net!usenet
Path: ...!cyclone-sf.pbi.net!216.218.192.242!news.he.net!
    news.kjsl.com!news-peer-lilac.gradwell.net!not-for-mail
Path: ...!cyclone-sf.pbi.net!216.218.192.242!news.he.net!news.kjsl.com!
    news-peer-lilac.gradwell.net!not-for-mail
Path: ...!cyclone-sf.pbi.net!216.218.192.242!news.he.net!news.kjsl.com!
    news-peer-lilac.gradwell.net!not-for-mail
Path: ...!worldnet.att.net!129.250.169.16!pln-e!spln!rex!extra.newsguy.com!
    newsp.newsguy.com!psp.co.uk!tfl
Path: ...!worldnet.att.net!207.14.113.17!news.alt.net!usenet

In the case of the worldnet.att.net, it is immediately apparent that the
following IP adress correlates with the host received-from. In the case of
the cyclone-sf.pbi.net, nslookup shows
Name:    news.he.net
Address:  216.218.192.242

So those two sites are clearly using the IP address in our "SEEN" sense.
There are probably other sites doing the same (not a huge number, but
enough).

To: [email-masked]
Xref: clerew local.usefor:23012
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Mon, 23 Jan 2006 15:34:33 GMT
Lines: 78
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Just to be sure, that's "invisible", as in my first guess what
>Harald's dots are supposed to do, hide the complete diagnostics
>in B!.MISMATCH.A!AA

>For B!.POSTED!A!AA I hope we have it clear that A!AA is either
>what B got, or B got nothing constructing whatever!not-for-mail
>(i.e. A = whatever path-identity, AA = not-for-mail).

>In other words no case of B adding !.POSTED!A! to a given AA,
>also no !.POSTED.A! added to a given AA.

OK, I think we are converging on a possible solution, but your syntactic
sugar is all wrong. We already have some usage of
!news.example.com.MISMATCH!, so we need a notation that is immediately
recognisable to people who are used to that. Hence my proposal for

B!A..MISMATCH!AA and B!..POSTED!A!AA

for your two examples above plus

B!A..MATCH!A!AA, or more likely just B!!A!AA.

>In that line of the discussion the following ABNF should work,
>it has among others these features:

But there are several things I don't like about your syntax, including the
lack of "!!".

>path-identity   =  ( 1*( label "." ) toplabel ) / path-legacy
>path-legacy     =  *( path-component "." ) path-component

Why on earth do you want to permit a <path-legacy> with both a "." and a
"_" in it? Can anyone point me to a significant quantity of such beasties
out there in the real world?

Here is the syntax I would recommend if we follow this sort of route:

   path = "Path" ":" SP [FWS] *( path-entry [FWS] ( "!" / "!!" ) )
             tail-entry [FWS] CRLF
   path-entry = path-identity / path-diagnostic
   path-identity = fqdn / bareword
   fqdn = 1*( label "." ) label ; subject to restriction below
   label = alphanum / alphanum *( alphanum / "-" ) alphanum
   bareword = ALPHA / ALPHA *( alphanum / [ "-" / "_" ] ) alphanum )
   path-diagnostic = [ path-identity / IP-address ] ".." keyword
   IP-address = IPv4address / IPv6address ;  see [RFC 3986]
   keyword = ; see below
   tail-entry = bareword

   ALthough the above syntax permits an IPv4address to appear as a
   <path-identity>, this is only provided to accomodate some existing
   usage (usually of a diagnostice nature). Whilst it MUST still be
   accepted, it MUST/SHOULD not be generated in future (although an
   IPv4address remains acceptable in its own right within a
   <path-diagnostic>).

For the syntax of <keyword>, you either give an explicit list (MATCH,
MISMATCH, SEEN, POSTED, ...), or else you give a generic syntax (1*ALPHA
or somesuch). In either case, you point to USEPRO for the gory details of
when/how to generate.

You also point to USEPRO for when to use/not use <bareword>s and when to
use/not use "!!" (although the semanics of "!!" could be mentioned here,
as in the present draft).

To: [email-masked]
Xref: clerew local.usefor:23011
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047
Date: Mon, 23 Jan 2006 14:53:42 GMT
Lines: 41
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Richard Clayton wrote:

>> My impression remains that although there may have been some
>> reinvention of wheels for reporting mismatches, there's been
>> far less enthusiasm for reporting matches.

I think that is just because the issues raised recently have related to
the other cases. Back when Brad sold the basic idea to the WG Way Back,
reporting matches was seen as THE way to make life harder for the Bad
Guys (together with indicating the POSTED point, of course).

>OTOH you say some sites use various kinds of MISMATCH.  These
>sites should know when they have a match, but they can't say
>so at the moment.  That's how we got Henry's (?) proposal WSP
>as path separator for matches, later replaced by "!!".

There were various notations proposed en route. "!!" was seen as the
shortest (for what was expected to be the commonest case), and the safest
(as not conflicting with present usage).

>Back to square one.  That's perfectly logical from my POV, if
>you want MISMATCH you should also offer MATCH, and from there
>you arrive at SEEN for sites wishing to use something that's
>(not really) better than only "!".

The only benefit of MATCH is to make a tidy and consistent set of
keywords. For practical purposes, the shorter "!!" is much better, for the
same meaning.

To: [email-masked]
Xref: clerew local.usefor:23009
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Mon, 23 Jan 2006 12:28:02 GMT
Lines: 37
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Right, I agree with this.  "demon" shouldn't be a general SHOULD NOT.  At
>most, the document could say something like "SHOULD NOT be used unless was
>already in use prior to this standard" (in some way that's less clunky).

Well the text in USEPRO (which I thought we had agreed at least as far as
this particular bit is concerned, though there are other issues with the
precise wording) is:

   <Path-identity>s can take the following forms (in decreasing order of
   preference):

   1. A fully qualified domain name (FQDN)....

   2. Some other (arbitrary) name believed to be unique and registered
      at least with all other news servers sending articles directly to
      the given one. This option SHOULD NOT be used unless the earlier
      option is unavailable (e.g. because the server in question is not
      connected to the Internet), or unless it is of longstanding usage
      and cessation would be unduly disruptive, or unless the earlier
      option is provided as well.

Note that <bareword>s (which is the term I prefer to "legacy") also turn
up in various specialized usages such as 'cybercancel'. Other specialized
needs may appear in the future.

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Injection-Info and <sender>
Organization: The Eyrie
Date: Mon, 23 Jan 2006 09:09:24 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> No, authorization is the one that you really want.  Authentication
>> identity is rarely useful and mostly internal to the site, if it's
>> different (it usually won't be).  However, there are circumstances
>> involving role accounts where one might wish to provide both.
>> (Consider newgroup messages posted through a news role account, for
>> instance.)

> Hmmmmm! See http://www.imc.org/ietf-usefor/mail-archive/msg00934.html
> and the messages in the thread immediately preceding it, from which I
> thought we had agreed that "authentication" was the one we needed as a
> separate parameter, and that the "authorization" would be better in the
> posting-account (and would probably be related to the Sender: of the
> article rather than the From:).

Oh, I see, you meant something else by "authentication is the one we
really want" than what I thought.  I thought you meant that in the sense
that it was the more important value, which isn't the case; you instead
meant that it was the one that should be a separate parameter.  That I
agree with, although really, the naming of the parameters is unimportant
to me as long as it's not horrible and as long as the document clearly
defines it.

The important bit is providing a place to put the authorization parameter,
encrypted, if the site wants to.  Providing an encrypted place for the
authentication parameter as well is possibly useful but not horribly
important to me.

>> A 1:1 mapping should not be required.  We can recommend it for improved
>> spam filtering, but it's not obviously the right thing to do from a
>> privacy standpoint and I don't think we're in a position to document
>> (or even determine) the right thing to do in general.  I'd rather leave
>> that decision to the local news administrator.

> There is wording currently in USEPRO which encourages consistency in the
> usage of Injection-Info parameters:

>       ....  Each injecting agent SHOULD
>       use a consistent form of the Injection-Info header field for all
>       articles emanating from the same or similar origins.

Yes, that's a SHOULD, which isn't a requirement.  The recommendation for a
consistent form is something that I think is on the borderline between a
SHOULD and a simple recommendation, although I would lean a little more
towards the latter due to the privacy issues involved.

> Suppose there is some Troll who is causing problems in some group, and
> who continuously morphs his identity so as to avoid killfiles, or to
> argue with himself, or generally to cause mayhem.
[...]

Yes, yes, I'm familiar with all this; this is old ground.  However, the
privacy issues are still important, and as has been painfully established
in other contexts, consistency of opaque identification is often
sufficient to penetrate a privacy shield.

I am considerably less interested in the use of Injection-Info for
killfiles (something that I frankly think is unlikely given the hideous
syntax that won't work with existing killfile tools) than I am in its use
for trace information, which should be the *primary* purpose.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: path-legacy (bareword) and toplabel in A ~ B
Date:  Mon, 23 Jan 2006 14:35:43 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 45
Sender: [email-masked]

Charles Lindsey wrote:

>>> Legacy is not the same thing as deprecated syntax.

Yes, e.g. I'm using a "legacy" browser (pre HTML 4 + CSS) with
"legacy" charsets (not Unicode) and "legacy" SSL3 (no TLS 1.1).
So far nothing's wrong with that.

> But it seems not everyone understands the word "legacy" in
> that way. For example, Frank seems to think it is equivalent
> to SHOULD NOT, which appears to be further than you want to
> go.

New browsers SHOULD NOT only support [see above], only old
browsers have an excuse.

> Seems to me that "legacy" is a word we should avoid using,

IBTD, it's perfectly clear.  While we do and can protect a
legacy path-identity like "demon", we cannot guarantee this
protection for "demon.cat" (anything with a dot) or "feed"
(hex. digits).

Therefore it's our duty to say that this might not work as
expected, and while we're at it it's also clear that "demon"
is not protected from other systems claiming to be "demon".

All we can guarantee is that a fully qualified host name
with at least one dot can't cause trouble for the legacy
"demon".

> we should make our intentions as to future use clear by
> explicit verbiage (either here or in USEPRO).

It's no question of _future_ use, any "demon.cat" unrelated
to TLD cat (now online for a few days) would be in trouble,
no matter how old it is.

This is the same discussion as with <id-domain> in <msg-id>.

A namespace is a namespace, if you want a "better" solution
let IANA start a name-registry, if they can do dictionaries
they can also do UUCP maps.
                            Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:23007
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Info and <sender>
Date: Mon, 23 Jan 2006 12:05:58 GMT
Lines: 64
Sender: [email-masked]

>> Hmm! I thought earlier discussions indicated that 'authentication' was
>> the one we really wanted, so I would like to hear a good reason why we
>> might want to provide both.

>No, authorization is the one that you really want.  Authentication
>identity is rarely useful and mostly internal to the site, if it's
>different (it usually won't be).  However, there are circumstances
>involving role accounts where one might wish to provide both.  (Consider
>newgroup messages posted through a news role account, for instance.)

Hmmmmm! See http://www.imc.org/ietf-usefor/mail-archive/msg00934.html and
the messages in the thread immediately preceding it, from which I thought
we had agreed that "authentication" was the one we needed as a separate
parameter, and that the "authorization" would be better in the
posting-account (and would probably be related to the Sender: of the
article rather than the From:).

>Yes.

>> with a 1:1 mapping so that a given authentication identity always showed
>> up as the same string when it was shown in an Injection-Info header?

>A 1:1 mapping should not be required.  We can recommend it for improved
>spam filtering, but it's not obviously the right thing to do from a
>privacy standpoint and I don't think we're in a position to document (or
>even determine) the right thing to do in general.  I'd rather leave that
>decision to the local news administrator.

There is wording currently in USEPRO which encourages consistency in the
usage of Injection-Info parameters:

      ....  Each injecting agent SHOULD
      use a consistent form of the Injection-Info header field for all
      articles emanating from the same or similar origins.

Suppose there is some Troll who is causing problems in some group, and who
continuously morphs his identity so as to avoid killfiles, or to argue
with himself, or generally to cause mayhem.

It is exceedingly useful for others on the group to be able to recognize
his in his various guises (not necessarily to find his true identity -
whatever that means - but at least to establish that he was the same guy
as published earlier articles).

An Injection-Info header is one of the tools which, if consistently used,
can aid the detection of such trolls (for sure, many people currently
manage to deduce it based on the present headers, but the more information
provided by the injecting agent, the easier it becomes).

To: [email-masked]
Xref: clerew local.usefor:23008
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Mon, 23 Jan 2006 12:11:35 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>What I don't want to see happening is the users of UUCP names (and there
>are a number besides "demon") ending up infringing a SHOULD NOT.

>Describing what they are doing as "legacy" doesn't put them in a
>position of arguing with barrack-room lawyers about "not conforming to
>standards" -- and it is precisely that which I wish to see avoided.

>>Legacy is not the same thing as deprecated syntax.  Legacy is "people who
>>have been doing this may want to continue for various reasons, but if you
>>have a choice, don't do this in the future."

>me too :)

But it seems not everyone understands the word "legacy" in that way. For
example, Frank seems to think it is equivalent to SHOULD NOT, which
appears to be further than you want to go.

Seems to me that "legacy" is a word we should avoid using, and we should
make our intentions as to future use clear by explicit verbiage (either
here or in USEPRO).

- 

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Organization: The Eyrie
Date: Sat, 21 Jan 2006 11:16:37 -0800
Sender: [email-masked]

Ralph Babel <[email-masked]> writes:
>> Ralph Babel wrote:

>>> why should such a trace header be
>>> restricted to the _injection_ server?

>> In practice, that's the only place
>> that anyone really cares about.

> Would it hurt to call it "Trace" instead of
> "Injection-whatever"? Or maybe even "Received"?

It wouldn't bother me in the slightest to call it Trace.  Received is
probably not a good idea since the format and contents are different (and
the Received header in e-mail is a mess that we don't really want to
emulate).

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Open issues
Date:  Sat, 21 Jan 2006 20:04:02 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 44
Sender: [email-masked]

Hi, I'm starting to get lost again with too many open issues
where I've no clue if they're recorded in the ticket system,
already fixed in the editor's copy, under discussion, or in
some other state.  

Last states seen:

#1155 USEFOR ABNF imports (should be "text proposed" ?)
<http://permalink.gmane.org/gmane.ietf.usenet.format/30341>

#1156 IANA considerations (NNTP-Posting-* not yet ready ?)
<http://permalink.gmane.org/gmane.ietf.usenet.format/30342>

#1157 Control cancel bug (should be "text proposed" ?)
<http://permalink.gmane.org/gmane.ietf.usenet.format/30345>

ISSUE 3.2.14 <sender> (no ticket yet ?  Under discussion)
<http://permalink.gmane.org/gmane.ietf.usenet.format/30351>

#1158 host-value (should be "text proposed" ?)
<http://permalink.gmane.org/gmane.ietf.usenet.format/30391>

ISSUE Newsgroups and Distribution WSP (no ticket yet ?)
<http://permalink.gmane.org/gmane.ietf.usenet.format/30395>

ISSUE Missing [CFWS] (no ticket yet ?  "Text proposed")
<http://permalink.gmane.org/gmane.ietf.usenet.format/30427>

ISSUE fields (no ticket yet ?  "Text proposed")
<http://permalink.gmane.org/gmane.ietf.usenet.format/30451>

#1032 Appendix B (three missing items nominated)
<http://permalink.gmane.org/gmane.ietf.usenet.format/30495>

Maybe it's possible to get rid of all minor issues (more like
nits) in a fresh draft.

A complete #1047 ABNF proposal exists, apparently it has all
required "features":

<http://permalink.gmane.org/gmane.ietf.usenet.format/30512>

                            Bye, Frank

Date: Sat, 21 Jan 2006 17:50:52 +0100
From: [email-masked] (Ralph Babel)
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Sender: [email-masked]

Russ Allbery wrote:

> Ralph Babel wrote:
>
>> why should such a trace header be
>> restricted to the _injection_ server?
>
> In practice, that's the only place
> that anyone really cares about.

Would it hurt to call it "Trace" instead of
"Injection-whatever"? Or maybe even "Received"?

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Organization: The Eyrie
Date: Fri, 20 Jan 2006 15:22:38 -0800
Sender: [email-masked]

>> after thinking this over, I agree with Charles.

> Just to be sure, that's "invisible", as in my first guess what
> Harald's dots are supposed to do, hide the complete diagnostics
> in B!.MISMATCH.A!AA

> Or do you here only agree for MISMATCH, but not B!.SEEN.A!AA ?

Right, and I agree for all of the diagnostics, including SEEN.

> For B!.POSTED!A!AA I hope we have it clear that A!AA is either
> what B got, or B got nothing constructing whatever!not-for-mail
> (i.e. A = whatever path-identity, AA = not-for-mail).

If B receives a post with existing Path content A!AA from host H, my
understanding is that it should be constructing a Path header saying
either:

    B!.POSTED!A!AA

or

    B!.POSTED.H!A!AA

The latter form is probably pointless since it can put that data into
Injection-Info, though.  In either case, the existing Path content should
not be hidden, I think.

> In other words no case of B adding !.POSTED!A! to a given AA,

Right.

> also no !.POSTED.A! added to a given AA.

We could do this, but I don't see a lot of point.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: path-legacy (bareword) and toplabel in A ~ B
Date:  Sat, 21 Jan 2006 00:02:44 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 53
Sender: [email-masked]

Russ Allbery wrote:

> after thinking this over, I agree with Charles.

Just to be sure, that's "invisible", as in my first guess what
Harald's dots are supposed to do, hide the complete diagnostics
in B!.MISMATCH.A!AA

Or do you here only agree for MISMATCH, but not B!.SEEN.A!AA ?

For B!.POSTED!A!AA I hope we have it clear that A!AA is either
what B got, or B got nothing constructing whatever!not-for-mail
(i.e. A = whatever path-identity, AA = not-for-mail).

In other words no case of B adding !.POSTED!A! to a given AA,
also no !.POSTED.A! added to a given AA.

In that line of the discussion the following ABNF should work,
it has among others these features:

1: legal "demon.cat" beats legacy "demon.cat"
2: no legal "demon", any "demon" is legacy (dito cat, aero, etc.)
3: no legal IP-literal, any IPv4 is legacy or <path-diagnostic>
4: no IPv6 outside of <path-diagnostic>
5: Charles's preferred position of the one and only [FWS]
6: remove the !.MATCH and <path-verified> if it's useless

                       Bye, Frank

path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF
path-list       =  *( path-entry [FWS] "!" )
path-entry      =  path-identity / path-verified / path-unverified /
                   path-mismatch / path-posted

path-verified   =  path-identity "!.MATCH"
path-unverified =  path-identity "!.SEEN."     path-diagnostic
path-mismatch   =  path-identity "!.MISMATCH." path-diagnostic
path-posted     =  path-identity "!.POSTED"

path-diagnostic =  IPv6address / IPv4address / path-identity
path-identity   =  ( 1*( label "." ) toplabel ) / path-legacy

path-legacy     =  *( path-component "." ) path-component
path-component  =  1*( ALPHA / DIGIT / "-" / "_" )
tail-entry      =  path-legacy                   ; allows underscore

label           =  letdig [ *( letdig / "-" ) letdig ]
letdig          =  ALPHA / DIGIT                 ; compare RFC 3696

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Organization: The Eyrie
Date: Fri, 20 Jan 2006 11:48:32 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> OTOH, if B was mistaken in supposing it truly (for some value of
> "truly") came from A, then C will not be misled into witholding the
> article from A (who in reality had nothing to do with AA). Therefore,
> the 2nd notation is safer. Observe that sending an article to A
> unnecessarily does no more than waste a little bandwidth, which is no
> problem provided the extent of such mistakes is small.

Yeah, after thinking this over, I agree with Charles.

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Organization: The Eyrie
Date: Fri, 20 Jan 2006 11:47:06 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Who knows? There may well be situations in the future where use of a
> bareword is right and proper - usually when the site in question does
> not have a domain name because, for example, it is using UUCP, or
> carrier pigeons, or whatever else (note that Usenet is not tied to the
> Internet, it just happens that the Internet is currently the most
> convenient medium for transporting it).

I'm quite comfortable with requiring domain names going forward, and I
think that's the right thing to do for any new site.  I think it's highly
unlikely that it will ever again be worthwhile to use an undifferentiated
flat namespace for names.

> No, my understanding of "legacy" is that it is only for continuation of
> some existing usage, and is not to be used in new situations. That is
> going too far in this case where we actually mean, at most, "SHOULD NOT
> use in future".

Whatever distinction you're drawing here is too subtle for me, as both
sentences above sound to me like they're saying the same thing.  If
anything, the second sentence is harsher and too strong; what we want to
say is "SHOULD NOT use in new installations".

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Injection-Info and <sender>
Organization: The Eyrie
Date: Fri, 20 Jan 2006 11:34:11 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> As previously discussed, I'm happy to have two separate fields defined
>> in Injection-Info, one for the authentication identity and one for the
>> authorization identity.  I do expect that this will come up.

> Hmm! I thought earlier discussions indicated that 'authentication' was
> the one we really wanted, so I would like to hear a good reason why we
> might want to provide both.

No, authorization is the one that you really want.  Authentication
identity is rarely useful and mostly internal to the site, if it's
different (it usually won't be).  However, there are circumstances
involving role accounts where one might wish to provide both.  (Consider
newgroup messages posted through a news role account, for instance.)

>> I believe they should both have the same language about how the value
>> SHOULD be encrypted.

> Not so clear what you mean there. Are you saying that the injector
> MAY/MIGHT/SHOULD map the supplied SASL authentication string into
> something not obviously related to it,

Yes.

> with a 1:1 mapping so that a given authentication identity always showed
> up as the same string when it was shown in an Injection-Info header?

A 1:1 mapping should not be required.  We can recommend it for improved
spam filtering, but it's not obviously the right thing to do from a
privacy standpoint and I don't think we're in a position to document (or
even determine) the right thing to do in general.  I'd rather leave that
decision to the local news administrator.

> That would make it much the same as the "posting-account" parameter IMO,
> so what would be the point of having a separate "authentication"
> parameter in that case?

To hold the authentication identity.  posting-account logically would be
the authorization identity.  (Well, I suppose it could be either, but if
it's supposed to be the authentication identity, we need another attribute
for the authorization identity.)

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Organization: The Eyrie
Date: Fri, 20 Jan 2006 11:26:59 -0800
Sender: [email-masked]

Richard Clayton <[email-masked]> writes:

> I have no problem with that view ... and the wording about such things
> supports that [which I thought was settled, despite Frank wanting to
> reopen the issue in another message :(]

> What I don't want to see happening is the users of UUCP names (and there
> are a number besides "demon") ending up infringing a SHOULD NOT.

Right, I agree with this.  "demon" shouldn't be a general SHOULD NOT.  At
most, the document could say something like "SHOULD NOT be used unless was
already in use prior to this standard" (in some way that's less clunky).

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1047
Organization: The Eyrie
Date: Fri, 20 Jan 2006 11:25:41 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> Same here.  Russ said that servers won't like to check matches,
> but maybe still want to insert what they "see" from their POV.
> That's how we got SEEN.

> Something like site!.SEEN.peer!peer!... would be excessively
> verbose, obviously "site" and "peer" agree on the name "peer",
> and site!!peer"... would be shorter.  Short is good for paths,
> less reasons to fold it.

The idea of .SEEN is not for !.SEEN.peer!peer!, but rather for something
like !.SEEN.feeder1.foo.com!news.foo.com! without carrying the implication
of MISMATCH that the server was actually configured with the right Path
header for foo.com and received the wrong thing.  In other words, it's a
way of saying "I don't know what the right Path identity for this server
is, so I don't know if the one it inserted (which didn't match its
configured hostname) is actually wrong or not, but just in case here's my
idea of what its hostname is."

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Organization: The Eyrie
Date: Fri, 20 Jan 2006 11:19:12 -0800
Sender: [email-masked]

>> I see value in providing servers with an *optional* place to record
>> trace information. Servers are going to do it *anyway* whether we
>> standardize a way or not [...], so it improves the situation for
>> everyone to provide a standardized way so that servers will stop
>> screwing with other headers.

> Yes, unfortunately. But why should such a trace header be restricted to
> the _injection_ server?

In practice, that's the only place that anyone really cares about.  The
other trace information goes into the Path header, and is mostly
irrelevant for tracing abuse unless things get to the point of wanting to
depeer a site (rare).

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047
Date:  Fri, 20 Jan 2006 18:49:01 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 87
Sender: [email-masked]

Richard Clayton wrote:

> My impression remains that although there may have been some
> reinvention of wheels for reporting mismatches, there's been
> far less enthusiasm for reporting matches.

Same here.  Russ said that servers won't like to check matches,
but maybe still want to insert what they "see" from their POV.
That's how we got SEEN.

Something like site!.SEEN.peer!peer!... would be excessively
verbose, obviously "site" and "peer" agree on the name "peer",
and site!!peer"... would be shorter.  Short is good for paths,
less reasons to fold it.

And !.MATCH!peer! is still shorter than !.SEEN.peer!peer! (or
a similar !.SEEN!peer!peer! - the question of "visibility" is
not yet settled).

If you say that that's all bogus, then the shortest solution
is of course to get rid of _all_ variants with SEEN or MATCH.

OTOH you say some sites use various kinds of MISMATCH.  These
sites should know when they have a match, but they can't say
so at the moment.  That's how we got Henry's (?) proposal WSP
as path separator for matches, later replaced by "!!".

Back to square one.  That's perfectly logical from my POV, if
you want MISMATCH you should also offer MATCH, and from there
you arrive at SEEN for sites wishing to use something that's
(not really) better than only "!".

> I remain unconvinced by the need for reporting matches -- but
> there seems little value in reiterating that any further.

I was surprised that A or B got much more support than C or D
in Harald's poll.  If all this "diagnostical" stuff is more or
less bogus, why don't we just kill it ?  Maybe keeping POSTED
if that's uncontroversial.

OTOH if we apparently want A or B, then we also should get it
right (syntactically), and not allow just anything determining
all details later in USEPRO.

>> With !.POSTED that task might be a bit easier.
> yes indeed

Okay, apparently we could live with POSTED, let's keep it, and
move the rest to <old-path-diagnostic> or rather <path-legacy>
together with IPv4address literals and other demons.

> if you are examining a single dubious article to assess where
> it has been injected into the system then you can apply
> extremely complex reasoning (and all sorts of other
> databases)

When there still was a public peering database I could at least
try to analyze a path, but today I'd give up immediately (if it
is not the question "how does clara.net normally get de.all ?")

As you said a "MISMATCH" in the path does not really help, and
at best it's an unintended warning "not only this is dubious"
for clueless folks.

> I would expect the POSTED to be entirely misleading!
> So it's a snare and a delusion.

We can remove the complete diagnostics idea, and just stick to
"legal" (FQDN) vs. "legacy" (IPv4, demon, etc.) syntactically:

>> Of course we can also decree that IPs in the path are always
>> "deprecated", and cover this rare practice by <path-legacy>

> I thought we already closed the issue of IPv4 addresses (and
> the use of UUCP names)  :(

We had "at least one dot and toplabel" as syntactical solution.

Russ said "no dot" is not good enough for the legacy side of
things.  Without that trick we're forced to use an ABNF hack,
any future legal "demon.cat" would beat any legacy "demon.cat".

That's a reason to say "legacy" plus/minus "SHOULD NOT".  The
owner of "demon" would know why this doesn't affect his name
(= it's a bare word without dot).
                                     Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: path-legacy (bareword) and toplabel in A ~ B
Date:  Fri, 20 Jan 2006 17:03:49 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 67
Sender: [email-masked]

Charles Lindsey wrote:

> There may well be situations in the future where use of a
> bareword is right and proper - usually when the site in
> question does not have a domain name because, for example,
> it is using UUCP, or carrier pigeons

Then it will solve this hypothetical problem in a proper way,
like publish an RfC how its namespace is supposed to work in
the case of conflicts, and use say "what.ever.invalid" until
that's clear.  What it can't do is simply use "demon.cat" in
TLD cat against the will of the owner of domain "demon.cat".

> note that Usenet is not tied to the Internet

Other nets have their own rules.  A site name "2:24/0" IIRC
won't fly in FTSC nets if it's not related to the region 24
coordinator in zone 2 of FidoNet.

> it just happens that the Internet is currently the most
> convenient medium for transporting it

IBTD, that's not only happenstance.  You could use "proper"
names in in the e164.arpa zone for a notation of phone numbers
compatibile with DNS.  But you MUST NOT abuse phone numbers
without the consent of the owner, that's just what "right and
proper" means.

>> legacy is "people who have been doing this may want to
>> continue for various reasons, but if you have a choice,
>> don't do this in the future."

> No, my understanding of "legacy" is that it is only for
> continuation of some existing usage, and is not to be used
> in new situations

For me "SHOULD NOT" means that you need a _good_ reason if you
violate it, it can have harmful and undesirable side-effects.

As an example it might be no good idea if a major site decides
to pick "demon" as its name.  OTOH another site that already
uses this name has plausible reasons to continue this practice.

We had this perfectly clear for some months:  Anything with a
dot is FQDN or at worst IPv4, and all names without dots are
legacy.  Potential conflicts with new TLDs avoided by adopting
the 2821 "at least one dot" rule.

But apparently that's not good enough, and we have to allow
legacy names with dots.  Destroying our masterplan to avoid
conflicts between legal and legacy names by looking for a dot.

There's no way I know of how "demon" could produce a world map
entry for this name in 2006, and without this it hits a MUST
in s-o-1036.  You said "add MUSTard", I didn't want this, as
far as I'm concerned "path-legacy" is clear enough.

But if you want it in prose then yes, "legacy" means obviously
SHOULD NOT.  That's no question of "barrack lawyers", that's a
question of name spaces, potential abuse, conflicts, and harm:

| a relayer name MUST be either a UUCP name registered in the
| UUCP maps (without any domain suffix such as ".UUCP"), or a
| complete Internet domain name.
                                Bye

Date: Fri, 20 Jan 2006 13:56:35 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Sender: [email-masked]

In message <[email-masked]>, Russ Allbery
<[email-masked]> writes
>> What you have called <path-component> (and I would call <bareword>) is
>> NOT a legacy item. It is a perfectly respectable <path-identity> which
>> people can and should generate whenever a domain-name is not
>> appropriate, for whatever reason (for example 'demon', which Richard
>> Clayton insisted was essential on account of the large number of sites
>> configured to use it).  Use of the name 'legacy' is quite wrong in such
>> situations.
>
>Why?  I have no problem at all with calling demon legacy.  If they were
>setting up their service from scratch right now, they shouldn't use that
>Path identity.

I have no problem with that view ... and the wording about such things
supports that [which I thought was settled, despite Frank wanting to
reopen the issue in another message :(]

What I don't want to see happening is the users of UUCP names (and there
are a number besides "demon") ending up infringing a SHOULD NOT.

Describing what they are doing as "legacy" doesn't put them in a
position of arguing with barrack-room lawyers about "not conforming to
standards" -- and it is precisely that which I wish to see avoided.

>Legacy is not the same thing as deprecated syntax.  Legacy is "people who
>have been doing this may want to continue for various reasons, but if you
>have a choice, don't do this in the future."

me too :)

- -- 
richard                                                   Richard Clayton

Date: Fri, 20 Jan 2006 13:55:33 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1047
Sender: [email-masked]

In message <[email-masked]>, Frank Ellermann
<[email-masked]> writes
>
>Richard Clayton wrote:
>
>>> Yes, "!!" is a new feature, and it will not appear in use
>>> overnight. But it is now around 6 or 7 years since this WG
>>> decided that such a feature was needed (all we are doing now
>>> is fiddling with the details).
> 
>> and in that time, no-one has added it.
>
>This can't be six years old.  My oldest article mentioning "!!"
>(2004-11-09) is apparently <[email-masked]> or
><http://mid.gmane.org/[email-masked]>
>  
>In <http://article.gmane.org/gmane.ietf.usenet.format/27783>
>(2004-11-04) Charles summarized the WG history of this issue.
>
>Anything before 2004 that's not closely related to the first
>WG Last Call in 2002 is somewhat outside of my radar, I've no
>idea how prior incarnations of this WG arrived at the former
>"path punctuation" syntax.

OK, so it's relatively new.

My impression remains that although there may have been some reinvention
of wheels for reporting mismatches, there's been far less enthusiasm for
reporting matches.

>In anti-spam communities "worldwide adoption" is an instant
>killer argument (as a special kind of FUSSP).  But with your
>statistics of about 12,000 machines in the "core plus first
>class" it's not completely impossible.
>
>If different folks apparently tried to invent essentially the
>same "feature", and we manage to _offer_ one "official" way to
>achieve what they want, then this might be better then letting
>them re-invent the same fifth wheel in different ways.

If we end up documenting existing practice for mismatches, or something
that is so close that DNews (or whoever) can make trivial changes to
embrace it, then my objection that this is inventing systems for the
sake of it falls, and I shall be quiet.

However, I remain unconvinced by the need for reporting matches -- but
there seems little value in reiterating that any further.

If we take up another suggestion of just providing a generic system in
USEFOR and leave the actual recommended usages for later then I can live
with that (and I can then argue against the ongoing folly of inventing
usages for the sake of it at a later stage)

>Your argument "it doesn't really help to analyze a path" (at
>least not now) is convincing, but you also said that you used
>some heuristics to get rid of tail-entries in your statistics.
>
>With !.POSTED that task might be a bit easier.

yes indeed

but what I was doing was processing millions of articles in an automated
manner for the purpose of estimating the current size of Usenet; and
having POSTED makes that much easier.

However, if you are examining a single dubious article to assess where
it has been injected into the system then you can apply extremely
complex reasoning (and all sorts of other databases) to the analysis of
the header fields. Where there is wickedness and the forging of core
server identities, I would expect the POSTED to be entirely misleading!

So it's a snare and a delusion.

Usenet injection points are hard to ascertain. Adding syntax to the Path
header field is only going to catch out the clueless.

>  We discussed
>that IP addresses within the path might be dubious, you found
>that that's rare, less that 3% of the "core" in your sample.
>
>A new syntax offering a way to get this "right" for those who
>insist on it is not necessarily a bad idea.  Of course we can
>also decree that IPs in the path are always "deprecated", and
>cover this rare practice by <path-legacy> - then nobody knows
>what it is, an unnamed server or some wannabe-diagnostics.

I thought we already closed the issue of IPv4 addresses (and the use of
UUCP names)  :(

- -- 
richard                                                   Richard Clayton

Date: Fri, 20 Jan 2006 13:31:20 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes

[ delayed response, I fear that USEFOR is not a top-priority item in a
busy week :( ]

>In <[email-masked]> Richard Clayton <[email-masked]> 
>writes:
>
>>In message <[email-masked]>, Charles Lindsey
>><[email-masked]> writes
>>>
>>>But there is also a signficant quantity of sites inserting the IP of where
>>>they received the article from (and there may also be sites inserting the
>>>domain name thereof, but there is no way we can identify how much this
>>>happens). Hence the proposal for a further keyword such as "SEEN"
>
>
>>so unless you believe that any of these machines only peers with two,
>>three or four other machines, then there is NO EVIDENCE of any
>>widespread addition of IPv4 addresses as diagnostics except in the case
>>of adding MISMATCH
>
>>Perhaps Charles could substantiate his "significant quantity of sites"?
>
>In the cases I looked at, the IP address had clearly been added by the
>site to the left (since the site to the left always appeared with some IP
>address to its right - which is essentially what you are reporting).

I see IPv4 addresses a lot as well...  but only right at the edge of
network (at the far right of the path)  Usually there is something else
nearby in the Path header field which is equivalent to POSTED.

My analysis generally tries to discard tails because they don't tell you
much useful information about Usenet size & structure

>However, on doing a reverse DNS, it was always clear that the IP address
>quoted always related to the site to the right. So it seemed that the site
>to the left was routinely reporting (diagnostically) the IP address from
>which it had received the article (without apparently checking whether it
>MATCHed or MISMATCHed), and Russ then confirmed that this practice was
>fairly common.

I didn't see that in my data (but this may be common in tails)

>I found no case where an IP address seemed to have been added by a site in
>order to indicate its own path-identity (though my sample was small), 

There are some sites doing this, they're relatively rare

>and
>on the strength of that the WG decided to disallow IPaddresses as
>Path-identities, reserving them for diagnostics only.

and of course because it is straightforward for sites to address the
problem by finding some sort of textual identity

>>If every site uses the diagnostics then the best that the wicked person
>>can arrange for you to see is this:
>
>> Path: a!!b!1.2.3.4.MISMATCH!!d!5.6.7.8.MISMATCH!f!!g!!h.POSTED.i
>
>>and you conclude that he injected the article at "b" while pretending to
>>be "c".
>
>I do not see any "c" in your example.

apologies, I reworked it a few times to prevent wrapping, "d"

>> However, if "b" is misconfigured (and the IP address actually
>>belongs to "d" then the injection point was "d"). The only way to
>>distinguish these is to examine "1.2.3.4" and form an opinion as to what
>>it is currently being used for. This can be tricky :(
>
>Detecting wickedness is always tricky, but the more clues that are
>provided, and the more sites implement those clues, the easier it becomes.

it is never easy unless the person doing it is clueless :(

- -- 
richard                                                   Richard Clayton

To: [email-masked]
Xref: clerew local.usefor:22986
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Thu, 19 Jan 2006 14:01:23 GMT
Lines: 50
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>hmm... if I have an attack scenario:

>  A-------->B
>            ^
>    M-------

>where M is trying to inject articles to B pretending to be A, then putting 
>M's identifier into the path (B!M!MISMATCH!A) is preventing articles from 
>going to M, but doing nothing about the fact that the attck articles won't 
>be going to A.

Yes, but that problem has always been around, with not much to be done
about it. And none of the proposed notations will prevent it.

>On the other hand, in a misconfiguration scenario:

>  A----->B---->C
>  ^            !
>  +------------+
>(A misconfigured, calls himself AA without telling B first)

>the two paths will be

> B!A!MISMATCH!AA
> B!.MISMATCH.A!AA

>If C has been told that A is now AA, he will not send A either article.
>If C thinks that A is still called A, he will send him the second, but not 
>the first.

OTOH, if B was mistaken in supposing it truly (for some value of "truly")
came from A, then C will not be misled into witholding the article from A
(who in reality had nothing to do with AA). Therefore, the 2nd notation is
safer. Observe that sending an article to A unnecessarily does no more
than waste a little bandwidth, which is no problem provided the extent of
such mistakes is small.

To: [email-masked]
Xref: clerew local.usefor:22985
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Info and <sender>
Date: Thu, 19 Jan 2006 13:02:36 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>As previously discussed, I'm happy to have two separate fields defined in
>Injection-Info, one for the authentication identity and one for the
>authorization identity.  I do expect that this will come up.

Hmm! I thought earlier discussions indicated that 'authentication' was the
one we really wanted, so I would like to hear a good reason why we might
want to provide both.

>I believe they should both have the same language about how the value
>SHOULD be encrypted.

Not so clear what you mean there. Are you saying that the injector
MAY/MIGHT/SHOULD map the supplied SASL authentication string into
something not obviously related to it, with a 1:1 mapping so that a given
authentication identity always showed up as the same string when it was
shown in an Injection-Info header? That would make it much the same as the
"posting-account" parameter IMO, so what would be the point of having a
separate "authentication" parameter in that case?

To: [email-masked]
Xref: clerew local.usefor:22987
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Thu, 19 Jan 2006 14:12:36 GMT
Lines: 46
Sender: [email-masked]

>> What you have called <path-component> (and I would call <bareword>) is
>> NOT a legacy item. It is a perfectly respectable <path-identity> which
>> people can and should generate whenever a domain-name is not
>> appropriate, for whatever reason (for example 'demon', which Richard
>> Clayton insisted was essential on account of the large number of sites
>> configured to use it).  Use of the name 'legacy' is quite wrong in such
>> situations.

>Why?  I have no problem at all with calling demon legacy.  If they were
>setting up their service from scratch right now, they shouldn't use that
>Path identity.

Who knows? There may well be situations in the future where use of a
bareword is right and proper - usually when the site in question does not
have a domain name because, for example, it is using UUCP, or carrier
pigeons, or whatever else (note that Usenet is not tied to the Internet,
it just happens that the Internet is currently the most convenient medium
for transporting it).

OTOH, there is already wording in USEPRO that makes it clear that, when a
site is choosing an identity for itself, a domain-name is the preferred
choice and barewords should only be used where there is a good reason.

>Legacy is not the same thing as deprecated syntax.  Legacy is "people who
>have been doing this may want to continue for various reasons, but if you
>have a choice, don't do this in the future."

No, my understanding of "legacy" is that it is only for continuation of
some existing usage, and is not to be used in new situations. That is
going too far in this case where we actually mean, at most, "SHOULD NOT use
in future".

To: [email-masked]
Xref: clerew local.usefor:22988
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 POLL: Path-diagnostics
Date: Thu, 19 Jan 2006 14:41:24 GMT
Lines: 62
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>To clarify:
>I haven't made a specific proposal so far.
>If I were to make a proposal, I'd like to suggest ABNF like this (fragment):

>  path-element = path-identity / path-diagnostic / old-path-diagnostic

>  path-diagnostic = "." (ALPHA / NUMBER / "-" / "." / "_" / ":")*

>  old-path-diagnostic = (ALPHA / NUMBER / "-" / "." / "_" )*

>with text saying

>  Rules for construction of a path-diagnostic are given in [USEPRO]

No, that is not right because the basic idea that we have been discussing
is that each diagnostic should not only be distinguishable from a
<path-identity>, but should also always be attached to an explanation (i.e. a
keyword) of what it is trying to diagnose (though the exact syntactic
sugar of the attachment is under debate). I see that Frank also seems to
be holding this view.

So if you want to stick to the minimum is USEFOR and leave the actual
mechanisms to USEPRO, then I would suggest something like the following:

path-element = path-identity / path-diagnostic

path-identity = label 1*( "." label ) ; which allows IpV4 addresses for
                                      ; legacy use only with MUSTard
                                      ; against future use
                / bareword ; with suitable syntax for bareword
path-diagnostic = [ path-identity / IpV4address / IpV6address ]
                    ".." keyword
keyword = 1*( ALPHA / DIGIT ) ; or somesuch

and leave it to USEPRO to define whatever <keyword>s were deemed
necessary, with instructions on how/when to generate them, and with a
clear statement of whether/how other keywords could be introduced (e.g.
free use of <keyword>s beginning with "X").

That assumes we don't want diagnostics to be "visible" (we seem to be
coming round to that view). So that would allow stuff like:

Path: site-a!!site-b!site-m..MISMATCH!site-x!123.234.123.234..SEEN!
      site-c!..POSTED!xyzzy!not-for-mail

So anything consisting of or ending with "..<keyword>" is a diagnostic,
everything else is a <path-identity>, and the dead:beef situation only
arises on account of IpV6addresses, and so all-hex <bareword>s are the
only ones that need be avoided.

Date: Fri, 20 Jan 2006 13:51:01 +0100
From: [email-masked] (Ralph Babel)
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Sender: [email-masked]

Russ Allbery wrote:

> Ideally, all servers should be able to just use the
> message ID to look up information in logs, I agree.

Yes. Using Usenet (instead of /var/log) to store local
logging data is a waste of everybody's resources.

> I see value in providing servers with an *optional* place
> to record trace information. Servers are going to do it
> *anyway* whether we standardize a way or not [...],
> so it improves the situation for everyone to provide
> a standardized way so that servers will stop screwing
> with other headers.

Yes, unfortunately. But why should such a trace header
be restricted to the _injection_ server?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1032
Date:  Thu, 19 Jan 2006 17:05:50 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 48
Sender: [email-masked]

Charles Lindsey wrote:

> I received <[email-masked]> as well as
> <[email-masked]> (which is the article I
> am replying to).

Okay, I checked the official list archive for further lost
articles from GMaNe's POV, but apparently the list of losses
is complete:

One "Subject: cmsg cancel" never made it to the list, and
one "Subject: #1032 (was: cmsg cancel)" made it to the list,
but not back to GMaMe:

<http://www.imc.org/ietf-usefor/mail-archive/msg02708.html>

So far that's three additional points for (#1032) appendix B:

== 1: cmsg ==

| The old convention about interpreting all articles with a
| subject starting with a string "cmsg " in the header field
| body as control message was obsoleted by s-o-1036, and it
| has been completely removed by this standard.

With an explanation in 3.1.4 "Subject":

| The header field body of subjects SHOULD NOT be interpreted
| by news servers, it is <unstructured> text.

Complete rant in any desired length up to ten pages quoting
RFC 4096 on demand, but an absolute minimum is stated above.

== 2: path punctuation ==

| The only recognized delimiter in a path is an exclamation
| mark reflecting common practice already stated in s-o-1036.
| This allows underscores in legacy names and colons in IPv6
| address literals.

== 3: path identities ==

| Path identities, formerly known as relayer names, are now
| restricted to complete Internet domain names for hosts.
| This reflects a requirement in s-o-1036.  Legacy constructs
| are still accepted.
                              Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Injection-Info and <sender>
Date:  Thu, 19 Jan 2006 07:48:27 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

Russ Allbery wrote:

> As previously discussed, I'm happy to have two separate
> fields defined in Injection-Info, one for the authentication
> identity and one for the authorization identity.  I do expect
> that this will come up.

> I believe they should both have the same language about how
> the value SHOULD be encrypted.

One option would be to allow only B64 characters, and then let
servers do whatever they like.  For that we'd say B64-encoded
version of something that SHOULD be encrypted, or similar.  Bye

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  "MUST be" "a complete Internet domain name" (was: path-legacy (bareword) and toplabel in A ~ B)
Date:  Thu, 19 Jan 2006 07:38:19 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 43
Sender: [email-masked]

Charles Lindsey wrote:
 
>> without MUSTard, maybe "SHOULD NOT generate <path-legacy>"
>> for readers needing a clue.
 
> No! No! No! No!
 
> What you have called <path-component> (and I would call
> <bareword>) is NOT a legacy item. It is a perfectly
> respectable <path-identity>

IBTD.  All "demon", "wolf", "dead", etc. are legacy when
they occur outside of <tail-entry>.  S-o-1036 5.6 Path says:

| a relayer name MUST be either a UUCP name registered in the
| UUCP maps (without any  domain suffix such as ".UUCP"), or
| a complete Internet domain name.

There are no more UUCP maps in this millennium, and therefore
we get "MUST be" "a complete Internet domain name".  That's
not the case for "demon", it violates a MUST in s-o-1036.  We
are perfectly entitled to say SHOULD NOT and call this legacy.

> I think we need to be liberal in tail entries.

<path-legacy> or Harald'd <o-p-d> _are_ already very liberal.

>> Don't shift the [FWS] before the "!", it doesn't work for
>> the "!!" case.  With "!.MATCH!" instead of "!!" it's
>> possible but IMHO ugly, a single <path-entry> should stay in
>> one line.
 
> Then fix the syntax so that it is allowed. We certainly need
> the [FWS] before the "!".

I certainly do NOT need our ersatz-comma "!" at the begin of
a folded line.  If a server wishes to fold the Path it can do
this after his <path-entry> terminated by ersatz-comma "!".

If you insist on folding before "!" I take it you've given up
on the "!!" shorthand for a "!.MATCH!".  Then shifting [FWS]
resulting in "!.MATCH" [FWS] "!" is no big issue, see above.

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Organization: The Eyrie
Date: Wed, 18 Jan 2006 15:08:46 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> What you have called <path-component> (and I would call <bareword>) is
> NOT a legacy item. It is a perfectly respectable <path-identity> which
> people can and should generate whenever a domain-name is not
> appropriate, for whatever reason (for example 'demon', which Richard
> Clayton insisted was essential on account of the large number of sites
> configured to use it).  Use of the name 'legacy' is quite wrong in such
> situations.

Why?  I have no problem at all with calling demon legacy.  If they were
setting up their service from scratch right now, they shouldn't use that
Path identity.

Legacy is not the same thing as deprecated syntax.  Legacy is "people who
have been doing this may want to continue for various reasons, but if you
have a choice, don't do this in the future."

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Injection-Info and <sender>
Organization: The Eyrie
Date: Wed, 18 Jan 2006 11:01:43 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> But would you be willing to include an "authentication" parameter, for
> use with SASL authentications (which will surely become wqite common
> once the new NNTP spec comes into use)? Again, they might or might not
> be email addresses, but at least a person who does not wish to expose
> himself too far can negotiate some pseudonym for his authentication
> identity.

As previously discussed, I'm happy to have two separate fields defined in
Injection-Info, one for the authentication identity and one for the
authorization identity.  I do expect that this will come up.

I believe they should both have the same language about how the value
SHOULD be encrypted.

To: [email-masked]
Xref: clerew local.usefor:22977
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Wed, 18 Jan 2006 18:16:10 GMT
Lines: 88
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>The stuff with (1) underscores, or (2) leading / trailing
>hyphens, or (3) only digits is <path-legacy> "junk" incl.
><tail-entry>, but not a host name.  That explains it for...

>path-identity   =  ( 1*( label "." ) toplabel ) / path-legacy

>path-legacy     =  *( path-component "." ) path-component
>path-component  =  1*( ALPHA / DIGIT / "-" / "_" )
>tail-entry      =  path-legacy                   ; allows underscore

>...without MUSTard, maybe "SHOULD NOT generate <path-legacy>"
>for readers needing a clue.

No! No! No! No!

What you have called <path-component> (and I would call <bareword>) is NOT
a legacy item. It is a perfectly respectable <path-identity> which people
can and should generate whenever a domain-name is not appropriate, for
whatever reason (for example 'demon', which Richard Clayton insisted was
essential on account of the large number of sites configured to use it).
Use of the name 'legacy' is quite wrong in such situations.

I agree that stuff with leading/trailing oddities (notably '.') would be
legacy, and maybe continuing use of IpV4addresses.

As to what legacy oddities exist out there, I did a trawl through my own
news spool, and found comparatively little.

....!masternews.telia.net.!newsb.telia.net.POSTED!not-for-mail
....!bnewsoutpeer00.bru.ops.eu.uu.nat!$3ef82b3c!133.256.1.103.MISMATCH!not-for-mail
....!demon!pcserviceselectronics.co.uk!paul$
....!easynews!news.alt.net!�׷Mogadon-John׷!

So we have a domain with a spurious '.' at the end from telia.net, a weird
'$3ef82b3c' from bnewsoutpeer00.bru.ops.eu.uu.nat (which itself has a
strange TLD), and a couple of tail-entries, of which one has an illegal '!'
at the end.

So I think we need to be liberal in tail entries.

But my news spool is small. Someone ought to look through a bigger spool.
Here is my Perl script in case others want to use it (it needs to be
embedded in a big 'find').

#!/usr/local/bin/perl -w

use English;
use File::CheckTree;

my $path_entry = "([\\w-]+(\\.[\\w-]+)*|[_\\w-]+)";
my $fws = "(\\n?[ \\t]+)";

$INPUT_RECORD_SEPARATOR = ""; # to get the whole article header
  foreach $arg (@ARGV) {
    if (!validate("$arg -f")) {
      open FILE, "< $arg";
      $head = <FILE>; 
      if ($head =~ /Path:[ \t]+(([^\n]*(\n[ \t]+)?)*)\n/o) {$path = $1;}
      close FILE;
      unless ($path =~ /^($path_entry$fws?!)*$path_entry$/o) {
        print $arg," ",$path,"\n"
      }
    }
  }

>Don't shift the [FWS] before the "!", it doesn't work for the
>"!!" case.  With "!.MATCH!" instead of "!!" it's possible but
>IMHO ugly, a single <path-entry> should stay in one line.

Then fix the syntax so that it is allowed. We certainly need the [FWS]
before the "!". The only issue is whether we need it after as well. BTW,
my Perl script will catch any cases of people using it after, so we shall
see.

To: [email-masked]
Xref: clerew local.usefor:22975
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Info and <sender>
Date: Wed, 18 Jan 2006 16:49:13 GMT
Lines: 43
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Having reviewed the current text in the light of the current discussion, I
>think that the "sender" attribute of the Injection-Info header is a
>mistake and should be removed completely.  "posting-account" is
>sufficient for tracing purposes and can be an addr-spec if some site
>really wants it to be (and that violates a SHOULD, which after a some
>consideration I think is correct).

OK, that is certainly a valid position to take. And I agree that it would
not often be used, because that information is not available to most
injectors.

But would you be willing to include an "authentication" parameter, for use
with SASL authentications (which will surely become wqite common once the
new NNTP spec comes into use)? Again, they might or might not be email
addresses, but at least a person who does not wish to expose himself too
far can negotiate some pseudonym for his authentication identity.

>If a site really wishes to enforce putting an e-mail address for the
>poster known to that particular site into a header (such as for
>internal-only news servers inside an organization), it can enforce the
>contents of the Sender header by rejecting messages with a Sender header
>set to something other than what it wants and adding a Sender header to
>posts that don't have it.  Usually this is a bad idea; I don't see any
>particular purpose served by adding a new machinery in the standard for
>doing something that's only appropriate in a few edge cases.

Posters are currently supposed to include a Sender header when RFC 2822
says they should. I have no great problem with an injecting agent adding
it if it not already present (USEPRO currently permits it, but USEAGE
could address whether it is a GOOD THING or not).

To: [email-masked]
Xref: clerew local.usefor:22974
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Wed, 18 Jan 2006 16:38:34 GMT
Lines: 37
Sender: [email-masked]

>> Are you saying that if you see

>>     !site-a!some.site.MISMATCH!some.sote!........

>> (where "some.sote" is a misconfiguration of the correct "some.site")
>> then you want later sites that peer with some.sight to refrain sending
>> stuff back to them?  I am not sure I agree (sites which misconfigure
>> themselves deserve what they get), but if that were wanted then we would
>> be better with the notation in the present draft

>>     !site-a!some.site!MISMATCH!some.sote!........

>Ah, hm.  Good question.

>You're right, the visibility question isn't as obvious as I thought it
>was.  I'm not sure which of those would be better.

OK, then I think we need to discuss that and decide, because it will
determine the notation that we use.

My view is that things before MISMATCH or SEEN should not be visible in
that sense, but things before POSTED most definitely should.

Date: Wed, 18 Jan 2006 16:02:32 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1047 POLL RESULT: Path-diagnostics
Sender: [email-masked]

I note a lot of discussion following this thread. Still, these are the 
messages I picked up that clearly respond to the poll:

A: A path-diagnostic must be syntactically distinguishable from a 
path-identity. Stuff that isn't a path-identity or a syntactically marked 
path-diagnostic (such as an IP address) should be syntacitcally illegal.

- Seth Breidbart "Preference"
- Frank Ellermann "I could support"
- Richard Clayton "would be closest to my viewpoint"
- Charles Lindsey "Prefer"

B: A path-diagnostic must be syntactically distinguishable from a 
path-identity. Stuff that isn't a path-identity or a syntactically marked 
path-diagnostic (such as an IP address) should be syntacitcally legal, but 
have no standardized meaning.

- Russ Allbery "most realistic"
- Seth Breidbart "also acceptable"
- ??Charles Lindsey "might be moving a little closer to" (unclear)

C: A path-diagnostic need not be syntactically distinguishable from a 
path-identity.

- Russ Allbery "I can live with"

D: None of the above alternatives is close to my opinion.

- Frank Ellermann "I could support"

5 people, 9 opinions. I didn't count myself.

Based on this, C seems to be definitely off the table. And there seems to 
be a strong opinion among the 5 of us that we should make stuff that 
doesn't match our syntax for diagnostics illegal.

Are there further comments or people who want to be listed differently?

                     Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 POLL: Path-diagnostics
Date:  Wed, 18 Jan 2006 15:48:28 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 42
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> To clarify:  I haven't made a specific proposal so far.

You said that "leading dot" might be a part of it, no problem
with that idea from my POV - iff "we" want a <path-diagnostic>.

> path-element = path-identity / path-diagnostic / old-path-diagnostic
> path-diagnostic = "." (ALPHA / NUMBER / "-" / "." / "_" / ":")*

That removes MATCH (aka !!), MISMATCH, POSTED, and SEEN from
the picture.  It allows colons outside of an IPv6address.

It removes the <path-entry> concept limiting [FWS} to the end
of "<path-identity> plus optional <path-disgnostic>" construct.

And it offers no syntactical hint about the semantics of this
<path-diagnostic>, a better name might be <path-opaque-junk>.

> old-path-diagnostic = (ALPHA / NUMBER / "-" / "." / "_" )*

With <path-legacy> (incl. bareword) as part of <path-identity>
<old-path-diagnostic> would be almost never reached by a depth
first left to right parser.  (Only "almost", because you allow
<o-p-d> to be empty, killing the "!!" idea also here).

If you remove <path-legacy> from <path-identity> the bareword
and IPv4address cases would both reach <o-p-d>.

But at least bareword is not necessarily any diagnostic, it's
just a machine name.  So better use <path-legacy> instead of
<o-p-d> for that catch-all construct.

You've dots anywhere in <o-p-d>, the parser must really try to
match "leading dot" first.

I don't like the "path-opaque-junk" <path-diagnostic> aspect of
your proposal, it loses the clear distinction MATCH, MISMATCH,
POSTED, and SEEN with the accompanying restrictions for colons
and [FWS].
                          Bye, Frank

Date: Wed, 18 Jan 2006 14:38:55 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: #1047 POLL: Path-diagnostics
Sender: [email-masked]

--On onsdag, januar 18, 2006 11:27:43 +0000 Charles Lindsey 
<[email-masked]> wrote:

>> Harald's proposal "!.MISMATCH" is clearer and more robust.
>> He and Russ both said they want an "alleged.example.net"
>> visible, in other words "!.MISMATCH!alleged.example.net!".
>
> That is not Harald's proposal as I understood it. I thought he wanted
> something like !.alleged.example.net! with or without a "MISMATCH" tagged
> on somewhere.

To clarify:
I haven't made a specific proposal so far.
If I were to make a proposal, I'd like to suggest ABNF like this (fragment):

  path-element = path-identity / path-diagnostic / old-path-diagnostic

  path-diagnostic = "." (ALPHA / NUMBER / "-" / "." / "_" / ":")*

  old-path-diagnostic = (ALPHA / NUMBER / "-" / "." / "_" )*

with text saying

  Rules for construction of a path-diagnostic are given in [USEPRO]

Personally, I'd like to embed !MISMATCH! in the diagnostic too, mostly 
because it reduces the number of special cases for the grammar, and makes 
the question of "do we have enough keywords" a problem that doesn't block 
USEFOR.

But it's no showstopper for me if people want to keep MISMATCH a separate 
entry in the path - I just don't see the point of doing so.

                     Harald

To: [email-masked]
Xref: clerew local.usefor:22973
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 POLL: Path-diagnostics
Date: Wed, 18 Jan 2006 11:27:43 GMT
Lines: 47
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:
> 
>> "alleged.example.net.MISMATCH" is an FQDN

>Actually it's a potential hostname (LDH-rule as in 3696).

Indeed, but a simple-minded checker just looking at the syntax would
accept it as a FQDN. Which is why I included the words "syntactically an
FQDN".

>> (hence my proposal, taken from a suggestion by Forrest,
>> for "alleged.example.net..MISMATCH").

>Harald's proposal "!.MISMATCH" is clearer and more robust.
>He and Russ both said they want an "alleged.example.net"
>visible, in other words "!.MISMATCH!alleged.example.net!".

That is not Harald's proposal as I understood it. I thought he wanted
something like !.alleged.example.net! with or without a "MISMATCH" tagged
on somewhere.

But in any case, your example should have been
"alleged.example.net!.MISMATCH!"  if you want to leave
"alleged.example.net" visible (but Russ seems to be having second thoughts
on that after my "sote" example).

So I think the question of whether to indicate diagnostics by including
the keyword inside that same path-item, or by its presence as the
immediately following path-item will be decided by whether or not we want
it to be "visible" (to a relaying agent deciding whether to send the
article there). Clearly, we do want it to be so "visible" in the case of
POSTED, so the case to be decided is MISMATCH and SEEN (and MATCH if we
have it).

Date: Wed, 18 Jan 2006 08:40:55 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Russ Allbery <[email-masked]>, [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Sender: [email-masked]

--On tirsdag, januar 17, 2006 09:57:34 -0800 Russ Allbery 
<[email-masked]> wrote:

>> (where "some.sote" is a misconfiguration of the correct "some.site")
>> then you want later sites that peer with some.sight to refrain sending
>> stuff back to them?  I am not sure I agree (sites which misconfigure
>> themselves deserve what they get), but if that were wanted then we would
>> be better with the notation in the present draft
>
>>     !site-a!some.site!MISMATCH!some.sote!........
>
> Ah, hm.  Good question.
>
> You're right, the visibility question isn't as obvious as I thought it
> was.  I'm not sure which of those would be better.

hmm... if I have an attack scenario:

  A-------->B
            ^
    M-------

where M is trying to inject articles to B pretending to be A, then putting 
M's identifier into the path (B!M!MISMATCH!A) is preventing articles from 
going to M, but doing nothing about the fact that the attck articles won't 
be going to A.

B!.MISMATCH.M!A will allow distribution to M. Not sure we care.

On the other hand, in a misconfiguration scenario:

  A----->B---->C
  ^            !
  +------------+
(A misconfigured, calls himself AA without telling B first)

the two paths will be

 B!A!MISMATCH!AA
 B!.MISMATCH.A!AA

If C has been told that A is now AA, he will not send A either article.
If C thinks that A is still called A, he will send him the second, but not 
the first.

But since I don't have much belief in "true verified identity", I don't 
think it matters that much - what B puts next to the MISMATCH is unlikely 
to look anything like a valid name for A.

My NOK 0.50.

                     Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Injection-Info and <sender>
Date:  Tue, 17 Jan 2006 20:08:59 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 30
Sender: [email-masked]

Russ Allbery wrote:

> Having reviewed the current text in the light of the current
> discussion, I think that the "sender" attribute of the
> Injection-Info header is a mistake and should be removed
> completely.  "posting-account" is sufficient for tracing
> purposes

ACK so far, that was also my impression.

> and can be an addr-spec if some site really wants it to be

I considered "verified" / 9*( ALPHA / DIGIT / "+" / "/" / "-" )
to preserve "verified" somehow, but that's probably a bad idea.

> it can enforce the contents of the Sender header by rejecting
> messages with a Sender header set to something other than
> what it wants
[...]
At least we don't explictly support Resent-*, so it's either
Sender or "single-From" without Sender.

> and adding a Sender header to posts that don't have it.
> Usually this is a bad idea;

Also known as 2476bis 8.1 "MAY add Sender".  For the complete
horror trip visit SenderID-PRA, two appeals so far, and I push
for an escalation to the IAB.
                                Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Visibility of <path-diagnostics> (was: path-legacy (bareword) and toplabel in A ~ B)
Date:  Tue, 17 Jan 2006 19:47:38 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 22
Sender: [email-masked]

Russ Allbery wrote:
 
>>     !site-a!some.site!MISMATCH!some.sote!........
 
> Ah, hm.  Good question.
 
> You're right, the visibility question isn't as obvious as I
> thought it was.  I'm not sure which of those would be better.

Toss a coin.  

We can hide some.site by ...!.MISMATCH.some.site!some.sote!...

We might be also able to hide some.sote by something like
...!some.site!.MISMATCH.some.sote!... but that would result
in some syntactical assymetries for the SEEN / MATCH cases.

And we can "hide" nothing, after all any "obscurity" affects
only old sites, new sites could extract the "hidden" info if
they insist on it.
                          Bye, Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Injection-Info and <sender>
Organization: The Eyrie
Date: Tue, 17 Jan 2006 10:10:03 -0800
Sender: [email-masked]

Having reviewed the current text in the light of the current discussion, I
think that the "sender" attribute of the Injection-Info header is a
mistake and should be removed completely.  "posting-account" is
sufficient for tracing purposes and can be an addr-spec if some site
really wants it to be (and that violates a SHOULD, which after a some
consideration I think is correct).

If a site really wishes to enforce putting an e-mail address for the
poster known to that particular site into a header (such as for
internal-only news servers inside an organization), it can enforce the
contents of the Sender header by rejecting messages with a Sender header
set to something other than what it wants and adding a Sender header to
posts that don't have it.  Usually this is a bad idea; I don't see any
particular purpose served by adding a new machinery in the standard for
doing something that's only appropriate in a few edge cases.

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Organization: The Eyrie
Date: Tue, 17 Jan 2006 10:03:41 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> However, we are not supposed to be discussing USEPRO at the present
> time.

Fair point.  I'll queue the discussion of reinjection for that time.

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Organization: The Eyrie
Date: Tue, 17 Jan 2006 09:59:55 -0800
Sender: [email-masked]

John Stanley <[email-masked]> writes:
> Russ Allbery <rra@xxxxxxxxxxxx>:

>> Within the framework that we're discussing, I'm happy about it in the
>> sense that I think it's completely out of scope for us to care.

> The "framework" of the question was concerning you as a user, who has
> just had his news service username and password revealed to the world.
> If you don't care, ok. I do.

I don't think there's any way that I can respond further to this or the
rest of John's message without simply reiterating my previous point.  I
think we just disagree on the proper role of a standard.

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Organization: The Eyrie
Date: Tue, 17 Jan 2006 09:57:34 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> Oh, yes, you do want it to be visible since the most likely cause of a
>> mismatch is a simple misconfiguration and you still want the path to
>> have an effect then.

> Are you saying that if you see

>     !site-a!some.site.MISMATCH!some.sote!........

> (where "some.sote" is a misconfiguration of the correct "some.site")
> then you want later sites that peer with some.sight to refrain sending
> stuff back to them?  I am not sure I agree (sites which misconfigure
> themselves deserve what they get), but if that were wanted then we would
> be better with the notation in the present draft

>     !site-a!some.site!MISMATCH!some.sote!........

Ah, hm.  Good question.

You're right, the visibility question isn't as obvious as I thought it
was.  I'm not sure which of those would be better.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: path-legacy (bareword) and toplabel in A ~ B
Date:  Tue, 17 Jan 2006 18:39:38 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 48
Sender: [email-masked]

Charles Lindsey wrote:

>> my attempt to get rid of <toplabel> might be wrong,
>> <IPv4address> matches <path-identity> without <toplabel>:

> I agree. If we don't want <IPaddress>es (of either flavour)
> as Path identities, then either we have to forbid it by
> syntax (using <toplabel>), or else we have to forbid it by
> MUSTard.

No MUSTard necessary, just some prose explaining why there's
an ambiguity in the ABNF reflecting Russ' proposal:  Each
potential host name is also a potential <path-legacy>.

The stuff with (1) underscores, or (2) leading / trailing
hyphens, or (3) only digits is <path-legacy> "junk" incl.
<tail-entry>, but not a host name.  That explains it for...

path-identity   =  ( 1*( label "." ) toplabel ) / path-legacy

path-legacy     =  *( path-component "." ) path-component
path-component  =  1*( ALPHA / DIGIT / "-" / "_" )
tail-entry      =  path-legacy                   ; allows underscore

...without MUSTard, maybe "SHOULD NOT generate <path-legacy>"
for readers needing a clue.

For <path-diagnostic> we could be more restrictive and forbid
<path-legacy>.  That would also forbid <path-legacy> without
dots, i.e. bare words.  But I don't see the point of this ABNF
exercise, one general SHOULD NOT is clear.

For Harald's proposal "the first match depth first left to
right can resolve ambiguities" all we need is <IPv4address>
before <path-identity> in <path-diagnostic>, that's trivial:

path-diagnostic =  IPv6address / IPv4address / path-identity

Complete proposal apparently satisfying most of Russ' points
in <news://news.gmane.org/[email-masked]>
and <http://mid.gmane.org/[email-masked]>

Don't shift the [FWS] before the "!", it doesn't work for the
"!!" case.  With "!.MATCH!" instead of "!!" it's possible but
IMHO ugly, a single <path-entry> should stay in one line.

                                 Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:22962
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Date: Tue, 17 Jan 2006 12:38:15 GMT
Lines: 90
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>In message <[email-masked]>, Charles Lindsey
><[email-masked]> writes
>>
>>But there is also a signficant quantity of sites inserting the IP of where
>>they received the article from (and there may also be sites inserting the
>>domain name thereof, but there is no way we can identify how much this
>>happens). Hence the proposal for a further keyword such as "SEEN"

>so unless you believe that any of these machines only peers with two,
>three or four other machines, then there is NO EVIDENCE of any
>widespread addition of IPv4 addresses as diagnostics except in the case
>of adding MISMATCH

>Perhaps Charles could substantiate his "significant quantity of sites"?

In the cases I looked at, the IP address had clearly been added by the
site to the left (since the site to the left always appeared with some IP
address to its right - which is essentially what you are reporting).

However, on doing a reverse DNS, it was always clear that the IP address
quoted always related to the site to the right. So it seemed that the site
to the left was routinely reporting (diagnostically) the IP address from
which it had received the article (without apparently checking whether it
MATCHed or MISMATCHed), and Russ then confirmed that this practice was
fairly common.

I found no case where an IP address seemed to have been added by a site in
order to indicate its own path-identity (though my sample was small), and
on the strength of that the WG decided to disallow IPaddresses as
Path-identities, reserving them for diagnostics only.

>>>The reason it has no value, so far as I can see, is that it is
>>>impossible to determine who added the diagnostic (someone who handled
>>>the article afterwards can forge it).
>>
>>Yes, but the wicked guys do not have the ability to add bogus diagnostics
>>"afterwards", and certainly not to make their bogosity visible at all
>>sites receiving the article. The purpose is to make it easier to spot
>>whereabouts in the Path the bogosity started. 

>It doesn't do that :(

>If every site uses the diagnostics then the best that the wicked person
>can arrange for you to see is this:

> Path: a!!b!1.2.3.4.MISMATCH!!d!5.6.7.8.MISMATCH!f!!g!!h.POSTED.i

>and you conclude that he injected the article at "b" while pretending to
>be "c".

I do not see any "c" in your example.

> However, if "b" is misconfigured (and the IP address actually
>belongs to "d" then the injection point was "d"). The only way to
>distinguish these is to examine "1.2.3.4" and form an opinion as to what
>it is currently being used for. This can be tricky :(

Detecting wickedness is always tricky, but the more clues that are
provided, and the more sites implement those clues, the easier it becomes.
If site b or site d (whichever was the true injector) had added a
"POSTED", then it would have been much easier to spot.

>>and a news server that regularly exhibits Rogue behaviour will soon be
>>spotted and be leant on by its peers, or UDPed, or whatever.

>There's even less evidence of this :(  In the past few weeks a case came
>to light where it took a month to prevent someone injecting fake
>moderator approvals. This sort of "Rogue"ness is an order of magnitude
>of less concern :( 

Yes, I saw that case. A month looks like par for the course. You certainly
cannot organize a UDP in less than that.

To: [email-masked]
Xref: clerew local.usefor:22961
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: 3.2.14 : <sender>
Date: Tue, 17 Jan 2006 12:18:52 GMT
Lines: 40
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>> Russ Allbery <[email-masked]> writes:

>>> People who want to reinject messages should remove Injection-Info from
>>> the post when they do so, or rename it if we really want to keep trace
>>> information.

>> Yes, but that operation might conceivably be performed by the injecting
>> agent rather than by the entity submitting the reinjection.

>Reinjection should never be done by the server unless the server is some
>sort of special case that isn't implementing NNTP at all and is instead a
>tightly coupled component of some larger system (in which case it's
>outside the scope of our work).  In the case of NNTP, you cannot safely
>determine when reinjection should be permitted on the server, and
>therefore the decision must be made by the client.

You may well be right there (though the injecting agent still needs to
determine somehow whether that preloaded Injection-Info is a genuine
reinjection by a guy who understands what he is doing, or the product of
some idjit or scammer. But that is a policy issue for that injector.

Currently, USEPRO is written on the basis that the injecting agent will
discard/replace any pre-existing Injection-Info, but there is no reason
why that should not be revisited at the proper time.

However, we are not supposed to be discussing USEPRO at the present time.

To: [email-masked]
Xref: clerew local.usefor:22963
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Tue, 17 Jan 2006 12:43:37 GMT
Lines: 30
Sender: [email-masked]

>>> path-legacy looks like exactly what I'm talking about.

>> That's all, formerly known as "barwword", no dots allowed,
>> but underscore is okay ?

I think Frank's use of the term "path-legacy" to cover the "bareword" case
is unfortunate. I think we have already decided that we need barewords.
Richard was quite insistent that "demon" was necessary in addition to
"news.demon.net" because so many sites had "demon" configured into then -
we even wrote some special wording into USEPRO to that effect.

So please can we reserve the word "legacy" for cases which we do not want
to promote for future use, but which are still likely to turn up and
therefore may need special treatment in our draft.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 POLL: Path-diagnostics
Date:  Tue, 17 Jan 2006 18:03:48 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 36
Sender: [email-masked]

Charles Lindsey wrote:
 
> "alleged.example.net.MISMATCH" is an FQDN

Actually it's a potential hostname (LDH-rule as in 3696).

FQDNs might be anything consisting of lables separated
by dots with certain size limits, each label consisting
of arbitrary octets %x00-FF  (Sorry, I had an overdose
of RfC 4343 recently, but it's really interesting... ;-)

> (hence my proposal, taken from a suggestion by Forrest,
> for "alleged.example.net..MISMATCH").

Harald's proposal "!.MISMATCH" is clearer and more robust.
He and Russ both said they want an "alleged.example.net"
visible, in other words "!.MISMATCH!alleged.example.net!".

> I doubt ICANN would be so stupid to invent "MISMATCH" as
> a TLD (or maybe they wouldn't :-( ).

The latter, it's no question of stupidity,  If they adopt
a policy allowing random TLD registrations by everybody,
then I'm free to register MISMATCH if I feel like it.

For example.de (exists) see the invalid.de (exists) test.de,
the latter is a domain of "Stiftung Warentest", and their
main publication is "test".  This makes perfect sense for
them.  A TLD "MISMATCH" could also make sense for somebody.

We could update 2606 and claim it for our purposes, but just
squat a potential TLD without following "proper" procedures
(after inventing what "proper" is :-) strikes me as bad idea.

                       Bye, Frank

Date: Tue, 17 Jan 2006 09:01:21 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Sender: [email-masked]

Russ Allbery <rra@xxxxxxxxxxxx>:

>Ideally, all servers should be able to just use the message ID to look up
>information in logs, I agree.

Servers don't care. Admins care. If the admin cannot create a simple 
script to take a message id and return all of the data from a local 
database for that id, then he's in the wrong job.

>Within the framework that we're discussing, I'm happy about it in the
>sense that I think it's completely out of scope for us to care.

The "framework" of the question was concerning you as a user, who has
just had his news service username and password revealed to the world.
If you don't care, ok. I do.

>It's obviously a stupid thing to do from a security standpoint, but it is
>emphatically NOT the purpose of a standard to legislate against every
>stupid thing that anyone could possibly do.

You've lept from us saying MUST NOT for one thing all the way to "every 
stupid thing". Nobody is trying to legislate against "every stupid thing"
here.

It's not just "a stupid thing". It's not just an admin thing. It's an 
issue of releasing personal data about a system user in a manner that 
changes the meaning of two other headers that I assume we do care about.

>It is the business of a
>standard to define fields and their uses,

Yes, and implicit in that is defining what it is NOT to be used for.

>and when the use is "local
>information used at the discretion of the server admin," it means exactly
>that.

I would not call putting information into the header of an article that is 
distributed globally, available to anyone who wants to read it, "local", 
in any sense of the word. Perhaps that is the confusion. People think that 
defining a header field to be "local" means that it won't propogate with 
the rest of the article?

>>     ... so why you think
>> that spammers (and worse) won't jump on the Injection-Info header as a
>> source of verified information is beyond me.

> I don't think anything of the sort.

You seem to think that a simple statement that the information is not to 
be used as an email address even if it looks like one is sufficient to 
solve the problem, so apparently you think spammers (and worse) will 
listen to that statement and not jump on the data. If you don't think that
statement will solve the problem, ok, but then why bother saying it?

>If your news server is putting things into Injection-Info that you don't
>like, use a different news server.

Yes, in a perfect world, that's the perfect answer. Unfortunately, it's
an answer that closes the door after the horse has fled the barn. I hope
I don't charge you by the minute for your news connection, because there 
will be an awful lot of minutes being used by 'you' after I publish your
authentication data.

>Making rules in a standard about local
>site behavior isn't going to change the real world.

Then we shall just stop right here and disband this WG, shall we? After
all, at the most basic level, ALL "site behaviour" is local.

To: [email-masked]
Xref: clerew local.usefor:22957
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 POLL: Path-diagnostics
Date: Tue, 17 Jan 2006 11:49:09 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>If we must have them [and just documenting what's out there as being a
>mess would be preferable to inventing YARITWCIFD]...

>... then using the existing deployed scheme would seem to be best way
>forward (.MISMATCH) ... but if there's only one piece of software
>creating them, then maybe that's not necessary.  Understanding that
>point would seem to be useful for the next stage of decision-making.

There are two problems with the existing diagnostics DNEWS-style:

1. "alleged.example.net.MISMATCH" is an FQDN (hence my proposal, taken
from a suggestion by Forrest, for "alleged.example.net..MISMATCH"). But I
doubt ICANN would be so stupid to invent "MISMATCH" as a TLD (or maybe
they wouldn't :-( ).

2. "news.example.net.POSTED: is always added by the genuine site
"news.example.net" (which is a true <path-identity>, not a <diagnostic>).
Nevertheless, present systems seeing then will not realise that they are
not supposed to be sending the article back to news.example.net (unless
they have carefully configured the full news.example.net.POSTED into their
'sys' files as an alias). That is the reason we invented the
"!news.example.net!POSTED!" notation as it appears in the present draft.

I think #2 is an important argument against the present deployed usage. I
agree that #1 is more of a red herring.

To: [email-masked]
Xref: clerew local.usefor:22955
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 POLL: Path-diagnostics
Date: Tue, 17 Jan 2006 11:31:11 GMT
Lines: 39
Sender: [email-masked]

In <EB731E6472DBE44DCEBB6C4C@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>Please indicate whether your point of view matches:

>A: A path-diagnostic must be syntactically distinguishable from a=20
>path-identity. Stuff that isn't a path-identity or a syntactically marked=20
>path-diagnostic (such as an IP address) should be syntacitcally illegal.

I prefer A - assuming that <path-identity> will cover FQDNs and barewords
but not IP addresses.

There might also be some further legacy cases to consider, though I doubt
there is any existing usage of things starting with a dot, or with 2 dots
consecutive. So that might be moving a little closer to B.

>If A or B is the rough consensus of the WG, I'll suggest that a=20
>path-diagnostic always start with a dot; that's a simple marker.

I think an intial dot is a bit too inconspicuous. If you want something
that is clearly not an FQDN, then Forrest published a list of several
alternatives a while back, from which I selected

       !news.example.com..MISMATCH!

as the best (two dots in succession is the marker, but it is still
recognizable to people used to the current DNEWS method). Doesn't work so
well for POSTED, though (how about !..POSTED!).

To: [email-masked]
Xref: clerew local.usefor:22960
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047
Date: Tue, 17 Jan 2006 12:09:47 GMT
Lines: 34
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Richard Clayton wrote:

>>> Yes, "!!" is a new feature, and it will not appear in use
>>> overnight. But it is now around 6 or 7 years since this WG
>>> decided that such a feature was needed (all we are doing now
>>> is fiddling with the details).
> 
>> and in that time, no-one has added it.

>This can't be six years old.  My oldest article mentioning "!!"
>(2004-11-09) is apparently <[email-masked]> or
><http://mid.gmane.org/[email-masked]>

The basic idea is 6 or 7 years old, but the original notation used
assorted special delimiters such as '%', '*', '/'. The "!!" is a
more recent syntactic sugar arising from complaints by Bruce that some
legacy site somewhere might regard (or disregard) those special delimiters
(together with much theological argument about what delimiters 1036
actually allowed, and whether the actual implementation of BNews should be
taken into account). So we stopped the theological argument by going for
"!!", "MISMATCH" and "POSTED".

To: [email-masked]
Xref: clerew local.usefor:22959
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Tue, 17 Jan 2006 12:03:06 GMT
Lines: 31
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>> ACK, and apparently you prefer to have the "some.site" visible,
>> that was my question - I didn't have this feature in my syntax.

>Oh, yes, you do want it to be visible since the most likely cause of a
>mismatch is a simple misconfiguration and you still want the path to have
>an effect then.

Are you saying that if you see

    !site-a!some.site.MISMATCH!some.sote!........

(where "some.sote" is a misconfiguration of the correct "some.site") then
you want later sites that peer with some.sight to refrain sending stuff
back to them? I am not sure I agree (sites which misconfigure themselves
deserve what they get), but if that were wanted then we would be better
with the notation in the present draft

    !site-a!some.site!MISMATCH!some.sote!........

To: [email-masked]
Xref: clerew local.usefor:22956
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 POLL: Path-diagnostics
Date: Tue, 17 Jan 2006 11:36:21 GMT
Lines: 40
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Seth Breidbart <[email-masked]> writes:

>>> A:

>> B is also acceptable.

>> I don't like C.

>>> If A or B is the rough consensus of the WG, I'll suggest that a=20
>>> path-diagnostic always start with a dot; that's a simple marker.

>> However, that has a bad effect on existing path-diagnostics.

>The only way that you're going to not have a bad effect on existing
>path-diagnostics is with C.

I don't follow that. Existing diagnostics appear to be of the form

   !news.example.net.MISMATCH!

which is, syntactically speaking, an FQDN.

Unless you are meaning existing diagnostic usages such as

   !123.123.234.234!

which maybe we need to allow as a 'legacy case'.

To: [email-masked]
Xref: clerew local.usefor:22958
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B (was: #1047 POLL: Path-diagnostics)
Date: Tue, 17 Jan 2006 11:55:14 GMT
Lines: 24
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>-3-

>And my attempt to get rid of <toplabel> might be wrong,
><IPv4address> matches <path-identity> without <toplabel>:

I agree. If we don't want <IPaddress>es (of either flavour) as Path
identities, then either we have to forbid it by syntax (using <toplabel>),
or else we have to forbid it by MUSTard.

OTOH, is we need to allow !123.234.123.234! as a legacy case, then the
MUSTard might be a better way to go about it

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Organization: The Eyrie
Date: Mon, 16 Jan 2006 15:04:33 -0800
Sender: [email-masked]

John Stanley <[email-masked]> writes:
> Russ Allbery <rra@xxxxxxxxxxxx>:

>> Sites should have some protocol-defined place to put a local identity
>> into the headers if they wish to do that for tracing or accountability
>> purposes under their local site policy.  Whether that be an e-mail
>> address, a username, a UID, or some encrypted, base64-encoded string
>> should be entirely up to the local site policy, its privacy policy, and
>> so forth.

> Of course. And such a place exists. It's called the "Message-ID". It is
> a globally unique identifier applied to each article either before or as
> it is injected. Any site that feels a need to track users against the
> articles they post can do so trivially by recording
> message-id/authentication pairs locally, plus whatever other information
> they deem reasonable or necessary or even just convenient.

Ideally, all servers should be able to just use the message ID to look up
information in logs, I agree.  In practice, that can be a pain in the
neck.  I see value in providing servers with an *optional* place to record
trace information.  Servers are going to do it *anyway* whether we
standardize a way or not (there is no standardized way now, and as a
result there are several dozen different unstandardized ways in use), so
it improves the situation for everyone to provide a standardized way so
that servers will stop screwing with other headers.  This is particularly
true because clients that need to do reinjection must be aware of what the
header is named.

> Let's take that to an extreme, ok? I'll run a server that uses
> authentication for it's readers, and I'll let you have an account
> there. Now, _I_ think the "convenient" form for your identity
> information is to put in the unencoded ascii for your username AND
> password. That's convenient for me, but perhaps not so convenient for
> you. Oh, by the way, I didn't bother telling you I was doing that before
> you posted your last article. It wasn't convenient for me to do so. Are
> you happy about this?

Within the framework that we're discussing, I'm happy about it in the
sense that I think it's completely out of scope for us to care.  Local
means local.  It's absolutely none of our business what people decide to
put in such a field from a *protocol* standpoint.

It's obviously a stupid thing to do from a security standpoint, but it is
emphatically NOT the purpose of a standard to legislate against every
stupid thing that anyone could possibly do.  It is the business of a
standard to define fields and their uses, and when the use is "local
information used at the discretion of the server admin," it means exactly
that.

Taking your extreme to its obvious conclusion, we'd have to sprinkle the
document with things like "don't put user passwords into the body of
Usenet articles."  People get to figure out that sort of thing all by
themselves.

> Once you accept that 'convenient and useful for the server admin' aren't
> the sole criteria here, we can start to talk about the other criteria
> like privacy and security.

If you want to write a separate RFC on how to choose trace information in
a way that protects privacy, that may very well be a useful document.
We're working on a document specifying the syntactic and semantic
structure of a news article, and shoehorning that sort of document into
this one would be a mess.  (And, to warn, I'm going to make exactly the
same argument when it comes up in USEPRO and USEAGE.)

> We've just had the discussion in this list about spammers sending junk
> email to MESSAGE IDs because they look like an email address (and yes,
> even though I didn't say "me too" at the time, I've seen a considerable
> amount of spam sent to message ids on my own systems), so why you think
> that spammers (and worse) won't jump on the Injection-Info header as a
> source of verified information is beyond me.

I don't think anything of the sort.

I also don't care when it comes to specifying the syntax and use of header
fields.

If your news server is putting things into Injection-Info that you don't
like, use a different news server.  Making rules in a standard about local
site behavior isn't going to change the real world.  I agree with you up
to the point of not wanting to see any *encouragement* for putting e-mail
addresses into the local authenticated identity field, and even
encouraging sites to use opaque identifiers, but beyond that there could
be any number of special circumstances and it is *really* none of our
business.

Date: Mon, 16 Jan 2006 14:02:06 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1047 POLL: Path-diagnostics
Sender: [email-masked]

>B: A path-diagnostic must be syntactically distinguishable from a 
>path-identity. Stuff that isn't a path-identity or a syntactically marked 
>path-diagnostic (such as an IP address) should be syntacitcally legal, but 
>have no standardized meaning.

If it is important enough to have, then it is important enough to both
DEFINE and DIFFERENTIATE. Leading dot will differentiate. Now it needs to 
be defined. It should have been defined first, just to make sure it has
a need to exist, but at least a definition after the fact will be better 
than none at all.

(Note that this option removes the stuff like MISMATCH and SEEN which are 
path-markers.)

Date: Mon, 16 Jan 2006 13:58:53 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1047
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>But if you omit to inform your peers of the new name/alias, then you run
>the risk of having them insert MISMATCH into the Path.

I can neither prevent them nor force them to insert MISMATCH into the path 
header field. I can only determine what the "true" name of my own system
is. The point remains that _I_ am the one who determines that, not my
peers, and yet WE tell my peers that they are the ones who decide.

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>The point being that if you get an NNTP connection from 12.13.45.67, that 
>site claims to be bozo.example.org, and 12.13.45.67 has a DNS reverse 
>mapping that points to 12-13-45-67.dialup.example.net, the only thing you 
>can sensibly log is the IP address - you can't trust the claimed name, and 
>the reversemap has very limited value.

If you are a server admin who accepts articles from a dialup address, you
need to have some better authentication available than just the reverse 
DNS. If you don't, well, you have NO way of determining "true established 
<path-identity>".

>But an IP address is NOT a "true established identity". It's an IP 
>address.

In your example case, it is much more of a "true established identity"
than anything else.

>And then, both "true" and "established" become very slippery concepts.

Yes. Even when not thinking like an attacker.

Date: Mon, 16 Jan 2006 13:53:27 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Sender: [email-masked]

Russ Allbery <rra@xxxxxxxxxxxx>:

>Sites should have some protocol-defined place to put a local identity into
>the headers if they wish to do that for tracing or accountability purposes
>under their local site policy.  Whether that be an e-mail address, a
>username, a UID, or some encrypted, base64-encoded string should be
>entirely up to the local site policy, its privacy policy, and so forth.

Of course. And such a place exists. It's called the "Message-ID". It is a
globally unique identifier applied to each article either before or as it 
is injected. Any site that feels a need to track users against the 
articles they post can do so trivially by recording 
message-id/authentication pairs locally, plus whatever other information 
they deem reasonable or necessary or even just convenient.

>The information in Injection-Info should not be the same information as
>>From or Sender semantically.  It should be site-specific poster identity
>information in whatever form is convenient and useful for that site.

Let's take that to an extreme, ok? I'll run a server that uses 
authentication for it's readers, and I'll let you have an account there. 
Now, _I_ think the "convenient" form for your identity information is to
put in the unencoded ascii for your username AND password. That's 
convenient for me, but perhaps not so convenient for you. Oh, by the way,
I didn't bother telling you I was doing that before you posted your last 
article. It wasn't convenient for me to do so. Are you happy about this?

Once you accept that 'convenient and useful for the server admin' aren't 
the sole criteria here, we can start to talk about the other criteria like 
privacy and security. A site is free to record anything it wants about the
articles its users post in its own database, in whatever format they want 
to record it. It is when they decide to release user information to the 
public that they cross the line. Injection-Info that contains user 
information crosses that line.

>If it happens to have the form of an e-mail address, well, that's an 
>interesting coincidence from the perspective of the standard, but not 
>something that software is allowed to draw automated conclusions from.

We've just had the discussion in this list about spammers sending junk 
email to MESSAGE IDs because they look like an email address (and yes, 
even though I didn't say "me too" at the time, I've seen a considerable 
amount of spam sent to message ids on my own systems), so why you think 
that spammers (and worse) won't jump on the Injection-Info header as a 
source of verified information is beyond me.

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>But we are not proposing to "modify" any header. We are discussing what
>should be in the Injection-Info at the time it is _created_.

If the Injection-Info header says "here's who really posted this article", 
then it is a de-facto change to the meaning of the other headers. Whether 
or not the contents of the other headers changes, their meaning changes.

>Exactly so. If the entity that posted the article possesses an email
>address (like what would have been put in the Sender header, if present)
>AND if the injecting agent has the ability to determine that (or some
>such) address, then it can go in the Injection-Info as a 'sender'
>parameter.

The question is not CAN it do so, but SHOULD it be allowed to do so. If an 
injection agent is going to say "that's not the real sender, here's who 
really sent this..." then it MUST know that information (not just guess),
and even if it KNOWS it is still a privacy and security issue. AND 
irrelevant as well, since the email address had nothing to do with 
injection, and thus has no place in an Injection-* header.

>If neither of those prequisites applies, then the injecting
>agent will have to use something else.

No, there is no "have to" involved here.

>It is for the injecting agent's admin to decide (and pros, cons and
>privacy implications are discussed in USEAGE).

If we are not the group deciding the contents and meanings of headers, why
are we spending all this time discussing the contents and meanings of the
headers? Yes, sir, we do have a say in prohibiting user identification
being placed in Injection-* headers. It is NOT for the injecting agent's
admin to decide the meaning of the news headers.

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Organization: The Eyrie
Date: Mon, 16 Jan 2006 10:58:23 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> Injection-Info should not be provided by the posting agent, and if it
>> is, the post should be rejected.

> Except in the rare case of reinjection, when you just replace it with a
> new one (maybe renaming the old one with an X-header).

>> People who want to reinject messages should remove Injection-Info from
>> the post when they do so, or rename it if we really want to keep trace
>> information.

> Yes, but that operation might conceivably be performed by the injecting
> agent rather than by the entity submitting the reinjection.

Reinjection should never be done by the server unless the server is some
sort of special case that isn't implementing NNTP at all and is instead a
tightly coupled component of some larger system (in which case it's
outside the scope of our work).  In the case of NNTP, you cannot safely
determine when reinjection should be permitted on the server, and
therefore the decision must be made by the client.

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Organization: The Eyrie
Date: Mon, 16 Jan 2006 10:56:07 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> With the "leading-dot" concept (but see below) we might win the
> leaway for legacy constructs without a leading dot.  Raw idea:

> path-identity  = ( 1*( label "." ) toplabel ) / path-legacy ; as before
> path-legacy    = *( path-component "." ) path-component     ; MODIFIED
> path-component = letdig [ *( letdig / "-" / "_" ) letdig ]  ; allows "_"

> tail-entry     = path-legacy                                ; as before, NOW WITH DOT
> label          = letdig [ *( letdig / "-" ) letdig ]        ; as before

> If dots in <tail-entry> are wrong s/path-legacy/path-component/
> If dots in <tail-entry> are okay my prior proposals were wrong.

> This syntax is ambiguous, all FQDNs are also <path-legacy>,
> but for one or more underscores only <path-legacy> matches.

Yeah, that sounds like what I was thinking about.  It doesn't assign
productions unambiguously to different elements, but it does represent
what's out there now with the addition of diagnostics.

> If you also want to allow adjacent dots for the <path-legacy>
> "like..this" it's more difficult, but not impossible, and we
> solved a similar problem for <id-unique> of a Message-ID.

I don't.  :)  I doubt anyone is using such things in anger, and I have no
problem with it being syntactically invalid and then falling into the
general question of how much leeway one wants to give to syntactically
invalid messages.

> ACK, and apparently you prefer to have the "some.site" visible,
> that was my question - I didn't have this feature in my syntax.

Oh, yes, you do want it to be visible since the most likely cause of a
mismatch is a simple misconfiguration and you still want the path to have
an effect then.

Date: Mon, 16 Jan 2006 11:35:22 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Sender: [email-masked]

>> People who want
>>to reinject messages should remove Injection-Info from the post when they
>>do so, or rename it if we really want to keep trace information.
> 
> 
> Yes, but that operation might conceivably be performed by the injecting
> agent rather than by the entity submitting the reinjection.
> 

Charles, which situation do you feel more likely to occur....

1. Someone knows what they are doing, knows how Usenet works, knows
    about loop avoidance, and knows the reasons to not re-inject but has good
    reason to re-inject, but is incapable of obtaining or writing software to
    remove the header.

OR

2. Someone does not know very well how Usenet works and INCORRECTLY
    thinks that re-injection is going to help propagation and be a service
    to everyone.

OR

3. Someone misconfigures some gatewaying tool and queues up a couple
    of days worth of messages to a newsgroup for re-injection.

OK.

Next up, a discussion of the "First, Do No Harm" portion of the
Hippocratic Oath and its application as an engineering principle
in protocol design.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047
Date:  Mon, 16 Jan 2006 17:01:18 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 73
Sender: [email-masked]

Richard Clayton wrote:

>> Yes, "!!" is a new feature, and it will not appear in use
>> overnight. But it is now around 6 or 7 years since this WG
>> decided that such a feature was needed (all we are doing now
>> is fiddling with the details).
 
> and in that time, no-one has added it.

This can't be six years old.  My oldest article mentioning "!!"
(2004-11-09) is apparently <[email-masked]> or
<http://mid.gmane.org/[email-masked]>
  
In <http://article.gmane.org/gmane.ietf.usenet.format/27783>
(2004-11-04) Charles summarized the WG history of this issue.

Anything before 2004 that's not closely related to the first
WG Last Call in 2002 is somewhat outside of my radar, I've no
idea how prior incarnations of this WG arrived at the former
"path punctuation" syntax.

> I could only join a consensus that it was a Good Thing if I
> saw a committment from some (many?) of the NNTP server
> authors to add it to their code.  If Russ said that it would
> definitely be in INN, then that would make me considerably
> happier

[...] 
> A mechanism that only works when almost everyone adopts would
> have to be fairly impressive to work toward -- and I don't
> think this proposal is especially impressive even when we get
> there :(

In anti-spam communities "worldwide adoption" is an instant
killer argument (as a special kind of FUSSP).  But with your
statistics of about 12,000 machines in the "core plus first
class" it's not completely impossible.

If different folks apparently tried to invent essentially the
same "feature", and we manage to _offer_ one "official" way to
achieve what they want, then this might be better then letting
them re-invent the same fifth wheel in different ways.

Your argument "it doesn't really help to analyze a path" (at
least not now) is convincing, but you also said that you used
some heuristics to get rid of tail-entries in your statistics.

With !.POSTED that task might be a bit easier.  We discussed
that IP addresses within the path might be dubious, you found
that that's rare, less that 3% of the "core" in your sample.

A new syntax offering a way to get this "right" for those who
insist on it is not necessarily a bad idea.  Of course we can
also decree that IPs in the path are always "deprecated", and
cover this rare practice by <path-legacy> - then nobody knows
what it is, an unnamed server or some wannabe-diagnostics.

In your fallback plan you offered ip.MISMATCH.  And IIRC Russ
proposed to add something like "SEEN" for cases where a server
doesn't want to verify what it got (neither MATCH nor MISMATCH)
but still wants to record what it believes.

In another article you wrote:
| sites called "dead" or "beef" can have some problems not of
| their making... but since they don't seem to exist, I can
| live with a scheme that disadvantages them.

That disadvantage should be limited to cases where their peers
don't support the s-o-1036 Path syntax published 12 years ago.
I'm all for adding that point to appendix B and then forget it.

                              Bye, Frank

Date: Mon, 16 Jan 2006 14:16:26 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: "True established identity" (this is not #1047)
Sender: [email-masked]

In message <[email-masked]>,
Harald Tveit Alvestrand <[email-masked]> writes

>The point being that if you get an NNTP connection from 12.13.45.67, that 
>site claims to be bozo.example.org, and 12.13.45.67 has a DNS reverse 
>mapping that points to 12-13-45-67.dialup.example.net, the only thing you 
>can sensibly log is the IP address - you can't trust the claimed name, and 
>the reversemap has very limited value.

without a timestamp, 12.13.45.67 may be less use than it appears (OK,
Usenet propagation speeds are high these days, but if some of the
machines on the path are congested (several hours of queues) then you
may be looking at a half dozen accounts...   and as soon as it is more
than one, chances are that abuse teams are going to move on to a simpler
problem :(

- -- 
richard                                                   Richard Clayton

Date: Mon, 16 Jan 2006 15:12:55 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Sender: [email-masked]

--On mandag, januar 16, 2006 14:40:42 +0100 Frank Ellermann 
<[email-masked]> wrote:

> Your 2822 optional-field example is interesting.  So we _can_
> have some "catch-all" rules and similar ambiguities, as long
> as they come after all matching more specific alternatives.

I think ABNF is actually silent on the subject; the running code seems to 
be "ambiguous grammars are OK, you have to look at the text to figure out 
how to resolve them".

Ugly....

Date: Mon, 16 Jan 2006 15:07:09 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: "True established identity" (this is not #1047)
Sender: [email-masked]

--On mandag, januar 16, 2006 11:18:11 +0000 Charles Lindsey 
<[email-masked]> wrote:

>> I'm curious. What is "the true established <path-identity>" of a
>> source? The name? Well, sites can have any number of names. Which is the
>> "true established" one?
>
> Yes, that is a good point. A peering site may have several aliases, and it
> should suffice that one of them MATCHes. I have made a note to review that
> wording for the next version.
>
>> If my server has one name but I change it and
>> my peers do not change their configs, then the "true established" name
>> is different than what those peers would insert as "the true established"
>> name.
>
> But if you omit to inform your peers of the new name/alias, then you run
> the risk of having them insert MISMATCH into the Path.

The point being that if you get an NNTP connection from 12.13.45.67, that 
site claims to be bozo.example.org, and 12.13.45.67 has a DNS reverse 
mapping that points to 12-13-45-67.dialup.example.net, the only thing you 
can sensibly log is the IP address - you can't trust the claimed name, and 
the reversemap has very limited value.

But an IP address is NOT a "true established identity". It's an IP address.

If you want anything to be logged that is helpful when you're under attack, 
you HAVE to try to think like an attacker.

And then, both "true" and "established" become very slippery concepts.

                  Harald

Date: Mon, 16 Jan 2006 14:02:42 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1047 POLL: Path-diagnostics
Sender: [email-masked]

In message <EB731E6472DBE44DCEBB6C4C@B50854F0A9192E8EC6CDA126>, Harald
Tveit Alvestrand <[email-masked]> writes

>OK, it seems that further discussion is not bringing us much closer to a 
>single point of view on this one. But I think I've seen enough to suggest 
>another poll.
>
>Please indicate whether your point of view matches:
>
>A: A path-diagnostic must be syntactically distinguishable from a 
>path-identity. Stuff that isn't a path-identity or a syntactically marked 
>path-diagnostic (such as an IP address) should be syntacitcally illegal.
>
>B: A path-diagnostic must be syntactically distinguishable from a 
>path-identity. Stuff that isn't a path-identity or a syntactically marked 
>path-diagnostic (such as an IP address) should be syntacitcally legal, but 
>have no standardized meaning.
>
>C: A path-diagnostic need not be syntactically distinguishable from a 
>path-identity.
>
>D: None of the above alternatives is close to my opinion.
>
>If A or B is the rough consensus of the WG, I'll suggest that a 
>path-diagnostic always start with a dot; that's a simple marker.

As noted elsewhere, I think the diagnostics are a snare and delusion
(and although Russ says they keep being reinvented, they don't seem to
be useful enough to stick).

If we must have them [and just documenting what's out there as being a
mess would be preferable to inventing YARITWCIFD]...

... then using the existing deployed scheme would seem to be best way
forward (.MISMATCH) ... but if there's only one piece of software
creating them, then maybe that's not necessary.  Understanding that
point would seem to be useful for the next stage of decision-making.

So to the poll -- of these current alternatives, then if we must have
diagnostics, then A would be closest to my viewpoint.

However, I don't feel very strongly about it, since no-one is going to
machine process this stuff and so it will be processed semantically
instead -- and all sorts of confusion can be dealt with very easily.

I do have some sympathy with what I understand to be John Stanley's view
that sites called "dead" or "beef" can have some problems not of their
making... but since they don't seem to exist, I can live with a scheme
that disadvantages them.

- -- 
richard                                                   Richard Clayton

Date: Mon, 16 Jan 2006 13:46:16 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes
>
>In <[email-masked]> Richard Clayton <[email-masked]> 
>writes:
>
>>At present [though my database of sightings is almost 18 months old], a
>>very small number of sites are adding diagnostics.

I appreciate that things have moved on a little to a poll, but I thought
some figures would be useful (and it's taken a couple of days to find
the time to create them).  I hope the figures I have come up with means
that the chair will excuse any further flogging of this horse.

>>The main diagnostic appears to be "this article arrived from an IP
>>address which does not seem to match the way I have been configured".
>>This is expressed as "MISMATCH".
>
>I think that is sites running DNEWS.
>
>But there is also a signficant quantity of sites inserting the IP of where
>they received the article from (and there may also be sites inserting the
>domain name thereof, but there is no way we can identify how much this
>happens). Hence the proposal for a further keyword such as "SEEN"

I've looked at my database (Jun-Oct 2004) and what was present at that
time in the Path headers.

Looking at who passed articles to whom (and using some heuristics to cut
off the "tail" of the paths [to the right of where POSTED might be])
then, as I have posted before, there are 2556 machines in the "peering
core" (these are machines that pass articles to each other) and 8507
"first class" machines that passed articles to two or more machines in
the peering core (but they were never given articles by the core, so
they are in some real sense "at the edges" of Usenet).

Note that these are machines, NOT sites (which may run multiple
machines). Reduce by about 10-15% for that.

Of the peering core:

        289 recorded a MISMATCH diagnostic at least once

         65 were immediately to the left of an IPv4 address

Of the first class machines

         31 recorded a MISMATCH diagnostic at least once

          5 were immediately to the left of an IPv4 address
        
I also found 820 other machines (which were also "at the edge") which
had IP addresses to the right of their identity.

The analysis so far does not of course reveal how many of these machines
were recording IPv4 addresses of diagnostics and how many were being
passed articles by machines that used an IPv4 address as their site
name.

However, looking at the 65 peering core machines whose identity was
immediately to the left of an IPv4 address, then only 5 of them were to
the left of more than one IPv4 address, and there were so few
occurrences that I can list them all !

        ash.uu.net, 198.5.241.25
        ash.uu.net, 198.6.0.7
        ash.uu.net, 198.6.0.85
        ash.uu.net, 198.6.0.86

        lnewsoutpeer00.lnd.ops.eu.uu.net, 195.129.110.7
        lnewsoutpeer01.lnd.ops.eu.uu.net, 195.129.110.8

        newshub1.wanet.net, 198.6.0.85
        newshub1.wanet.net, 198.6.0.86

        ngpeer.news.aol.com, 205.188.226.97
        ngpeer.news.aol.com, 64.12.151.225
        ngpeer.news.aol.com, 64.12.151.230

        uunet, 198.6.0.115
        uunet, 198.6.0.123
        uunet, 198.6.0.8

so unless you believe that any of these machines only peers with two,
three or four other machines, then there is NO EVIDENCE of any
widespread addition of IPv4 addresses as diagnostics except in the case
of adding MISMATCH

Perhaps Charles could substantiate his "significant quantity of sites"?

I would agree that many sites do currently add IPv4 addresses, but ONLY
in the "tail" of the Path header field (to the right of where POSTED
would occur).  ie: they're not doing it for propagation diagnostics

>>Of course, as I and others have observed before, the most likely
>>explanation is that the configuration is wrong rather than that some
>>wickedness is occurring. However, when wickedness has been detected in
>>some other way and its source is being sought then doubtless the
>>MISMATCH will be one of the clues that is examined.
>
>Yes, I have a NOTE in USEPRO somewhere to the effect that such
>misconfigured sites should clean themselves up.
>
>And yes, the detection of assorted forms of wickedness is the main purpose
>of diagnostics 

This isn't very precise thinking.

Detection of wickedness is done by the site that receives the article
and concludes that it has not come from where it is supposed to come
from.  Putting this information into the path is more about promulgating
the presence of the wickedness -- for the community to consider on its
merits (such as they are). And I don't think the community is able to
form much of an opinion of that wickedness because of the policy issues
I mentioned before ("what does the diagnostic mean"?) and which have
been snipped by Charles

>(though it may also help to expose interesting routeing
>anomalies).

In what sense "interesting" ?

I cannot see that the diagnostics add anything to the information
already present in valid Path header fields. If there is no wickedness,
but merely some suboptimal routeing, then all that is provided is a
handy mapping between machine name and IPv4 address. Since that mapping
could well be many to one or sometimes one to many, it is hardly going
to illuminate anything :(

>>I cannot understand where the value arises :( and I think a very clear
>>case for that value must be made before we add this to the document.
>>There do not appear to be any sites that are currently adding this
>>marker (either as !! or !SEEN!) at present, so it's Yet Another Rfc
>>Invention That Will Confuse Implementors For Decades.
>
>There are sites effectively using "SEEN", but without indicating it as
>such (see above). 

I'm afraid that I can find no evidence of that in my database.

Of course things may have changed in the past 18 months. Perhaps a
pointer to the code fragments in a current news peering program would
help to illuminate when such a change occurred ?

>But it would be better if they only did so when what
>they "SAW" differed from what the sending site claimed, and just used "!!"
>when it matched. 

it would certainly be shorter, but doesn't make it more useful

>Yes, "!!" is a new feature, and it will not appear in use
>overnight. But it is now around 6 or 7 years since this WG decided that
>such a feature was needed (all we are doing now is fiddling with the
>details).

and in that time, no-one has added it.

I could only join a consensus that it was a Good Thing if I saw a
committment from some (many?) of the NNTP server authors to add it to
their code.  If Russ said that it would definitely be in INN, then that
would make me considerably happier

In fact at the moment, most of the time when I have seen !! it can be
put down to incompetent forgery attempts (failing to preload a Path
header field correctly) and as such, !! could be seen as making existing
mechanisms fail :(

>>The reason it has no value, so far as I can see, is that it is
>>impossible to determine who added the diagnostic (someone who handled
>>the article afterwards can forge it).
>
>Yes, but the wicked guys do not have the ability to add bogus diagnostics
>"afterwards", and certainly not to make their bogosity visible at all
>sites receiving the article. The purpose is to make it easier to spot
>whereabouts in the Path the bogosity started. 

It doesn't do that :(

If every site uses the diagnostics then the best that the wicked person
can arrange for you to see is this:

 Path: a!!b!1.2.3.4.MISMATCH!!d!5.6.7.8.MISMATCH!f!!g!!h.POSTED.i

and you conclude that he injected the article at "b" while pretending to
be "c". However, if "b" is misconfigured (and the IP address actually
belongs to "d" then the injection point was "d"). The only way to
distinguish these is to examine "1.2.3.4" and form an opinion as to what
it is currently being used for. This can be tricky :(

However, if only _some_ sites use the diagnostics then you will see

 Path: a!b!c!d!!e!!f!1.2.3.4.MISMATCH!g!!h!!5.6.7.8.MISMATCH!i.POSTED.j

and you can only conclude that the article was injected at

        a, b, c, d, e, f, etc

because the Path may be preloaded with fake !! or fake MISMATCH info

A mechanism that only works when almost everyone adopts would have to be
fairly impressive to work toward -- and I don't think this proposal is
especially impressive even when we get there :(

>If the Bad Guy really wants
>to cover his tracks, then he has to subvert some news server somewhere,

there's much simpler ways to "cover his tracks" as a glance at the
headers from Google News articles would tell you.

This scheme is dealing (somewhat imperfectly) with a problem from 1999

>and a news server that regularly exhibits Rogue behaviour will soon be
>spotted and be leant on by its peers, or UDPed, or whatever.

There's even less evidence of this :(  In the past few weeks a case came
to light where it took a month to prevent someone injecting fake
moderator approvals. This sort of "Rogue"ness is an order of magnitude
of less concern :( 

- -- 
richard                                                   Richard Clayton

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  ISSUE: fields (was: path-legacy (bareword) and toplabel in A ~ B)
Date:  Mon, 16 Jan 2006 14:53:10 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 27
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> fields          =       *(trace
[...]
>                         optional-field)

Ugh, and we inherit this using fields =/ <our-stuff>.

> All the headers will match "optional-field"; it's a matter of giving
> priority in parsing (ugh) to parse "from" as a from-field and not as
> an optional-field.

You just killed all our header fields, because they come after
the <optional-field> catch-all.

> Messy, but not novel.

Okay, proposed fix, we rename our <fields> to <news-fields>,
and replace the "fields =/ <our-stuff>" in draft -06 by a 

            news-fields = <our-stuff> / fields

With that trick <optional-field> is again the last catch-all.

Besides Bill's parser never liked the old fields =/ approach,
it always proposed to join the two parts of the <fields>.  Bye

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: path-legacy (bareword) and toplabel in A ~ B
Date:  Mon, 16 Jan 2006 14:40:42 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 27
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
 
> don't understand this.... in the path
 
>   x!.y!z
 
> do you mean that the parser won't try to see y as a
> path-identity, or that it won't see z?
 
> If it won't see y, that's a Good Thing IMHO; if it won't
> see z, we have a problem.

Yes, I thought your idea was x!.y.z without second ! within
a <path-entry>.  For x!.y!z the z is visible, and if we want
this all is fine.

Your 2822 optional-field example is interesting.  So we _can_
have some "catch-all" rules and similar ambiguities, as long
as they come after all matching more specific alternatives.

Here <IPv4address> before <path-identity> before <path-legacy>
could help to define host names with a simple "at least two
LDH-labels" rule (ducking the <toplabel> issue if that really
is an issue), and a <path-legacy> could be dot-separated junk
of LDH or underscore.  
                          Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:22927
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: 3.2.14 : <sender>
Date: Mon, 16 Jan 2006 11:42:45 GMT
Lines: 63
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>John Stanley <[email-masked]> writes:

>> "Modifying the headers" to claim that it knows "the true email address"
>> for a user INCLUDES Injection-Info header fields.

But we are not proposing to "modify" any header. We are discussing what
should be in the Injection-Info at the time it is _created_.

> In fact, putting such
>> info into that header is incorrect, since the posting is not being done
>> from an email address. The posting is being done by a user logged into
>> some system somewhere; a system at which he need not even necessarily
>> HAVE an email address.

>Sites should have some protocol-defined place to put a local identity into
>the headers if they wish to do that for tracing or accountability purposes
>under their local site policy.  Whether that be an e-mail address, a
>username, a UID, or some encrypted, base64-encoded string should be
>entirely up to the local site policy, its privacy policy, and so forth.

Exactly so. If the entity that posted the article possesses an email
address (like what would have been put in the Sender header, if present)
AND if the injecting agent has the ability to determine that (or some
such) address, then it can go in the Injection-Info as a 'sender'
parameter. If neither of those prequisites applies, then the injecting
agent will have to use something else.

It is for the injecting agent's admin to decide (and pros, cons and
privacy implications are discussed in USEAGE).

>However, I agree with John that it has no business modifying information
>that's already there.

Indeed. Altering/deleting headers by the injecting agent has been a SHOULD
NOT in our drafts for many years (we fought long and hard over that
wording, and that is what we came to).

>  Injection-Info should not be provided by the
>posting agent, and if it is, the post should be rejected.

Except in the rare case of reinjection, when you just replace it with a
new one (maybe renaming the old one with an X-header).

>  People who want
>to reinject messages should remove Injection-Info from the post when they
>do so, or rename it if we really want to keep trace information.

Yes, but that operation might conceivably be performed by the injecting
agent rather than by the entity submitting the reinjection.

To: [email-masked]
Xref: clerew local.usefor:22925
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047
Date: Mon, 16 Jan 2006 11:09:05 GMT
Lines: 50
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>--On torsdag, januar 12, 2006 20:57:54 +0100 Frank Ellermann 
><[email-masked]> wrote:

>You're probably right to translate it into ABNF, Frank.

OK, does that now mean we are agreed that diagnostics need to be
syntactically identifiable (I see that Russ now accepts that point) and we
are moving on to fixing the details (presumably using the keywords
MISMATCH and SEEN, since no other method seems to be on the table)?

Next question. Are MISMATCH and SEEN (plus the '!!') sufficient, or do we
want an open-ended extensible list as Harald suggested at one stage
(nobody else has picked up that idea AFAICS, though I can see it has some
possibilities)? Default answer is stick with just those two.

And then there is the question of the notation for the keywords

version 1:
   Path: downstream.example.com!123.123.123.123!MISMATCH
   !injector.example.com!POSTED!not-for-mail
version 2:
   Path: downstream.example.com!123.123.123.123..MISMATCH
   !injector.example.com!POSTED!not-for-mail

Where version 1 is what we have been using mostly in recent discussions,
and is what is in the present draft, and hence is the default the version
unless we consciously decide to change it. Does anyone want to do that (FX
timidly raises my own hand)?

Once those two questions are decided (or if we stick with the default),
then I can dust down my syntax and post it again.

>I don't know whether Charles means "after" as "prepended afterwards" or "to 
>the right of", which is two opposite meanings; USEPRO-04 says:

Yes, it is easy to confuse oneself when building a list from right to left
:-( .

To: [email-masked]
Xref: clerew local.usefor:22926
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047
Date: Mon, 16 Jan 2006 11:18:11 GMT
Lines: 34
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>   1. It MUST establish the trusted identity of the source of the
>      article ... firstly the true established <path-identity> of
>      the source ...

>I'm curious. What is "the true established <path-identity>" of a
>source? The name? Well, sites can have any number of names. Which is the
>"true established" one?

Yes, that is a good point. A peering site may have several aliases, and it
should suffice that one of them MATCHes. I have made a note to review that
wording for the next version.

> If my server has one name but I change it and
>my peers do not change their configs, then the "true established" name
>is different than what those peers would insert as "the true established"
>name.

But if you omit to inform your peers of the new name/alias, then you run
the risk of having them insert MISMATCH into the Path.

To: [email-masked]
Xref: clerew local.usefor:22928
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Date: Mon, 16 Jan 2006 11:49:09 GMT
Lines: 22
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>I'll also point out that the extremely limited definition of 
>path-diagnostic we do have seems to be readily contradicted by every
>suggested use I've seen. It is something other than the name of a site,
>and yet the main use is to indicate the name of a site -- the "real" name
>of the site an article came from.

The only concrete use that has been proposed for these "diagnostics" is
for identifying the source site and whether it MATCHed or not. For that
reason, I much prefer the term "source-identity" to "diagnostic".

Date: Mon, 16 Jan 2006 11:33:18 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Russ Allbery <[email-masked]>, [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Sender: [email-masked]

--==========C9CD83CD075B5AB28CB9==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

--On 16. januar 2006 01:25 -0800 Russ Allbery <[email-masked]> wrote:

>>> path-legacy looks like exactly what I'm talking about.
>
>> That's all, formerly known as "barwword", no dots allowed,
>> but underscore is okay ?
>
> Well, once you add dots, it looks just like a regular path-identity =
except
> that it may not actually be a valid FQDN.  At that point, you can't
> *syntactically* tell the difference between legacy path entries and the
> path entries we're encouraging.

this is actually fairly common in ABNFs of various kinds.
For instance, the RFC 2822 ABNF has:

fields          =3D       *(trace
                          *(resent-date /
                           resent-from /
                           resent-sender /
                           resent-to /
                           resent-cc /
                           resent-bcc /
                           resent-msg-id))
                        *(orig-date /
                        from /
                        sender /
                       .....
                        keywords /
                        optional-field)

where "optional-field" is

 optional-field  =3D       field-name ":" unstructured CRLF

 field-name      =3D       1*ftext

 ftext           =3D       %d33-57 /               ; Any character except
                        %d59-126                ;  controls, SP, and
                                                ;  ":".

All the headers will match "optional-field"; it's a matter of giving=20
priority in parsing (ugh) to parse "from" as a from-field and not as an=20
optional-field.

The same type of construction can be used for legacy-junk-path-entry or=20
whatever we end up calling it (and we can attach one of the MUST be=20
accepted, SHOULD NOT be generated warning labels to it if we want to).

Messy, but not novel.

                    Harald

--==========C9CD83CD075B5AB28CB9==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========C9CD83CD075B5AB28CB9==========--

Date: Mon, 16 Jan 2006 11:27:26 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B (was: #1047 POLL: Path-diagnostics)
Sender: [email-masked]

--==========D02E56AB2CC03300DC55==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

--On 16. januar 2006 09:58 +0100 Frank Ellermann <[email-masked]> =

wrote:

> -2-
>
> I forgot to mention one disadvantage / advantage of the
> "leading dot" idea:  Old systems not knowing this syntax
> won't "see" any <path-identity> or IP hidden behind a dot.

don't understand this.... in the path

  x!.y!z

do you mean that the parser won't try to see y as a path-identity, or that=20
it won't see z?

If it won't see y, that's a Good Thing IMHO; if it won't see z, we have a=20
problem.
>
> Is that actually a feature or a problem or irrelevant ?

depends on what you mean.....

--==========D02E56AB2CC03300DC55==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========D02E56AB2CC03300DC55==========--

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: path-legacy (bareword) and toplabel in A ~ B
Date:  Mon, 16 Jan 2006 11:28:36 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 92
Sender: [email-masked]

Russ Allbery wrote:

> once you add dots, it looks just like a regular path-identity
> except that it may not actually be a valid FQDN.

Yes, that's why we didn't add dots so far.  We could resulting
in "like a valid host name" vs. "like an FQDN with underscores"

> At that point, you can't *syntactically* tell the difference
> between legacy path entries and the path entries we're
> encouraging.

With the "leading-dot" concept (but see below) we might win the
leaway for legacy constructs without a leading dot.  Raw idea:

path-identity  = ( 1*( label "." ) toplabel ) / path-legacy ; as before
path-legacy    = *( path-component "." ) path-component     ; MODIFIED
path-component = letdig [ *( letdig / "-" / "_" ) letdig ]  ; allows "_"

tail-entry     = path-legacy                                ; as before, NOW WITH DOT
label          = letdig [ *( letdig / "-" ) letdig ]        ; as before

If dots in <tail-entry> are wrong s/path-legacy/path-component/
If dots in <tail-entry> are okay my prior proposals were wrong.

This syntax is ambiguous, all FQDNs are also <path-legacy>,
but for one or more underscores only <path-legacy> matches.

Unfortunately <IPv4address> also matches if we allow dots.

If you want to allow a leading or trailing "-" or "_" in the
<path-component> of <path-legacy> its syntax can be simplified:

path-component = 1*( ALPHA / DIGIT / "-" / "_" )

If you also want to allow adjacent dots for the <path-legacy>
"like..this" it's more difficult, but not impossible, and we
solved a similar problem for <id-unique> of a Message-ID.

-2-

>> I forgot to mention one disadvantage / advantage of the
>> "leading dot" idea:  Old systems not knowing this syntax
>> won't "see" any <path-identity> or IP hidden behind a dot.

>> Is that actually a feature or a problem or irrelevant ?

> I'm not sure I understand.  I assume that we're talking about
> something like:

>     !.MISMATCH!some.site

So far I thought that Harald's idea is "!.some.site.MISMATCH"
( or maybe "!.MISMATCH.some.site", matter of taste ) and then
old systems won't see the hidden "some.site" in this construct.

With your proposal "some.site" is again visible:

> Old systems will interpret both of those as path identities,
> with harmless results for .MISMATCH.

ACK, and apparently you prefer to have the "some.site" visible,
that was my question - I didn't have this feature in my syntax.

Below is my first "Russ' plan B" attempt reflecting this.  I'm
not sure about allowing this broad new "<path-legacy> with dot"
within <path-diagnostic>.  Just disallow it there, maybe ?

path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF
path-list       =  *( path-entry "!" [FWS] )
path-entry      =  path-identity / path-verified / path-unverified /
                   path-mismatch / path-posted

path-verified   =  path-identity "!"             ; an additional "!"
path-unverified =  path-identity "!.SEEN!"     path-diagnostic
path-mismatch   =  path-identity "!.MISMATCH!" path-diagnostic
path-posted     =  path-identity "!.POSTED"

path-diagnostic =  IPv6address / IPv4address / path-identity
path-identity   =  ( 1*( label "." ) toplabel ) / path-legacy

path-legacy     =  *( path-component "." ) path-component
path-component  =  1*( ALPHA / DIGIT / "-" / "_" )
tail-entry      =  path-legacy                   ; allows underscore

label           =  letdig [ *( letdig / "-" ) letdig ]
letdig          =  ALPHA / DIGIT                 ; compare RFC 3696

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Organization: The Eyrie
Date: Mon, 16 Jan 2006 01:25:59 -0800
Sender: [email-masked]

>> path-legacy looks like exactly what I'm talking about.

> That's all, formerly known as "barwword", no dots allowed,
> but underscore is okay ?

Well, once you add dots, it looks just like a regular path-identity except
that it may not actually be a valid FQDN.  At that point, you can't
*syntactically* tell the difference between legacy path entries and the
path entries we're encouraging.

> I forgot to mention one disadvantage / advantage of the
> "leading dot" idea:  Old systems not knowing this syntax
> won't "see" any <path-identity> or IP hidden behind a dot.

> Is that actually a feature or a problem or irrelevant ?

I'm not sure I understand.  I assume that we're talking about something
like:

    !.MISMATCH!some.site

Old systems will interpret both of those as path identities, with harmless
results for .MISMATCH.  I don't think anything is going to be hidden.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  path-legacy (bareword) and toplabel in A ~ B (was: #1047 POLL: Path-diagnostics)
Date:  Mon, 16 Jan 2006 09:58:00 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 41
Sender: [email-masked]

Russ Allbery wrote:

> path-legacy looks like exactly what I'm talking about.

That's all, formerly known as "barwword", no dots allowed,
but underscore is okay ?

Well, if what I thought "A" could be is in fact what you
think "B" could be, then Harald has some leaway for his
Chair-magic.

-2-

I forgot to mention one disadvantage / advantage of the
"leading dot" idea:  Old systems not knowing this syntax
won't "see" any <path-identity> or IP hidden behind a dot.

Is that actually a feature or a problem or irrelevant ?

-3-

And my attempt to get rid of <toplabel> might be wrong,
<IPv4address> matches <path-identity> without <toplabel>:

If we want this ambiguity we should sort the alternatives
of <path-diagnostic> from most specific to less specific:

path-diagnostic =  IPv6address / IPv4address / path-identity

With <toplabel> it's IMO cleaner, but IIRC only Charles
agreed that <toplabel> is a good idea.

Maybe Harald could start a "toplabel-task-force" with
Donald (2606bis) and JohnK (3696 + IDNA).  It's harmless,
only updating a note in RfC 1123, nothing to worry about,
no appeals announced, ICANN will love more potential TLDs,
less bogus queries for all-digit "TLDs", it helps with a
detail in 2821bis...  so why do I feel so queasy ?

                         Bye, Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1047 POLL: Path-diagnostics
Organization: The Eyrie
Date: Sun, 15 Jan 2006 22:33:08 -0800
Sender: [email-masked]

>  [B] 
>> I think this is the most realistic option that still adds
>> path diagnostics (which is something for which there is a
>> clear need and utility, given that people keep independently
>> reinventing them
> [...]
>> There's a bunch of legacy stuff in the Path header that
>> realistically isn't going to go away.

> And how would you add the latter to the syntax ?  For A using
> Harald's "leading dot" idea I get something like the following
> verbose variant without <toplabel>:

Don't you already have it here?

> path-list       =  *( path-entry "!" [FWS] )
> path-entry      =  path-identity / path-verified / path-unverified /
>                    path-mismatch / path-posted

> path-verified   =  path-identity "!.MATCH"
> path-unverified =  path-identity "!." path-diagnostic ".SEEN"
> path-mismatch   =  path-identity "!." path-diagnostic ".MISMATCH"
> path-posted     =  path-identity "!.POSTED"

> path-diagnostic =  path-identity / IPv4address / IPv6address
> path-identity   =  ( 1*( label "." ) label ) / path-legacy

> path-legacy     =  1*( ALPHA / DIGIT / "-" / "_" )
> tail-entry      =  path-legacy                   ; allows underscore

path-legacy looks like exactly what I'm talking about.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 POLL: Path-diagnostics
Date:  Mon, 16 Jan 2006 07:23:10 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 48
Sender: [email-masked]

Russ Allbery wrote:

 [B] 
> I think this is the most realistic option that still adds
> path diagnostics (which is something for which there is a
> clear need and utility, given that people keep independently
> reinventing them
[...]
> There's a bunch of legacy stuff in the Path header that
> realistically isn't going to go away.

And how would you add the latter to the syntax ?  For A using
Harald's "leading dot" idea I get something like the following
verbose variant without <toplabel>:

path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF
path-list       =  *( path-entry "!" [FWS] )
path-entry      =  path-identity / path-verified / path-unverified /
                   path-mismatch / path-posted

path-verified   =  path-identity "!.MATCH"
path-unverified =  path-identity "!." path-diagnostic ".SEEN"
path-mismatch   =  path-identity "!." path-diagnostic ".MISMATCH"
path-posted     =  path-identity "!.POSTED"

path-diagnostic =  path-identity / IPv4address / IPv6address
path-identity   =  ( 1*( label "." ) label ) / path-legacy

path-legacy     =  1*( ALPHA / DIGIT / "-" / "_" )
tail-entry      =  path-legacy                   ; allows underscore

label           =  letdig [ *( letdig / "-" ) letdig ]
letdig          =  ALPHA / DIGIT 

It's possible to get rid of ".MATCH" for Henry's more elegant
"!!" idea, and a terse variant could also eliminate ".SEEN".

To allow more constructs you'd have to replace <path-identity>
in <path-entry> by some <path-deprecated>, starting like this:

path-deprecated =  path-identity / path-other-TBD

Do you have an idea for <path-other-TBD> ?  Harald wrote that
he has an idea, but without seeing it I'm reluctant to support
B, it could turn out to be essentially the same as C.

                           Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 POLL: Path-diagnostics
Date:  Mon, 16 Jan 2006 06:33:39 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 77
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
 
> I've seen enough to suggest another poll.

I could support A and D, but not C.  IFF we want or allow
<path-diagnostic> it should be syntactically distinguishable.

In both ...!x!y!SEEN!... and ...!x!y!MISMATCH!... it's IMO
already good enough:  x must be a <path-identity>, either a
FQDN with a dot, or some legacy given name not matching SEEN,
MISMATCH, or POSTED.  Definitely no IPv4address or IPv6address.

The obscure case FQDN without dot (= TLD) would be treated
like a legacy given name, if somebody would insist on TRAVEL,
CAT, JOBS, MOBI, COOP, AERO, MUSEUM etc. nothing breaks

The y in y!SEEN or y!MISMATCH is (a part of) the diagnostic,
it can be an IPv4address.  Some B-news systems interpreting
colon as delimiter might be confused by an IPv6address, as
discussed elsewhere.  

That's my POV for the state of the art assuming that we want
a diagnostic at all.  Without B-news compatible ersatz-IPv6,
for strict backwards compatibility we'd need a "structured"
Subject: cmsg and several other oddities.
 
> B:
[... starting like A, but then ...]
> Stuff that isn't a path-identity or a syntactically marked
> path-diagnostic (such as an IP address) should be
> syntacitcally legal, but have no standardized meaning.

It's already tricky to get the syntax for A right.  I'm lost
how to add "random garbage" as a third option in addition to
<path-identity> vs. <path-diagnostic>.

> C: A path-diagnostic need not be syntactically 
> distinguishable from a path-identity.

Either that makes no sense, if it's not distinguishable why
talk about it.  Or what you really wanted to propose is to
allow all kinds of random garbage not limited to FQDN, IPv4,
IPv6, legacy names, TLDs, obscure keywords, pseudo-TLDs, etc.

> If A or B is the rough consensus of the WG, I'll suggest
> that a path-diagnostic always start with a dot; that's a
> simple marker.

Is that something like the folowing ?

1:  ...!x!y!SEEN!...     => ...!x!.y!... 
2:  ...!x!y!MISMATCH!... => ...!x!.y.MISMATCH!...
3:  ...!x!POSTED!...     => ...!x!.POSTED!...
4:  ...!x!!...           => ...!x!!...

No obvious problem, you could also pick verbose variants for
(1) adding .SEEN, and for (4) using .MATCH.  Some advantages
of this idea:  

a: It's hard to misinterpret a leading dot.  With a trailing
   dot I'd be less confident, 2821bis might allow it.

b: With a mandatory leading dot .POSTED, and .y.MISMATCH can't
   be confused with a TLD.  They are not used as pseudo-TLD.

c: On that solid base we _could_ s/toblabel/label/ and delete
   <toplabel> - silently permitting IPv4 as <path-identity>.

Whatever it is, let's just get it right.  So far we have "!"
as the only permitted delimiter (as stated in s-o-1036), and
that's our excuse to allow the IPv6 colon.  

Like the removal of "Subject: cmsg" that's a very important
difference from RFC 1036, this must be stated in appendix B.

                          Bye, Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1047 POLL: Path-diagnostics
Organization: The Eyrie
Date: Sun, 15 Jan 2006 21:03:07 -0800
Sender: [email-masked]

Seth Breidbart <[email-masked]> writes:

>> A: A path-diagnostic must be syntactically distinguishable from a
>> path-identity. Stuff that isn't a path-identity or a syntactically
>> marked path-diagnostic (such as an IP address) should be syntacitcally
>> illegal.

> Preference.

> B is also acceptable.

> I don't like C.

>> If A or B is the rough consensus of the WG, I'll suggest that a=20
>> path-diagnostic always start with a dot; that's a simple marker.

> However, that has a bad effect on existing path-diagnostics.

The only way that you're going to not have a bad effect on existing
path-diagnostics is with C.

Date: Sun, 15 Jan 2006 23:53:32 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: #1047 POLL: Path-diagnostics
Sender: [email-masked]

> A: A path-diagnostic must be syntactically distinguishable from a=20
> path-identity. Stuff that isn't a path-identity or a syntactically marked=20
> path-diagnostic (such as an IP address) should be syntacitcally illegal.

Preference.

B is also acceptable.

I don't like C.

> If A or B is the rough consensus of the WG, I'll suggest that a=20
> path-diagnostic always start with a dot; that's a simple marker.

However, that has a bad effect on existing path-diagnostics.

Seth

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1047 POLL: Path-diagnostics
Organization: The Eyrie
Date: Sun, 15 Jan 2006 19:15:31 -0800
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> B: A path-diagnostic must be syntactically distinguishable from a
> path-identity. Stuff that isn't a path-identity or a syntactically
> marked path-diagnostic (such as an IP address) should be syntacitcally
> legal, but have no standardized meaning.

I think this is the most realistic option that still adds path diagnostics
(which is something for which there is a clear need and utility, given
that people keep independently reinventing them -- just because they're
not strongly authenticated doesn't mean they're not useful in tracking
down problems).  There's a bunch of legacy stuff in the Path header that
realistically isn't going to go away.

> C: A path-diagnostic need not be syntactically distinguishable from a 
> path-identity.

I can live with path diagnostics going away completely as too hard to
standardize.

Date: Mon, 16 Jan 2006 04:07:26 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1047 POLL: Path-diagnostics
Sender: [email-masked]

--==========CDDB141D02146C094953==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

OK, it seems that further discussion is not bringing us much closer to a=20
single point of view on this one. But I think I've seen enough to suggest=20
another poll.

Please indicate whether your point of view matches:

A: A path-diagnostic must be syntactically distinguishable from a=20
path-identity. Stuff that isn't a path-identity or a syntactically marked=20
path-diagnostic (such as an IP address) should be syntacitcally illegal.

B: A path-diagnostic must be syntactically distinguishable from a=20
path-identity. Stuff that isn't a path-identity or a syntactically marked=20
path-diagnostic (such as an IP address) should be syntacitcally legal, but=20
have no standardized meaning.

C: A path-diagnostic need not be syntactically distinguishable from a=20
path-identity.

D: None of the above alternatives is close to my opinion.

If A or B is the rough consensus of the WG, I'll suggest that a=20
path-diagnostic always start with a dot; that's a simple marker.

If B is the consensus, I think I'll propose a third category of "stuff in=20
path" for the ABNF - "path-legacy-rubbish" or something similarly=20
descriptive.

                  Harald

--==========CDDB141D02146C094953==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========CDDB141D02146C094953==========--

Date: Fri, 13 Jan 2006 23:08:21 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1047
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

   1. It MUST establish the trusted identity of the source of the
      article ... firstly the true established <path-identity> of
      the source ...

I'm curious. What is "the true established <path-identity>" of a
source? The name? Well, sites can have any number of names. Which is the
"true established" one? If my server has one name but I change it and
my peers do not change their configs, then the "true established" name
is different than what those peers would insert as "the true established"
name. How can it be "true" if it isn't the correct name?

Is it an IP address? No, can't be that, even though that is more unique
than the name. (While a server passing an article over the net to a peer
can have any number of names for that link, it came out of the server on
one IP address, even if there are other addresses it will respond to.)

How do we tell an agent that it MUST do something that isn't well defined?

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Organization: The Eyrie
Date: Fri, 13 Jan 2006 23:01:14 -0800
Sender: [email-masked]

John Stanley <[email-masked]> writes:

> "Modifying the headers" to claim that it knows "the true email address"
> for a user INCLUDES Injection-Info header fields. In fact, putting such
> info into that header is incorrect, since the posting is not being done
> from an email address. The posting is being done by a user logged into
> some system somewhere; a system at which he need not even necessarily
> HAVE an email address.

Sites should have some protocol-defined place to put a local identity into
the headers if they wish to do that for tracing or accountability purposes
under their local site policy.  Whether that be an e-mail address, a
username, a UID, or some encrypted, base64-encoded string should be
entirely up to the local site policy, its privacy policy, and so forth.

> No it is not. It is a basic reason why the injecting agent MUST NOT
> modify the information. It is unwise/insecure/unhelpful and patently
> absurd for it to claim it knows "the true email" address for someone who
> is posting an article, and "a true email address" is completely
> irrelevant.

News servers are permitted to add trace headers to locally originating
posts and have been for years.  Injection-Info is supposed to replace the
ton of unstandardized ones we have now.  That, on its face, is a fine
idea.

However, I agree with John that it has no business modifying information
that's already there.  Injection-Info should not be provided by the
posting agent, and if it is, the post should be rejected.  People who want
to reinject messages should remove Injection-Info from the post when they
do so, or rename it if we really want to keep trace information.  (In the
latter case, maybe it would be worthwhile to specify a separate header
name explicitly for that purpose that's allowed to occur multiple times in
the message, like Past-Injection-Info or something.)

> It makes no difference if the injecting agent is actually modifying the
> From or Sender header fields or inserting a header field that says
> "those other header fields are wrong, here's the correct data...". It is
> the same effect and has the same meaning.

The information in Injection-Info should not be the same information as
>From or Sender semantically.  It should be site-specific poster identity
information in whatever form is convenient and useful for that site.  If
it happens to have the form of an e-mail address, well, that's an
interesting coincidence from the perspective of the standard, but not
something that software is allowed to draw automated conclusions from.
(For instance, nothing should expect to be able to mail that address just
because it looks like an e-mail address.)

Date: Fri, 13 Jan 2006 22:58:01 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>Your response seemed more intent on saying that diagnostics should be
>outlawed entirely than on saying that, if present, they should be
>identifiable as such.

Only if you didn't read my response. Quote:

]USEFOR specifies article syntax, yes? Is there a reason to HAVE a 
]"diagnostic"? If so, then yes, USEFOR must specify a syntax that allows 
]one to tell the difference.

First three sentences of the paragraph.

Only after that did I point out that the "openness" of the question of 
being able to detect diagnostics called into question the need for them. 
If you don't think they need to be differentiable from path-identities, 
then why should they exist outside the path-identity? Do you have some 
"need" to describe? How is the system already in place broken?

If your only "need" is so that colons can be inserted into the path header 
and you think that we need a poorly defined thing called a 
"path-diagnostic" to do that, think again. There are two trivial solutions 
to creating yet another entity that has no definition, but I'll let 
you see if you can come up with them. (And only one of them is the trivial 
"encoding" of IPv6 that doesn't use colons.) Hint: the other one rhymes 
with "rattus flow".

>But if you are agreeing that, if/when present, they should be
>identifiable as such,

I am "agreeing" with nothing. I said what I said. There was a question
and I answered it.

>If nobody disagrees, then perhaps we can now
>proceed to implement that feature (by adding a "SEEN" keyword, or any
>other mechanism people may care to suggest).

I disagree with adding fluff that has no real purpose, and yet another
path-keyword that still isn't a path-diagnostic.

>>If the difference is significant
>>enough to spend time debating like this, then it is important enough to
>>define and differentiate. If not, why are we doing this?

>There I agree with you, and I have indeed proposed syntax to do just that
>(though maybe I had overdone it).

I'll admit that my search of the archive was not exhaustive, considering
it is now the end of a very long day here, but I found nothing proposed
that would actually differentiate between a "path-diagnostic" and a 
"path-identity", other than a path-identity cannot have a colon.

I'll also point out that the extremely limited definition of 
path-diagnostic we do have seems to be readily contradicted by every
suggested use I've seen. It is something other than the name of a site,
and yet the main use is to indicate the name of a site -- the "real" name
of the site an article came from.

It would appear that this is all the dregs of the complicated and 
confusing multi-separator system that once graced our pages, which was
very complicated and didn't solve the problem. Now we're at something
simpler, sort of, which still doesn't solve the problem, but does create
them.

If it solves some burning problem, then it needs to be defined and 
differentiable. As it stands, you can't tell a whosits from a whatsits
and thus it can't solve anything.

Date: Fri, 13 Jan 2006 22:34:02 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>But for sure some injecting agents in some setups can
>determine the/a 'true' email address of the sender whatever the sender
>actually put in his "From:", and such injecting agents _can_ then insert
>that information in an Injection-Info header.

The injecting agent cannot determine the "true" email address. Period.
"An" email address for the user is irrelevant -- it cannot know that
the address is false and thus MUST NOT modify the headers to claim that
it knows the "true" address. You're using the word "true", so I assume
you mean to imply everything that "true" implies.

"Modifying the headers" to claim that it knows "the true email address" 
for a user INCLUDES Injection-Info header fields. In fact, putting such 
info into that header is incorrect, since the posting is not being done 
from an email address. The posting is being done by a user logged into 
some system somewhere; a system at which he need not even necessarily HAVE 
an email address.

>Whether inserting such sender information in an Injection-Info header is
>wise/legal/helpful/whatever is a separate issue

No it is not. It is a basic reason why the injecting agent MUST NOT modify 
the information. It is unwise/insecure/unhelpful and patently absurd
for it to claim it knows "the true email" address for someone who is
posting an article, and "a true email address" is completely irrelevant.

>There is (and always has been, even in this group) a tension between those
>who want to hide the source of an article except by showing information
>that only the injecting ISP can associate with an individual poster/site,

I tire of your twisted interpretations of arguments you disagree with. 
I'm not saying anything should be hidden. I'm saying that the injecting
agent does not know the correct information to put in the headers and 
therefore MUST NOT put it there. Not only does it not know the correct
information, it doesn't know the information it sees in the article is 
incorrect.

>But, as I said, USEAGE is the place to discuss that tension and the pros
>and cons of both views.

No, it is not. We are discussing the jobs of different agents, and telling
an agent that it MUST NOT modify certain headers is well within the scope
of our discussion now. Stop trying to push it off until later, hoping that
it will simply go away as people tire of trying to explain to you the 
problem.

>USEPRO says (and has always said) that injecting agents MUST NOT modify
>headers that were placed there by the poster. But that is not the issue
>under discussion.

It makes no difference if the injecting agent is actually modifying
the From or Sender header fields or inserting a header field that says
"those other header fields are wrong, here's the correct data...". It
is the same effect and has the same meaning.

To: [email-masked]
Xref: clerew local.usefor:22918
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE Missing [CFWS] (was: ...ranting and maybe four missing [CFWS] ?!?)
Date: Fri, 13 Jan 2006 14:05:29 GMT
Lines: 25
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>JFTR:

>>> archive =  "Archive:" SP [CFWS] ("no" / "yes")
>>>            *( [CFWS] ";" [CFWS] archive-param ) [CFWS] CRLF
>{...]
>> injection-info  =  "Injection-Info:" SP [CFWS] path-identity
>>                    *([CFWS] ';' [CFWS] parameter) [CFWS] CRLF

>Plus Charles' proposed modification of the corresponding Note.

I concur. And in both cases it is now entirely consistent with Keith in
RFC 3834.

Date: Fri, 13 Jan 2006 11:50:29 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: #1047
Sender: [email-masked]

--On torsdag, januar 12, 2006 20:57:54 +0100 Frank Ellermann 
<[email-masked]> wrote:

>> the intention has always been that the MISMATCH/SEEN should
>> come _after_ the diagnostic, not before it.
>
> No problem, swap it.  And remove <diagnostics> from POSTED if
> that's not the idea.  I'm only trying to translate this debate
> in ABNF, otherwise I don't see where potential traps might be -
> like IPv4.MISMATCH also matching <path-identity> in Richard's
> fallback plan.

You're probably right to translate it into ABNF, Frank.
I don't know whether Charles means "after" as "prepended afterwards" or "to 
the right of", which is two opposite meanings; USEPRO-04 says:

   1. It MUST establish the trusted identity of the source of the
      article and compare it with the leftmost <path-identity> of the
      Path header field's content. If it matches it MUST then prepend
      its own <path-identity> and a '!!' <path-delimiter> to that
      content. If it does not match then it prepends instead two entries
      to that content; firstly the true established <path-identity> of
      the source followed by a '!', the <path-keyword> "MISMATCH" and a
      further '!', and then, to the left of that, its own <path-
      identity> followed by a '!!' <path-delimiter> as usual.

It took me multiple readings to parse that as something that resulted in 
"foo!!1.2.3.4!MISMATCH!", but now I agree that that's what Charles seems to 
be trying to say.

                     Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  ISSUE Missing [CFWS] (was: ...ranting and maybe four missing [CFWS] ?!?)
Date:  Thu, 12 Jan 2006 21:26:46 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 10
Sender: [email-masked]

JFTR:

>> archive =  "Archive:" SP [CFWS] ("no" / "yes")
>>            *( [CFWS] ";" [CFWS] archive-param ) [CFWS] CRLF
{...]
> injection-info  =  "Injection-Info:" SP [CFWS] path-identity
>                    *([CFWS] ';' [CFWS] parameter) [CFWS] CRLF

Plus Charles' proposed modification of the corresponding Note.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047
Date:  Thu, 12 Jan 2006 20:57:54 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 90
Sender: [email-masked]

Charles Lindsey wrote:

> there are all sorts of things that might appear between the
> POSTED and the <tail-entry>, such as sites you wanted to
> 'alias out', "cybercancel"s, and even assorted hosts between
> you and the true injector (such as 'xyzzy')

Also the re-injection discussed in USEPRO.  Assuming that we
still want some kind of POSTED, Richard proposed to forget it.

> Are you agreeing that the 'diagnostic' needs to be
> recognisable as such, and that the presence of MISMATCH or
> SEEN a reasonable way to go about it. If so, then maybe we
> have a consensus on that and can proceed to the details.

IFF (sic!) we want diagnostics they have to be recognizable,
otherwise they would be bogus.  IIRC we had a poll about "IP
anywhere or only in diagnostics", there aren't too many ways
to get this right.

> I think it a bit premature to be writing syntax if that
> consensus is not there

Without syntax I'm lost.  Harald proposed "diagnostics" months
ago, and our previous ABNF attempts didn't properly reflect the
new concept.

>> path-list       =  *( path-entry "!" [FWS] )

> No, if we are not going to allow [FWS] on both sides of a
> '!', then Russ expressed a preference for having it before
> the '!'

With "!" at the end a top down parser knows that there should
be a folded continuation line.

It also guarantees that the "!!" case doesn't have the FWS in
the middle.  And it allows to take an old very long path, add
a space and be done with that part, before creating your own
Path: <path-entry> "!" line.  All old stuff in the next lines.

You can't swap it, you would either get "!" [FWS] "!" if you
try, or you're thrown back to our old attempts

> so a folded line will always start with a '!'

Yes, Russ said that's what an "!" as operator looks like, but
in a Path it's more like a comma.  You wouldn't put a comma at
the start of the line.

BTW, that's all about limiting the places where an [FWS] is
allowed in a path.  The only way to limit it is "at the end of
a path-entry".

> the intention has always been that the MISMATCH/SEEN should
> come _after_ the diagnostic, not before it.

No problem, swap it.  And remove <diagnostics> from POSTED if
that's not the idea.  I'm only trying to translate this debate
in ABNF, otherwise I don't see where potential traps might be -
like IPv4.MISMATCH also matching <path-identity> in Richard's
fallback plan.

With your changes - ignoring the [FWS] for the moment until we
have a state where it's again possible to do something for it -
I would get:

path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF
path-list       =  *( path-entry "!" [FWS] )
path-entry      =  path-identity / path-verified / path-unverified /
                   path-mismatch / path-posted

path-verified   =  path-identity "!"             ; an additional "!"
path-unverified =  path-identity "!" path-diagnostic "!SEEN"
path-mismatch   =  path-identity "!" path-diagnostic "!MISMATCH"
path-posted     =  path-identity "!POSTED"

path-diagnostic =  path-identity / IPv4address / IPv6address
path-identity   =  ( 1*( label "." ) toplabel ) / path-legacy

path-legacy     =  1*( ALPHA / DIGIT / "-" / "_" )
tail-entry      =  path-legacy                   ; allows underscore

label           =  letdig [ *( letdig / "-" ) letdig ]
letdig          =  ALPHA / DIGIT                 ; compare RFC 3696

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ...ranting and maybe four missing [CFWS] ?!?
Date:  Thu, 12 Jan 2006 19:48:53 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 74
Sender: [email-masked]

Charles Lindsey wrote:

> I agree to the extent that [FWS] in places where folding
> cannot possibly occur in an RFC 2822 problem, and hence RFC
> 2822 is the place to fix it.  Whether it is so broken that
> it needs an immediate erratum rather than waiting for
> 2822-bis I am not so sure.

Many errata are only typos like Ned's pending RfC 2045 "/".
What they accepted for BCP 78 was a missing "s" in "RFCs",
ducking a real question about incomprehensible legalese ;-)

Nothing's "immediate" about this procedure, don't hold your
breath.  The 2069 issue (or PEBKAC) is pending for almost a
year now.

 [3834 paramter + USEFOR token vs. USEFOR archive etc.]
> I think Keith is wrong (and in any case he has failed to
> mention the possibility of [CFWS] inside the <parameter>).

He imports the complete 2045 / 2231 <parameter> construct in
RfC 3834, leaving the "gibbous" internal details in RfC 2231,
tricky.  Yes, it's possible, ignoring a SHOULD in 2822 about
comments and syntactical units for a moment.

> But it is an arguable point, depending on how you interpret
> bits of RFC 822 semantics in the midst of what is otherwise
> based on RFC 2822.

2822 wants to update 822, and an old 822 parser should still
work, the main difference is no NUL outside of quoted pair if
you want to accept all old obs-ceneties.

An interpretation of the [CFWS] mess made it into the security
considerations.

> We have indicated, in a NOTE, how we have interpreted it, so
> our intention should be clear.

For <injection-info>, not for <archive>.  It should be okay for
implementors to take the ABNF as is, put it into some magic
scripts, and get a proper parser without reading a note about
another header field.  At the moment it's illegal to say:

        Archive: yes (CC-BY-SA)

> we could equally well follow Keith's interpretation and amend
> the NOTE accordingly. In that case, the proper syntax would be

> archive =  "Archive:" SP [CFWS] ("no" / "yes")
>            *( [CFWS] ";" [CFWS] archive-param ) [CFWS] CRLF

Yes, with that version "Archive: no (but CC-BY-SA okay)" works,
on a syntactical level.

> injection-info  =  "Injection-Info:" SP [CFWS] path-identity
>                    [CFWS] *(';' [CFWS] parameter) [CFWS] CRLF

I think you want the last [CFWS] next to the <parameter> as in

  injection-info  =  "Injection-Info:" SP [CFWS] path-identity
                     [CFWS] *(';' [CFWS] parameter [CFWS]) CRLF

Otherwise in ... foo=bar ; aaa=bbb white space after foo=bar
before the semicolon would be invalid.  You could also say...

  injection-info  =  "Injection-Info:" SP [CFWS] path-identity
                     *([CFWS] ';' [CFWS] parameter) [CFWS] CRLF

...to get the normal [CFWS] CRLF construct at the end.  I like
a consistent look and feel for ABNF.

                           Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:22915
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047
Date: Thu, 12 Jan 2006 18:05:14 GMT
Lines: 79
Sender: [email-masked]

>> What's out there is a mess of
>> ...!server!diagnostic!....
>> ...!server!server!....
>> ...!server!POSTED!.... (possibly diagnostic before the
>> final-entry, not> sure)

>AFAIK my UA could submit articles as xyzzy!frank (at least in
>a pre-s-o-1036 parallel universe), and that would then result
>in server!POSTED!xyzzy!frank

Yes, there are all sorts of things that might appear between the POSTED
and the <tail-entry>, such as sites you wanted to 'alias out',
"cybercancel"s, and even assorted hosts between you and the true injector
(such as 'xyzzy').

>And your first case ...!server!diagnostic!...

>> Should the syntax bless, curse, grudgingly permit, or
>> ignore these practices?

>...would be cursed, if the diagnostics is an IP, because we
>offer an unambiguous solution with MISMATCH or SEEN.  But I'm
>of course not sure, I don't have a collection of "paths ITW".

Are you agreeing that the 'diagnostic' needs to be recognisable as such,
and that the presence of MISMATCH or SEEN a reasonable way to go about it.
If so, then maybe we have a consensus on that and can proceed to the
details.

But I think it a bit premature to be writing syntax if that consensus is
not there. Nervertheless I shall comment on your syntax.

>path-list       =  *( path-entry "!" [FWS] )

No, if we are not going to allow [FWS] on both sides of a '!', then Russ
expressed a preference for having it before the '!' (so a folded line will
always start with a '!' (after the WSP, of course).

>path-verified   =  path-identity "!"             ; an additional "!"
>path-unverified =  path-identity "!SEEN!"     path-diagnostic
>path-mismatch   =  path-identity "!MISMATCH!" path-diagnostic

OK, so you have omitted "MATCH", and I have no great problem with that.
However, the intention has always been that the MISMATCH/SEEN should come
_after_ the diagnostic, not before it.

>path-posted     =  path-identity "!POSTED!"   path-diagnostic

However, I don't think the thing after a POSTED is a diagnostic. In your
'xyzzy' example you (the owner of the host xyzzy) were asserting that the
article had passed through that host, so it is a <path-identity>.
Likewise, a 'cybercancel' in there has always meant "we pretend this
article passed through a host 'cybercancel'; if you don't want to receive
these cancels, then please arrange that 'cybercancel' is an alias for your
own host".  There is a feature in INN (maybe others) that facilitates that
usage (see description in USEPRO).

>path-diagnostic =  path-identity / IPv4address / IPv6address

Yes, that paves the way for minimizing the dead:beef problem, and the rest
of your syntax seems OK (modulo minor niggles).

To: [email-masked]
Xref: clerew local.usefor:22914
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Date: Thu, 12 Jan 2006 17:28:14 GMT
Lines: 66
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>At present [though my database of sightings is almost 18 months old], a
>very small number of sites are adding diagnostics.

>The main diagnostic appears to be "this article arrived from an IP
>address which does not seem to match the way I have been configured".
>This is expressed as "MISMATCH".

I think that is sites running DNEWS.

But there is also a signficant quantity of sites inserting the IP of where
they received the article from (and there may also be sites inserting the
domain name thereof, but there is no way we can identify how much this
happens). Hence the proposal for a further keyword such as "SEEN"

>Of course, as I and others have observed before, the most likely
>explanation is that the configuration is wrong rather than that some
>wickedness is occurring. However, when wickedness has been detected in
>some other way and its source is being sought then doubtless the
>MISMATCH will be one of the clues that is examined.

Yes, I have a NOTE in USEPRO somewhere to the effect that such
misconfigured sites should clean themselves up.

And yes, the detection of assorted forms of wickedness is the main purpose
of diagnostics (though it may also help to expose interesting routeing
anomalies).

>I cannot understand where the value arises :( and I think a very clear
>case for that value must be made before we add this to the document.
>There do not appear to be any sites that are currently adding this
>marker (either as !! or !SEEN!) at present, so it's Yet Another Rfc
>Invention That Will Confuse Implementors For Decades.

There are sites effectively using "SEEN", but without indicating it as
such (see above). But it would be better if they only did so when what
they "SAW" differed from what the sending site claimed, and just used "!!"
when it matched. Yes, "!!" is a new feature, and it will not appear in use
overnight. But it is now around 6 or 7 years since this WG decided that
such a feature was needed (all we are doing now is fiddling with the
details).

>The reason it has no value, so far as I can see, is that it is
>impossible to determine who added the diagnostic (someone who handled
>the article afterwards can forge it).

Yes, but the wicked guys do not have the ability to add bogus diagnostics
"afterwards", and certainly not to make their bogosity visible at all
sites receiving the article. The purpose is to make it easier to spot
whereabouts in the Path the bogosity started. If the Bad Guy really wants
to cover his tracks, then he has to subvert some news server somewhere,
and a news server that regularly exhibits Rogue behaviour will soon be
spotted and be leant on by its peers, or UDPed, or whatever.

To: [email-masked]
Xref: clerew local.usefor:22912
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ...ranting and maybe four missing [CFWS] ?!? (was: ISSUE: Newsgroups and Distribution WSP)
Date: Thu, 12 Jan 2006 16:56:32 GMT
Lines: 63
Sender: [email-masked]

>> could you clarify which section of the spec you are talking
>> about, and what the issue is?  At the moment, I can't tell
>> whether this is about Control:, Newsgroups:, Distribution:,
>> general philosophy of SP [FWS] or all 4.

>The latter plus a sudden urge to start some kind of ranting.
>I've submitted an erratum for 2822 <unstructured>, see also

I agree to the extent that [FWS] in places where folding cannot possibly
occur in an RFC 2822 problem, and hence RFC 2822 is the place to fix it.
Whether it is so broken that it needs an immediate erratum rather than
waiting for 2822-bis I am not so sure.

>Unrelated, I just stumbled over it, Keith has an explicit
>[CFWS] before <parameter> and before CRLF in RFC 3834.  We
>don't have those two [CFWS] in the case of <archive>.  One
>of these two solutions must be incorrect, 3834 or our idea.

>Dito <injection-info>, two missing [CFWS] _iff_ 3834 has it
>right for its <parameter>.  In our <user-agent> it's only a
><token>, but otherwise that follows Keith's pattern with an
>explicit [CFWS] CRLF at the end and before each <token>.

I think Keith is wrong (and in any case he has failed to mention the
possibility of [CFWS] inside the <parameter>). But it is an arguable point,
depending on how you interpret bits of RFC 822 semantics in the midst of
what is otherwise based on RFC 2822.

We have indicated, in a NOTE, how we have interpreted it, so our intention
should be clear. But we could equally well follow Keith's interpretation
and amend the NOTE accordingly. In that case, the proper syntax would be

   archive         =  "Archive:" SP [CFWS] ("no" / "yes")
                      *( [CFWS] ";" [CFWS] archive-param ) [CFWS] CRLF

   injection-info  =  "Injection-Info:" SP [CFWS] path-identity
                      [CFWS] *(';' [CFWS] parameter) [CFWS] CRLF

and the NOTE would have to read

      NOTE: The syntax of <parameter> ([RFC2045] as amended by
      [RFC2231]), taken in conjunction with the folding rules of
      [RFC0822], effectively allows [CFWS] to occur on either side of
      the "=" inside a <parameter>.

On balance, I think we should do that (it reduces the number of places
where you might possibly be unsure whether [CFWS] was allowed).

To: [email-masked]
Xref: clerew local.usefor:22913
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Diffs (Re: I-D ACTION:draft-ietf-usefor-usefor-06.txt)
Date: Thu, 12 Jan 2006 16:59:47 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>--On tirsdag, januar 10, 2006 09:56:06 -0800 John Stanley 
><[email-masked]> wrote:

>> I can generate a diff; that's not the issue. Of course, one must have the
>> -05 draft to do the diff against, and after searching the ietf website
>> for ten minutes, even their "drafttracker" cannot find the -05 draft. So,
>> no, I can't do the diff, nor can most other people who come across the
>> -06 draft and want to know what we've been doing.

>For those who might not know yet:

>There's a long-standing debate in the IETF about the IETF's 
>right/willingness to publish old and expired versions of I-Ds. At the 
>moment, the status is that the IETF only publishes the latest version.

But our earlier drafts can all be found on www.imc.org/ietf-usefor, and
there is also a pointer to the issue tracker there.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Date:  Wed, 11 Jan 2006 20:44:14 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 41
Sender: [email-masked]

Richard Clayton wrote:

> To try and add value, I'd propose that since so few systems
> add MISMATCH and it's basically valueless, we just document
> the 1036 scheme and hope that the existing (pretty trivial)
> usage dies away.

What _exactly_ would that be, something like the following ?

path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF
path-list       =  *( path-identity "!" [FWS] )

> As a fallback from that, just document what we see in the
> wild  "IPv4address.MISMATCH" and leave it at that.

Same question about the (following) variant:

path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF
path-list       =  *( path-identity "!" [ path-mismatch "!" ] [FWS] )

path-mismatch   =  IPv4address ".MISMATCH"

Common stuff for both variants:

path-identity   =  ( 1*( label "." ) toplabel ) / path-legacy

path-legacy     =  1*( ALPHA / DIGIT / "-" / "_" )
tail-entry      =  path-legacy                   ; allows underscore

label           =  letdig [ *( letdig / "-" ) letdig ]
letdig          =  ALPHA / DIGIT                 ; compare RFC 3696

Note that an IPv4 doesn't qualify as <path-identity>.  If you
want IPv4 as <path-identity> remove <toplabel> and replace its
one usage by <label>
                          Bye, Frank

Date: Wed, 11 Jan 2006 17:48:24 +0000
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes

>>And nothing in your response contradicted my answer of "YES, if 
>>diagnostics are important enough to create a definition for, they are 
>>important enough to be able to detect." That begs the question of what the
>>definition IS, and then why it should be in the path header field.
>
>Your response seemed more intent on saying that diagnostics should be
>outlawed entirely than on saying that, if present, they should be
>identifiable as such.
>
>But if you are agreeing that, if/when present, they should be
>identifiable as such, then thank you. That makes two of us who have
>expressed that opinion. If nobody disagrees, then perhaps we can now
>proceed to implement that feature (by adding a "SEEN" keyword, or any
>other mechanism people may care to suggest).

At present [though my database of sightings is almost 18 months old], a
very small number of sites are adding diagnostics.

The main diagnostic appears to be "this article arrived from an IP
address which does not seem to match the way I have been configured".
This is expressed as "MISMATCH".

Of course, as I and others have observed before, the most likely
explanation is that the configuration is wrong rather than that some
wickedness is occurring. However, when wickedness has been detected in
some other way and its source is being sought then doubtless the
MISMATCH will be one of the clues that is examined.

There does not appear to be any evidence of MISMATCH being treated as
something that is worthy of anyone's attention when no wickedness has
occurred.

Nevertheless, there are sites which are adding MISMATCH and they (or
their software author) clearly thinks it might be useful. Therefore this
group needs to either document the practice (because the aim is to put
what is being done on a formal basis) or fail to document it (hoping
that it will die out naturally and our disapprobation will hasten that).

However, it appears to be thought that adding the reverse of MISMATCH,
viz "I am prepared to vouch for this article arriving from where it says
it does" will be of value.

I cannot understand where the value arises :( and I think a very clear
case for that value must be made before we add this to the document.
There do not appear to be any sites that are currently adding this
marker (either as !! or !SEEN!) at present, so it's Yet Another Rfc
Invention That Will Confuse Implementors For Decades.

The reason it has no value, so far as I can see, is that it is
impossible to determine who added the diagnostic (someone who handled
the article afterwards can forge it).  Even if everyone is honest, then
a site that adds !SEEN! is just saying "I didn't add MISMATCH" so you
can deduce the value from the lack of the MISMATCH and finally, in order
to understand what exactly is being attested to...

        which might be: did the article come from a pre-ordained IP
        address? did it come from somewhere my local DNS resolver told
        me matched the name I am configured for? did it come from
        somewhere who could produce name/password credentials? did it
        come from my mother (who I trust implicitly)? etc etc

.. then you need to interview the sysadmins at the site which added the
marker to determine what their policy was a week last Tuesday when the
article passed through their system. And if you are going to all that
trouble then I expect they could look at some logs and tell you what you
needed to know anyway... in so far as anyone "needs to know" the source
of a Usenet article.

I fear, sorry chair, that I've just repeated myself -- since I am sure I
have wrote something similar some other year :(

To try and add value, I'd propose that since so few systems add MISMATCH
and it's basically valueless, we just document the 1036 scheme and hope
that the existing (pretty trivial) usage dies away.  As a fallback from
that, just document what we see in the wild  "IPv4address.MISMATCH" and
leave it at that.  YARITWCIFDs are the bane of Usenet. Why another ?

- -- 
richard                                                   Richard Clayton

To: [email-masked]
Xref: clerew local.usefor:22905
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Date: Wed, 11 Jan 2006 12:17:32 GMT
Lines: 42
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>The question which  I replied to was:

>>>  Still open whether USEFOR should specify syntax that allows one to
>>>  tell the difference between a path-identity and a diagnostic.

>And nothing in your response contradicted my answer of "YES, if 
>diagnostics are important enough to create a definition for, they are 
>important enough to be able to detect." That begs the question of what the
>definition IS, and then why it should be in the path header field.

Your response seemed more intent on saying that diagnostics should be
outlawed entirely than on saying that, if present, they should be
identifiable as such.

But if you are agreeing that, if/when present, they should be
identifiable as such, then thank you. That makes two of us who have
expressed that opinion. If nobody disagrees, then perhaps we can now
proceed to implement that feature (by adding a "SEEN" keyword, or any
other mechanism people may care to suggest).

>This situation is almost as bad as trying to enforce structure on a 
>completely unstructured header. Path-identities have very little 
>structure, and what little they do have overlaps in very great part with
>path-diagnostics and path-keywords. If the difference is significant 
>enough to spend time debating like this, then it is important enough to
>define and differentiate. If not, why are we doing this?

There I agree with you, and I have indeed proposed syntax to do just that
(though maybe I had overdone it).

To: [email-masked]
Xref: clerew local.usefor:22904
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: 3.2.14 : <sender>
Date: Wed, 11 Jan 2006 12:10:08 GMT
Lines: 62
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>If I can rephrase, and if John had all the time in the world to
>write verbosely, perhaps it would have come across as....

>An injecting agent, in general, cannot determine "the email address"
>that the user wishes to appear in a message.

Nobody claims that it can (and if I misinterpreted what John was claiming,
then my apologies). But for sure some injecting agents in some setups can
determine the/a 'true' email address of the sender whatever the sender
actually put in his "From:", and such injecting agents _can_ then insert
that information in an Injection-Info header.

>  There are security and
>privacy reasons to trust users will configure their user agents properly.
>Therefore the server must not try to be helpful to remote recipients wishing
>to do SPAM filtering (or other lesser goals) when it will compromise
>a primary requirement of a local user.

Whether inserting such sender information in an Injection-Info header is
wise/legal/helpful/whatever is a separate issue which should be (and is -
using a paragraph we agreed on years ago) discussed in USEAGE, not in the
standard. Our chair seems to agree with that, since he recently said that
a pointer to USEAGE would be in order. And it also seems that a "SHOULD"
in the present wording regarding that "sender" parameter will have to be
removed at the same time since the general opinion seems against it.

>The "in general" is implied, and no-one should need it to be present
>to understand John's point.  The primary vs secondary design goal
>"tie-breaker" is plain engineering choice.  And it is TOTALLY unacceptable to 
>expose a user@host pair as a matter of protocol. The standard must not
>even hint that a well-written server would do so.  How can anyone
>disagree on this?

There is (and always has been, even in this group) a tension between those
who want to hide the source of an article except by showing information
that only the injecting ISP can associate with an individual poster/site,
and exposing the poster/site to plain view on the grounds that many ISPs
(especially the ones patronized by spammers/abusers/trolls/other-bad-guys)
are reluctant to take action or to assist in tracing such malpractices.
But, as I said, USEAGE is the place to discuss that tension and the pros
and cons of both views. The standard should merely provide various tools
that can be used for these purposes.

>So I'm with John, compliant servers MUST NOT modify some of these
>user-supplied headers.  Period.

USEPRO says (and has always said) that injecting agents MUST NOT modify
headers that were placed there by the poster. But that is not the issue
under discussion.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047
Date:  Wed, 11 Jan 2006 11:47:20 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 59
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> What's out there is a mess of
> ...!server!diagnostic!....
> ...!server!server!....
> ...!server!POSTED!.... (possibly diagnostic before the
> final-entry, not> sure)

AFAIK my UA could submit articles as xyzzy!frank (at least in
a pre-s-o-1036 parallel universe), and that would then result
in server!POSTED!xyzzy!frank

That's shown in USEPRO 7.3.1, no problem for your third case.

Is your second case Russ' idea of unverified diagnostics,
only still without "SEEN", because that's not yet specified ?

For legacy formats we can only offer an unambiguous better
format.  The "multi-entry" is no real problem, a server is
free to pretend that its different identities talk with each
other.

And your first case ...!server!diagnostic!...

> Should the syntax bless, curse, grudgingly permit, or
> ignore these practices?

...would be cursed, if the diagnostics is an IP, because we
offer an unambiguous solution with MISMATCH or SEEN.  But I'm
of course not sure, I don't have a collection of "paths ITW".

Fresh attempt added below, Bill's parser would propose *"-"
instead of *( "-" ), but that's only an aesthetical nit.  I
hope the "features" are obvious.
                                Bye, Frank

path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF
path-list       =  *( path-entry "!" [FWS] )
path-entry      =  path-identity / path-verified / path-unverified /
                   path-mismatch / path-posted

path-verified   =  path-identity "!"             ; an additional "!"
path-unverified =  path-identity "!SEEN!"     path-diagnostic
path-mismatch   =  path-identity "!MISMATCH!" path-diagnostic
path-posted     =  path-identity "!POSTED!"   path-diagnostic

path-diagnostic =  path-identity / IPv4address / IPv6address
path-identity   =  ( 1*( label "." ) toplabel ) / path-legacy

path-legacy     =  1*( ALPHA / DIGIT / "-" / "_" )
tail-entry      =  path-legacy                   ; allows underscore

label           =  letdig [ *( letdig / "-" ) letdig ]
letdig          =  ALPHA / DIGIT                 ; compare RFC 3696

Date: Wed, 11 Jan 2006 07:42:34 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: John Stanley <[email-masked]>, [email-masked]
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Sender: [email-masked]

--On tirsdag, januar 10, 2006 18:24:51 -0800 John Stanley 
<[email-masked]> wrote:

> "Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:
>
>> We have already agreed that we can tolerate the occasional dead:beef
>> problem,
>
> No, "we" have not. The Chair may have declared a consensus to exist, but
> that does not imply agreement. I do not find it acceptable to create a
> problem for sites when the solution is trivial. It is patently trivial
> for a non-colon containing IPv6 encoding to be used for news headers.
> There is no reason not to require it.

John, welcome to the world of rough consensus; I believe your disagreement 
with the declared consensus on the topic is already well known.

You're reopening a closed issue, something that is only supposed to be done 
if you have new technical information to contribute.

Standard warnings apply; we've been here before.

                    Harald

Date: Wed, 11 Jan 2006 07:36:47 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: John Stanley <[email-masked]>, [email-masked]
Subject: Diffs (Re: I-D ACTION:draft-ietf-usefor-usefor-06.txt)
Sender: [email-masked]

Following up on this aspect in case someone doesn't know...

--On tirsdag, januar 10, 2006 09:56:06 -0800 John Stanley 
<[email-masked]> wrote:

>> I don't think it's a big issue - there are multiple places on the net
>> that  will generate a diff for you (I prefer bgp.potaroo.net, but that's
>> because  I like their diff format).
>
> I can generate a diff; that's not the issue. Of course, one must have the
> -05 draft to do the diff against, and after searching the ietf website
> for ten minutes, even their "drafttracker" cannot find the -05 draft. So,
> no, I can't do the diff, nor can most other people who come across the
> -06 draft and want to know what we've been doing.

For those who might not know yet:

There's a long-standing debate in the IETF about the IETF's 
right/willingness to publish old and expired versions of I-Ds. At the 
moment, the status is that the IETF only publishes the latest version.

bgp.potaroo.net, watersprings.org and others are archives that, in addition 
to the current version, keep the old versions around - something the IETF 
secretariat is unable to do at the moment, based on what the IETF has said 
its procedures are.

                 Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  ...ranting and maybe four missing [CFWS] ?!? (was: ISSUE: Newsgroups and Distribution WSP)
Date:  Wed, 11 Jan 2006 04:05:14 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 44
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> could you clarify which section of the spec you are talking
> about, and what the issue is?  At the moment, I can't tell
> whether this is about Control:, Newsgroups:, Distribution:,
> general philosophy of SP [FWS] or all 4.

The latter plus a sudden urge to start some kind of ranting.
I've submitted an erratum for 2822 <unstructured>, see also

 <http://permalink.gmane.org/gmane.ietf.rfc.interest/110>

Somebody CCed the author privately, and I've added a link
to the relevant discussion on the rfc822 list last year.

For obscure reasons I don't find the article with a question
which header fields in USEFOR are affected by the [FWS] CRLF
anomaly anymore.  Maybe it was a PM and I screwed up deleting
it, anyway, here's a list.  This might be incomplete, always
the last [FWS]:

 1: <unstructured>   (also submitted as 2822 erratum)
 2: <message-id>
 3: <newsgroup-list>
 4: <path-list>      (or its #1047 successor)
 5: <poster-text>
 6: <control>
 7: <supersedes>
 8: <dist-list>
 9: <xref>
10: <lines>

Unrelated, I just stumbled over it, Keith has an explicit
[CFWS] before <parameter> and before CRLF in RFC 3834.  We
don't have those two [CFWS] in the case of <archive>.  One
of these two solutions must be incorrect, 3834 or our idea.

Dito <injection-info>, two missing [CFWS] _iff_ 3834 has it
right for its <parameter>.  In our <user-agent> it's only a
<token>, but otherwise that follows Keith's pattern with an
explicit [CFWS] CRLF at the end and before each <token>.

                            Bye, Frank

Date: Tue, 10 Jan 2006 18:24:51 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>We have already agreed that we can tolerate the occasional dead:beef
>problem,

No, "we" have not. The Chair may have declared a consensus to exist, but 
that does not imply agreement. I do not find it acceptable to create a 
problem for sites when the solution is trivial. It is patently trivial for 
a non-colon containing IPv6 encoding to be used for news headers. There is 
no reason not to require it.

>>If a server chooses to identify itself by IPv4 address, there is no
>>problem.

>We have already agreed not to permit sites to identify _themselves_ with
>IP addresses

There is no problem if they do so; that string is no better or worse than
a fabricated alphanumeric string. (The same "problem" that occurs if a 
server changes IP address occurs if a server changes name, so if it's a 
problem in one case it's a problem for both.)

>Our drafts have had diagnostics for the past umpteen years.

Wrong. The word "diagnostic" does not appear in as recent a draft as 
draft-ietf-usefor-usefor-04. In draft 04, there is something called 
"path-keyword", which is limited to two very specific strings. In fact, 
draft 6 still has the same two "path-keywords" and has ADDED the 
path-diagnostic, so claiming that "MISMATCH" is a diagnostic is false.
MISMATCH has been and still is a path-keyword.

>All we are doing now is considering whether to tidy up
>some existing usages which don't currently have a keyword to identify
>them.

Well, prior to that, someone needs to define what a diagnostic is, why it 
is important enough to differentiate it from anything else in the path, 
and then HOW it is differentiated. Even before that, someone needs to 
figure out why it is so important to record things OTHER than the path an 
article travels in the header field defined to record the path an article 
travels. Do we want to record the weather at the injecting site in the 
date header field, too? Or maybe the process id of the injecting agent in 
the date header field.

Draft 6 is completely open-ended in how a path-diagnostic is defined: "an 
item inserted ... other than to indicate the name of a site." (And it is 
the rest of that paragraph that creates the problem that could be 
trivially solved.)

Currently, the only way to identify a "path-diagnostic" (without being
able to know what it means) is to see a colon in it. There are a lot of 
potential diagnostics that don't contain a colon. As it stands, every 
path-identity can be a path-diagnostic, and even the two "path-keywords"
could be path-identities.

The question which  I replied to was:

>>  Still open whether USEFOR should specify syntax that allows one to
>>  tell the difference between a path-identity and a diagnostic.

And nothing in your response contradicted my answer of "YES, if 
diagnostics are important enough to create a definition for, they are 
important enough to be able to detect." That begs the question of what the
definition IS, and then why it should be in the path header field.

This situation is almost as bad as trying to enforce structure on a 
completely unstructured header. Path-identities have very little 
structure, and what little they do have overlaps in very great part with
path-diagnostics and path-keywords. If the difference is significant 
enough to spend time debating like this, then it is important enough to
define and differentiate. If not, why are we doing this?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: 3.2.14 : <sender>
Date:  Wed, 11 Jan 2006 03:17:48 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 15
Sender: [email-masked]

Charles Lindsey wrote:

> the falsity of your statement is evident from the
> counter-example you are now looking at.

news.t-online.de used RADIUS and had some magic to relate
this to mail address(es) of the corresponding account.

IIRC it used that to verify From user@t-online addresses
in the case of cancels.  It's definitely possible in some
cases.  It might be even a desirable feature if the AUP
is clear about it, and legal in the country of the server.

                       Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: I-D ACTION:draft-ietf-usefor-usefor-06.txt
Date:  Wed, 11 Jan 2006 03:00:54 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 7
Sender: [email-masked]

Ken Murchison wrote:

> Would it help if I posted a diff between -05 and -06?

You did that already and it was very helpful:
http://article.gmane.org/gmane.ietf.usenet.format/30273/raw

Date: Tue, 10 Jan 2006 18:01:57 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

Lots of details of Charles's setup deleted. Relevant bit:

>The injecting agent in my CNews knows perfectly well that
>it came from me BECAUSE THAT FROM HEADER WAS ACTUALLY INSERTED BY THAT
>INJECTING AGENT (and not by nn).

Now say something that proves I am wrong. Please.

Yes, your injecting agent was able to insert AN email address allegedly
belonging to you into a news article. I somehow doubt that that email
address is THE email address for you, however. THE WORDS MEAN THINGS.
ALL THE WORDS. DEFINITE ARTICLES ARE NOT THE SAME AS INDEFINITE ARTICLES.

Had you written "the possibility for an injecting agent to know an email 
address of the sender", that would be different.

The point remains, that knowing "an" email address for a sender is NOT 
sufficient to claim knowledge of what addresses are NOT the sender's.

>>An injecting agent cannot possibly know "the email address" I want to use
>>for USENET-related correspondence EXCEPT by my using it in the From or
>>Sender header I SEND IT.

>Hence the falsity of your statement is evident from the counter-example
>you are now looking at.

Your "counter example" has nothing to do with the statement I made. I 
don't use the same setup you do. There is NO injecting agent that can know
the email address I want to use for USENET EXCEPT by seeing it in the
>From or Sender header I send it. The fact that YOUR injecting agent 
happens to pick ONE of the addresses you have and it is the one you want
used is a wonderful bit of luck for you, but has nothing at all to do with
the setup I use.

IETF standards are not supposed to be based on "wonderful bits of luck".
They are supposed to be factual. That's why the word "know" is important
is that statement I just made. It's not equivalent to "guess".

>I grant you that your statement is true as regards user agents that
>communicate with their servers via NNTP.

You just got done telling me that you had proven my statement false. Now
you say you grant that it is true. Pick one.

>I have never claimed otherwise.

]Hence the falsity of your statement is evident from the counter-example
]you are now looking at.

Did you or did you not write the words I quoted just above? Do you or do
you not understand the meaning of "falsity", and that it is the opposite
of "truth"?

"Forrest J. Cavalier III" <mibsoft@xxxxxxxxxxxxxxx>:

>John did NOT write "No injecting agent can determine the email address".
>But your counter-example seems to be refuting that exactly, and not
>what he wrote.

Actually, what Charles is busy refuting is a statement similar to "no 
injection agent can determine an email address", but that is still not
what I wrote, so he's busy refuting things that haven't been said and
ignoring those that have. (And yes, I actually did write something
that means "no injecting agent can determine the email address" that I
use for USENET-related email, because it cannot, and Charles's claims to
the contrary, he has not and did not prove me wrong on this. This has
been explained to him ad nauseum previously, and only appears again 
because HE brought it up again.)

The rest of Forrest's text is spot on. Either reject or accept, but MUST
NOT modify.

Date: Tue, 10 Jan 2006 13:19:52 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Sender: [email-masked]

Charles Lindsey wrote:
> 
> I grant you that your statement is true as regards user agents that
> communicate with their servers via NNTP. I have never claimed otherwise.
> 

Thanks for the example of how one single-user toy news servers works.  Excuse
me if fail to see how such examples are convincing when discussing an IETF 
protocol. Glad it works for you, but you cannot generalize world-class
solutions from its example.

And Charles, are you being deliberately obtuse to frustrate?

When John wrote:

 >An injecting agent cannot possibly know "the email address" I want to use
 >for USENET-related correspondence EXCEPT by my using it in the From or
 >Sender header I SEND IT.

John did NOT write "No injecting agent can determine the email address".
But your counter-example seems to be refuting that exactly, and not
what he wrote.

If I can rephrase, and if John had all the time in the world to
write verbosely, perhaps it would have come across as....

An injecting agent, in general, cannot determine "the email address"
that the user wishes to appear in a message.  There are security and
privacy reasons to trust users will configure their user agents properly.
Therefore the server must not try to be helpful to remote recipients wishing
to do SPAM filtering (or other lesser goals) when it will compromise
a primary requirement of a local user.

The "in general" is implied, and no-one should need it to be present
to understand John's point.  The primary vs secondary design goal
"tie-breaker" is plain engineering choice.  And it is TOTALLY unacceptable to 
expose a user@host pair as a matter of protocol. The standard must not
even hint that a well-written server would do so.  How can anyone
disagree on this?

So I'm with John, compliant servers MUST NOT modify some of these
user-supplied headers.  Period.  End of story.  No SHOULD NOT, no
MAY.  Refuse the article if the server determines it violates policy,
but privacy and security are primary goals here.

But, you know, this is just a drop in the bucket.  If we cannot even
agree on how to write clearly so that we agree on the issues to discuss,
it's all pointless.  Please uncharter this working group.

Date: Tue, 10 Jan 2006 09:56:06 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: I-D ACTION:draft-ietf-usefor-usefor-06.txt
Sender: [email-masked]

Ken Murchison <murch@xxxxxxxxxxxxxx>:

>The change section mentions the ticket number along with a description of 
>what issue the ticket corresponds to, which hopefully should point you to 
>the proper area of the document.

Here's the first "change" listed in 06:

    o  Resolved ticket #1003 (msg-id).

That tells me that something about the "msg-id" has changed. It doesn't 
tell me what a "ticket #1003" is. I shouldn't have to find some arbitrary 
website somewhere, with an invalid SSL cert and a username/password tossed 
on top, to see what the topic for the change was, or if the ticket even 
lists what the specific change was.

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>I don't think it's a big issue - there are multiple places on the net that 
>will generate a diff for you (I prefer bgp.potaroo.net, but that's because 
>I like their diff format).

I can generate a diff; that's not the issue. Of course, one must have the 
-05 draft to do the diff against, and after searching the ietf website for 
ten minutes, even their "drafttracker" cannot find the -05 draft. So, no, 
I can't do the diff, nor can most other people who come across the -06 
draft and want to know what we've been doing.

>Next time, it might be nice to include the section number in the "resolved 
>ticket" note - I've tried to keep section numbers in the ticket titles, so 
>that shouldn't be too much work.

Maybe I wasn't clear enough. What appears in this ticket system is 
irrelevant. Unless I'm a member of this group already, I won't know
what this "ticket system" is, where it is, what the username is, and 
unless I'm lax on my security (by accepting invalid certs) can't get 
logged in anyway. Keep whatever you want in the "notes". It doesn't help.

BUT, I see that as recently as draft 4, things were done in a productive
and informative way.

]Changes since draft-ietf-usefor-usefor-03

]   o  Reworked ABNFs of several headers.

]   o  Used Charles' ABNF for <msg-id>.

Well, I'm not sure which of "several headers" to look at, but gosh if I
don't know precisely what bit of the draft to look at to see what the 
second change did. And the third, and the fourth, and the fifth. Etc.

To: [email-masked]
Xref: clerew local.usefor:22895
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Date: Tue, 10 Jan 2006 13:56:04 GMT
Lines: 58
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>... all it needs to do it to pick out the
>>delimiters, pick out the stuff between them (it might or might not worry
>>about illegal characters in the 'stuff'), and then see if the 'stuff' is
>>recorded in its 'sys' file as a place that it need not relay the article
>>to.

>In other words, we expect it to do the wrong thing if there is an IPv6
>colonized "stuff between them" that matches a UUCP name.

We have already agreed that we can tolerate the occasional dead:beef
problem, provided we ensure that it can only arise with barewords composed
only from hex digits (though the present text in draft-06 does not ensure
this yet).

>Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx>:

[snip]

>The Path header field should be a list of the servers through which an 
>article has passed. That's why it's called "Path" and not 
>"PathAndWhateverElse".

>If a server chooses to identify itself by IPv4 address, there is no 
>problem.

We have already agreed not to permit sites to identify _themselves_ with
IP addresses (I might agree with you that it is an unnecessary
restriction, but it is the consensus reached, and so I will stand by it).

>Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>>  Still open whether USEFOR should specify syntax that allows one to
>>  tell the difference between a path-identity and a diagnostic.

>USEFOR specifies article syntax, yes? Is there a reason to HAVE a 
>"diagnostic"?

Our drafts have had diagnostics for the past umpteen years. Firstly with
Brad's special delimiters, and more recently with the keyword MISMATCH and
the '!!' notation. All we are doing now is considering whether to tidy up
some existing usages which don't currently have a keyword to identify
them.

To: [email-masked]
Xref: clerew local.usefor:22896
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: Newsgroups and Distribution WSP (was: Distribution)
Date: Tue, 10 Jan 2006 14:05:13 GMT
Lines: 37
Sender: [email-masked]

>> If you want a similar "SHOULD NOT" for control messages,
>> then that would be fine (and consistent).

>No, and I want no crappy [FWS] CRLF anywhere.  It's forbidden
>by a MUST NOT in 2822 3.2.3 for CFWS (minus comments), and it
>is also forbidden by a MUST in draft -06 2.2 bullet 2.

Yes, but that is nothing to do with control messages - it applies to all
headers. Do you accept that a "SHOULD NOT" wording (as in Newsgroups and
References) would be an acceptable resolution of the control message
situation (and Distribuition too, while we are at it)?

>The very first ABNF specified, same section 2.2 bullet 2 is...

>| unstructured    =  1*( [FWS] utext ) [FWS]
>                                       ^^^^^

That [FWS] at the end is already in RFC 2822. But even there it is
meaningless as regards folding because there is verbiage telling you not
to fold there (and we have similar, but slightly stricter, verbiage). We
have enough differences from RFC 2822 that matter, without introducing
further differences that don't.

To: [email-masked]
Xref: clerew local.usefor:22894
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: 3.2.14 : <sender>
Date: Tue, 10 Jan 2006 13:06:21 GMT
Lines: 50
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>And in practice
>>the possibility for an injecting agent to know the email address of the
>>sender is restricted to organizations which run their own injecting agent 
>...

>It does not exist even there. Since this issue is being brought up
>again and the same incorrect statements are being made, I'm going to
>take the opportunity to correct them again.

And again and again it has been explained to you that you are wrong.

It so happens (as I have said many times) that I gateway this list into a
local newsgroup on my machine. Hence I am writing this reply using my
newsreader.

My newsreader is 'nn', which is quite commonly used.

My news server is CNews, which is also quite commonly used.

You will observe that this reply contains a From: header, indicating that
is came from me. The injecting agent in my CNews knows perfectly well that
it came from me BECAUSE THAT FROM HEADER WAS ACTUALLY INSERTED BY THAT
INJECTING AGENT (and not by nn).

>An injecting agent cannot possibly know "the email address" I want to use
>for USENET-related correspondence EXCEPT by my using it in the From or
>Sender header I SEND IT.

Hence the falsity of your statement is evident from the counter-example
you are now looking at. And if my CNews were to be upgraded to generate an
Injection-Info header, it could quite easily generate a sender parameter
if I configured it so.

I grant you that your statement is true as regards user agents that
communicate with their servers via NNTP. I have never claimed otherwise.

Date: Tue, 10 Jan 2006 14:30:01 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Ken Murchison <[email-masked]>, [email-masked]
Subject: Re: I-D ACTION:draft-ietf-usefor-usefor-06.txt
Sender: [email-masked]

--On mandag, januar 09, 2006 22:31:59 -0500 Ken Murchison 
<[email-masked]> wrote:

>
> John Stanley wrote:
>>
>>
>> A document change section that says nothing more for most items than
>> "resolved ticket x" is meaningless. Could our esteemed editor please
>> properly document changes in the draft by saying what the change was
>> and where it was made? Our drafts go to more than just the dedicated few
>> here who have the time and ability to memorize arcane "ticket numbers",
>> and they ought to mean something.
>
> The change section mentions the ticket number along with a description of
> what issue the ticket corresponds to, which hopefully should point you to
> the proper area of the document.  I'll concede that the draft should
> mention where to find the content of the tickets.  I'll try to do a
> better job of documenting changes in -07.  Would it help if I posted a
> diff between -05 and -06?

I don't think it's a big issue - there are multiple places on the net that 
will generate a diff for you (I prefer bgp.potaroo.net, but that's because 
I like their diff format).

Next time, it might be nice to include the section number in the "resolved 
ticket" note - I've tried to keep section numbers in the ticket titles, so 
that shouldn't be too much work.

                   Harald

Date: Mon, 09 Jan 2006 22:31:59 -0500
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: [email-masked]
Subject: Re: I-D ACTION:draft-ietf-usefor-usefor-06.txt
Sender: [email-masked]

John Stanley wrote:
> 
> 
> A document change section that says nothing more for most items than
> "resolved ticket x" is meaningless. Could our esteemed editor please
> properly document changes in the draft by saying what the change was
> and where it was made? Our drafts go to more than just the dedicated few
> here who have the time and ability to memorize arcane "ticket numbers",
> and they ought to mean something.

The change section mentions the ticket number along with a description 
of what issue the ticket corresponds to, which hopefully should point 
you to the proper area of the document.  I'll concede that the draft 
should mention where to find the content of the tickets.  I'll try to do 
a better job of documenting changes in -07.  Would it help if I posted a 
diff between -05 and -06?

-- 
Kenneth Murchison
Project Cyrus Developer/Maintainer
Carnegie Mellon University

Date: Fri, 6 Jan 2006 18:18:03 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>... all it needs to do it to pick out the
>delimiters, pick out the stuff between them (it might or might not worry
>about illegal characters in the 'stuff'), and then see if the 'stuff' is
>recorded in its 'sys' file as a place that it need not relay the article
>to.

In other words, we expect it to do the wrong thing if there is an IPv6
colonized "stuff between them" that matches a UUCP name.

Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx>:

>It MUST be possible to get this right, assuming that it really
>is what we want (the main problem, what should the Path be ?).

The Path header field should be a list of the servers through which an 
article has passed. That's why it's called "Path" and not 
"PathAndWhateverElse".

If a server chooses to identify itself by IPv4 address, there is no 
problem.

If a server chooses to identify itself by IPv6 address, it can do so by 
simply removing all colons as I've already proposed, and there is no 
problem.

If we want to say that a receiving server can prepend the known identity 
of a server if the peer has used something it doesn't know, then it can do 
so. That can be an IP address.

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>  Still open whether USEFOR should specify syntax that allows one to
>  tell the difference between a path-identity and a diagnostic.

USEFOR specifies article syntax, yes? Is there a reason to HAVE a 
"diagnostic"? If so, then yes, USEFOR must specify a syntax that allows 
one to tell the difference. If the question of being able to differentiate 
is "still open", then I suggest that the question of the need for defining 
a "diagnostic" is still open. Just what IS the need? (Oh yes, colons MUST 
be allowed somehow, even if we could just get rid of them altogether, 
and instead of allowing them in "path-identity", we create a new thing
that does allow them but can appear in all the same places a 
path-identity does.)

Date: Fri, 6 Jan 2006 18:14:19 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>And in practice
>the possibility for an injecting agent to know the email address of the
>sender is restricted to organizations which run their own injecting agent 
...

It does not exist even there. Since this issue is being brought up
again and the same incorrect statements are being made, I'm going to
take the opportunity to correct them again.

An injecting agent cannot possibly know "the email address" I want to use
for USENET-related correspondence EXCEPT by my using it in the From or
Sender header I SEND IT.

Before an agent should be allowed to CHANGE any of that data (or insert
"correct" data in another header) it MUST know that the data I give it
is incorrect. It cannot know this.

Not only is such action a privacy violation, it is a security issue. If
a server inserts any header containing "the email address" for me in a 
form
"username@host", it creates a security issue where a cracker now knows a
valid username and host that he otherwise would not have known. It is an
invitation for a brute-force attack.

Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx>:

>"Killfiling" is an
>important feature for those who have it, but enforcing that
>possibility with a SHOULD goes to far.

Revealing account and mailbox information against the poster's
expressed wishes is not a "killfiling" issue. You can killfile on
the From header easier than on almost any other.

Date: Fri, 6 Jan 2006 18:18:55 -0800 (PST)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: I-D ACTION:draft-ietf-usefor-usefor-06.txt
Sender: [email-masked]

A document change section that says nothing more for most items than
"resolved ticket x" is meaningless. Could our esteemed editor please
properly document changes in the draft by saying what the change was
and where it was made? Our drafts go to more than just the dedicated few
here who have the time and ability to memorize arcane "ticket numbers",
and they ought to mean something.

Date: Mon, 09 Jan 2006 16:17:49 -0500
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: ISSUE: Newsgroups and Distribution WSP
Sender: [email-masked]

Frank Ellermann wrote:
> Charles Lindsey wrote:
> 
> 
>>If you want a similar "SHOULD NOT" for control messages,
>>then that would be fine (and consistent).
> 
> 
> No, and I want no crappy [FWS] CRLF anywhere.  It's forbidden
> by a MUST NOT in 2822 3.2.3 for CFWS (minus comments), and it
> is also forbidden by a MUST in draft -06 2.2 bullet 2.
> 
> The very first ABNF specified, same section 2.2 bullet 2 is...
> 
> | unstructured    =  1*( [FWS] utext ) [FWS]
>                                        ^^^^^
> ...declaring "red is green":  This [FWS] can never be folded
> without violating the preceding MUST, and the similar idea in
> RFC 2822 3.2.3 for CFWS (excl. comments).  It should be *WSP.
> 
> This infuriates me, especially when folks think that I support
> this stuff, which isn't the case, and come to the conclusion
> that this must be some kind of sly manipulative dishonesty, or
> a similar stunt.  Better no ABNF than intentionally wrong ABNF.

Frank, are you saying that something in -06 is incorrect/inconsistent? 
If so, could you please clarify?

-- 
Kenneth Murchison
Project Cyrus Developer/Maintainer
Carnegie Mellon University

Date: Mon, 09 Jan 2006 17:30:29 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: ISSUE: Newsgroups and Distribution WSP (was: Distribution)
Sender: [email-masked]

Frank,

could you clarify which section of the spec you are talking about, and what 
the issue is?

At the moment, I can't tell whether this is about Control:, Newsgroups:, 
Distribution:, general philosophy of SP [FWS] or all 4.

--On l�rdag, januar 07, 2006 11:04:03 +0100 Frank Ellermann 
<[email-masked]> wrote:

>
> Charles Lindsey wrote:
>
>> In the case of Control messages, it is hard to imagine a case
>> where breaking would be useful, and the sky is unlikely to
>> fall in if Russ leaves implementing that feature until he
>> has time to do it (which may be a long time off, but not
>> "never").  And the worst that can happen if some control
>> messages _does_ get folded is that some sites may not
>> create/remove the group, and if their users care it will soon
>> get reported and fixed.
>
> IMHO that misses the point.  As an example, in a popular TLH
> the checkgroups guy proposed to change the key from pgp 2.6.x
> to a modern gpg and more than 1024 bits.  He was newly elected,
> knew his trade, and intended to do it a.s.a.p.   That got him
> almost "killed", some Usenet legends in this TLH (who hadn't
> posted for ages) were seriously pissed, "did you discuss this
> in the relevant international group and above all with Russ ?",
> "most news servers will _never_ upgrade, they only run because
>  nobody wants the trouble to shut them down at this time" were
> the most striking rants.
>
> Needless to say that the key wasn't changed.  So even if Russ
> or somebody else implements USEFOR + USEPRO on the same day
> when the IESG posts its approval major parts of Usenet won't
> use this "soon".  That might be "never" for many news servers.
>
> OTOH there will be folks trying to create simple gateways or
> new UAs based on the new standard, after all that's the purpose
> of this WG, the old 1036 doesn't reflect reality anymore, and
> parts of (2)822 are misleading wrt NetNews, to put it mildly.
>
> The standard is for those poor folks, and if they try to fold
> a control for whatever reasons they screwed up.  It would hurt
> their product, their users, and they'd never trust a single
> word of the standard if they found any lie "FWS" the hard way.
>
> So we have to get the working idea to them.  One way to do this
> is to say FWS in the ABNF, then add some "MUSTard" essentially
> explaining why FWS actually stands for 1*WSP.  Another way is
> to simply say 1*WSP and be done with it.
>
> The first way would never work for me, I won't read the prose
> if the ABNF is apparently clear.  We could decree that that's
> my fault, and besides it's unlikely that I start to implement
> a gateway / UA / server.  But apparently it's not only my
> problem, Russ said it's precisely what INN does at the moment.
>
> Wishful thinking won't help, timetravel is unavailable, we're
> forced to describe common practice.  Unless there are serious
> and important reasons to change something just leave it as it
> is, if it ain't broken don't fix it.
>
> If a news2mail gateway manages to export control messages, a
> stupid idea in most contexts, and something behind it folds
> them, no harm done.  But the important point is "won't work
> if folded".
>
> We already discussed this in several hundreds of articles,
> doing it again won't help.  In your USEPRO diffs I found:
>
> ! Because of its importance to all serving agents, the whitespace
> ! and folding in Newsgroups header fields newly permitted by
> ! [USEFOR] SHOULD NOT be generated (though it MUST be accepted);
> ! this restriction may well be removed in a future version of this
> ! standard.
> + [That last bit needs discussion. It should probably be moved to
> + USEFOR if it is to be retained.]
>
> Yes, let's move it.  Maybe with a note for "Distribution" that
> it's in theory the same issue there.  In USEPRO we could still
> explain why a decent UA or Injection-Agent might wish to fix
> that for human users, it's too easy to get this wrong.
>
> Or maybe "reject".  The one thing it shouldn't do is "accept"
> and then silently ignore anything after "," 1*WSP white space.
>
>>> IMHO we should keep this stuff consistent, dishonest or not.
>
>> Yes. A remark in the bit where it says SHOULD NOT use FWS in
>> Newsgroups to the effect "this also applies to Distribution"
>> would be in order.
>
> ACK, let's get a ticket for that, subject adjusted.  Bye, Frank
>
>
>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: Newsgroups and Distribution WSP (was: Distribution)
Date:  Mon, 09 Jan 2006 16:38:28 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 24
Sender: [email-masked]

Charles Lindsey wrote:

> If you want a similar "SHOULD NOT" for control messages,
> then that would be fine (and consistent).

No, and I want no crappy [FWS] CRLF anywhere.  It's forbidden
by a MUST NOT in 2822 3.2.3 for CFWS (minus comments), and it
is also forbidden by a MUST in draft -06 2.2 bullet 2.

The very first ABNF specified, same section 2.2 bullet 2 is...

| unstructured    =  1*( [FWS] utext ) [FWS]
                                       ^^^^^
...declaring "red is green":  This [FWS] can never be folded
without violating the preceding MUST, and the similar idea in
RFC 2822 3.2.3 for CFWS (excl. comments).  It should be *WSP.

This infuriates me, especially when folks think that I support
this stuff, which isn't the case, and come to the conclusion
that this must be some kind of sly manipulative dishonesty, or
a similar stunt.  Better no ABNF than intentionally wrong ABNF.

                            Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:22886
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: Newsgroups and Distribution WSP (was: Distribution)
Date: Mon, 9 Jan 2006 11:08:12 GMT
Lines: 39
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>OTOH there will be folks trying to create simple gateways or
>new UAs based on the new standard, after all that's the purpose
>of this WG, the old 1036 doesn't reflect reality anymore, and
>parts of (2)822 are misleading wrt NetNews, to put it mildly.

Yes, but control messages are not issued by gateways or UAs (except maybe
for cancels). They are issued by people who know (or ought to know) what
they are doing.

>We already discussed this in several hundreds of articles,
>doing it again won't help.  In your USEPRO diffs I found:

>! Because of its importance to all serving agents, the whitespace
>! and folding in Newsgroups header fields newly permitted by
>! [USEFOR] SHOULD NOT be generated (though it MUST be accepted);
>! this restriction may well be removed in a future version of this
>! standard.
>+ [That last bit needs discussion. It should probably be moved to
>+ USEFOR if it is to be retained.]

Yes, that "SHOULD NOT" (and the similar "SHOULD NOT" for comments in
References headers) was the wording that we agreed for those cases. If you
want a similar "SHOULD NOT" for control messages, then that would be fine
(and consistent). I just don't want to see a large collection of similar
cases which are all treated slightly differently.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  ISSUE: Newsgroups and Distribution WSP (was: Distribution)
Date:  Sat, 07 Jan 2006 11:04:03 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 85
Sender: [email-masked]

Charles Lindsey wrote:

> In the case of Control messages, it is hard to imagine a case
> where breaking would be useful, and the sky is unlikely to
> fall in if Russ leaves implementing that feature until he
> has time to do it (which may be a long time off, but not
> "never").  And the worst that can happen if some control
> messages _does_ get folded is that some sites may not
> create/remove the group, and if their users care it will soon
> get reported and fixed.

IMHO that misses the point.  As an example, in a popular TLH
the checkgroups guy proposed to change the key from pgp 2.6.x
to a modern gpg and more than 1024 bits.  He was newly elected,
knew his trade, and intended to do it a.s.a.p.   That got him
almost "killed", some Usenet legends in this TLH (who hadn't
posted for ages) were seriously pissed, "did you discuss this
in the relevant international group and above all with Russ ?",
"most news servers will _never_ upgrade, they only run because
 nobody wants the trouble to shut them down at this time" were
the most striking rants.

Needless to say that the key wasn't changed.  So even if Russ
or somebody else implements USEFOR + USEPRO on the same day
when the IESG posts its approval major parts of Usenet won't
use this "soon".  That might be "never" for many news servers.

OTOH there will be folks trying to create simple gateways or
new UAs based on the new standard, after all that's the purpose
of this WG, the old 1036 doesn't reflect reality anymore, and
parts of (2)822 are misleading wrt NetNews, to put it mildly.

The standard is for those poor folks, and if they try to fold
a control for whatever reasons they screwed up.  It would hurt
their product, their users, and they'd never trust a single
word of the standard if they found any lie "FWS" the hard way.

So we have to get the working idea to them.  One way to do this
is to say FWS in the ABNF, then add some "MUSTard" essentially
explaining why FWS actually stands for 1*WSP.  Another way is
to simply say 1*WSP and be done with it.

The first way would never work for me, I won't read the prose
if the ABNF is apparently clear.  We could decree that that's
my fault, and besides it's unlikely that I start to implement
a gateway / UA / server.  But apparently it's not only my
problem, Russ said it's precisely what INN does at the moment.

Wishful thinking won't help, timetravel is unavailable, we're
forced to describe common practice.  Unless there are serious
and important reasons to change something just leave it as it
is, if it ain't broken don't fix it.

If a news2mail gateway manages to export control messages, a
stupid idea in most contexts, and something behind it folds
them, no harm done.  But the important point is "won't work
if folded".

We already discussed this in several hundreds of articles,
doing it again won't help.  In your USEPRO diffs I found:

! Because of its importance to all serving agents, the whitespace
! and folding in Newsgroups header fields newly permitted by
! [USEFOR] SHOULD NOT be generated (though it MUST be accepted);
! this restriction may well be removed in a future version of this
! standard.
+ [That last bit needs discussion. It should probably be moved to
+ USEFOR if it is to be retained.]

Yes, let's move it.  Maybe with a note for "Distribution" that
it's in theory the same issue there.  In USEPRO we could still
explain why a decent UA or Injection-Agent might wish to fix
that for human users, it's too easy to get this wrong.

Or maybe "reject".  The one thing it shouldn't do is "accept"
and then silently ignore anything after "," 1*WSP white space.

>> IMHO we should keep this stuff consistent, dishonest or not.

> Yes. A remark in the bit where it says SHOULD NOT use FWS in
> Newsgroups to the effect "this also applies to Distribution"
> would be in order.

ACK, let's get a ticket for that, subject adjusted.  Bye, Frank

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-usefor-usepro-05.txt 
Date: Fri, 06 Jan 2006 15:50:02 -0500
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Usenet Article Standard Update Working Group of the IETF.

	Title		: News Article Architecture and Protocols
	Author(s)	: C. Lindsey
	Filename	: draft-ietf-usefor-usepro-05.txt
	Pages		: 54
	Date		: 2006-1-6
	
This Draft, together with its companion draft [USEFOR], are
   intended as standards track documents, together obsoleting RFC
   1036, which itself dates from 1987.

   This Standard defines the architecture of Netnews systems and
   specifies the requirements to be met by software which originates,
   distributes, stores and displays Netnews articles.

   Backward compatibility has been a major goal of this endeavour, but
   where this standard and earlier documents or practices conflict, this
   standard should be followed. In most such cases, current practice is
   already compatible with these changes.

   A companion Best Current Practice document [USEAGE], addressing
   requirements which are present for Social rather than Normative
   reasons is in preparation.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usepro-05.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-usefor-usepro-05.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-usefor-usepro-05.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-usefor-usepro-05.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-usefor-usepro-05.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

To: [email-masked]
Xref: clerew local.usefor:22883
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: draft-ietf-usefor-usepro-05
Date: Fri, 6 Jan 2006 17:16:41 GMT
Lines: 948
Sender: [email-masked]

See www.imc.org/ietf/usefor/drafts/draft-ietf-usefor-usepro-05.txt
and www.imc.org/ietf/usefor/drafts/draft-ietf-usefor-usepro-05.unpaged
and also on its way to the drafts editor.

The main changes are the rmeoval of all texts that were duplicated in
USEFOR, with other consequential changes to bring into line with
USEFOR-06. Also removed the historical appendices and replaced with
wording referring to Henry's about-to-be-published Son-of-1036. Also
assorted niggles and boilerplate fixes. No changes to the protocols
(though there will be work to be done when #1047 is resolved).

For those who want to check, here are the full diffs.

*** draft-ietf-usefor-usepro-04.unpaged	Fri Jul 15 18:20:42 2005
--- draft-ietf-usefor-usepro-05.unpaged	Thu Jan  5 22:44:15 2006
***************
*** 3,5 ****
  Usenet Format Working Group                  University of Manchester
!                                              July 2005
  
--- 3,5 ----
  Usenet Format Working Group                  University of Manchester
!                                              January 2006
  
***************
*** 6,8 ****
                  News Article Architecture and Protocols
!                    <draft-ietf-usefor-usepro-04.txt>
  
--- 6,8 ----
                  News Article Architecture and Protocols
!                    <draft-ietf-usefor-usepro-05.txt>
  
***************
*** 31,33 ****
  
!    This Internet-Draft will expire in January 2006.
  
--- 31,33 ----
  
!    This Internet-Draft will expire in July 2006.
  
***************
*** 43,48 ****
  
-    Since the 1980s, Usenet has grown explosively, and many Internet and
-    non-Internet sites now participate. In addition, the Netnews
-    technology is now in widespread use for other purposes.
- 
     Backward compatibility has been a major goal of this endeavour, but
--- 43,44 ----
***************
*** 52,54 ****
  
!    A companion Current Best Practice document [USEAGE], addressing
     requirements which are present for Social rather than Normative
--- 48,50 ----
  
!    A companion Best Current Practice document [USEAGE], addressing
     requirements which are present for Social rather than Normative
***************
*** 81,83 ****
    2.2.  Defining the Architecture .................................    0
!   2.3.  Identification of news-servers ............................    0
    2.4.  Variant Header Fields .....................................    0
--- 78,80 ----
    2.2.  Defining the Architecture .................................    0
!   2.3.  Identification of news servers ............................    0
    2.4.  Variant Header Fields .....................................    0
***************
*** 85,87 ****
  3.  Changes to the existing protocols .............................    0
!   3.1.  Principal Changes .........................................    0
    3.2.  Transitional Arrangements .................................    0
--- 82,84 ----
  3.  Changes to the existing protocols .............................    0
!   3.1.  Protocol Changes ..........................................    0
    3.2.  Transitional Arrangements .................................    0
***************
*** 138,142 ****
  12.  Contact Address ..............................................    0
! Appendix A.1 - A-News Article Format ..............................    0
! Appendix A.2 - Early B-News Article Format ........................    0
! Appendix A.3 - Obsolete Control Messages ..........................    0
  Appendix B - Notices ..............................................    0
--- 135,137 ----
  12.  Contact Address ..............................................    0
! Appendix A - Obsolete Control Messages ............................    0
  Appendix B - Notices ..............................................    0
***************
*** 145,159 ****
  
- [This draft [USEPRO] and its partner [USEFOR] are an interim stage in
- the splitting into two parts of the earlier draft [ARTICLE].  There is a
- certain amount of material - basic concepts, definitions, etc - which
- ultimately need occur in only one of the documents, and further such
- material which may not be needed at all (e.g. terms currently defined
- which in the event may not get used). For the moment, all such material
- has been retained in the present draft (it being, in any case, easier to
- take unwanted stuff out than to put new stuff in). It has also to be
- decided, for such material which is needed by both documents, which one
- (the "Primary") should contain it and which one should incorporate it by
- reference (essentially, this draft is written so that it could be the
- Primary).]
- 
  1.  Introduction
--- 140,141 ----
***************
*** 173,181 ****
  
-    An important characteristic of Netnews is the lack of any requirement
-    for a central administration or for the establishment of any
-    controlling host to manage the network. A set of hosts within a
-    network which, by mutual arrangement, operates some variant (whether
-    more or less restrictive) of the Netnews protocols is a "cooperating
-    subnet".
- 
     "Usenet" is a particular worldwide publicly accessible network based
--- 155,156 ----
***************
*** 186,187 ****
--- 161,168 ----
  
+    An important characteristic of Usenet is the lack of any requirement
+    for a central administration or for the establishment of any
+    controlling host to manage the network. Nevertheless, administrative
+    agencies do exists with varying degrees of authority to establish
+    "policies" applicable to particular parts of Usenet.
+ 
     A "policy" is a rule intended to facilitate the smooth operation of a
***************
*** 194,195 ****
--- 175,177 ----
     their recipients.
+ [Could omit that last sentence.]
  
***************
*** 203,207 ****
     between the various agents comprising that architecture. In this
!    standard, references to sections in the companion [USEFOR] are
!    prefixed with "F-".
  
     It is NOT the purpose of this standard to settle matters of policy,
--- 185,194 ----
     between the various agents comprising that architecture. In this
!    standard, references to individual sections in the companion [USEFOR]
!    are prefixed with "F-".
  
+    A set of hosts within a network which, by mutual arrangement,
+    operates some variant (whether more or less restrictive) of the
+    Netnews protocols is a "cooperating subnet".
+ [It is not clear whether we still need that definition.]
+ 
     It is NOT the purpose of this standard to settle matters of policy,
***************
*** 226,246 ****
  
!    The earliest news interchange used the so-called "A News" article
!    format.  Shortly thereafter, an article format vaguely resembling
!    Internet Mail was devised and used briefly.  Both of those formats
!    are completely obsolete; they are documented in Appendix A.1 and
!    Appendix A.2 for historical reasons only.  With publication of [RFC
!    850] in 1983, news articles came to closely resemble Internet Mail
!    messages, with some restrictions and some additional header fields.
!    [RFC 1036] in 1987 updated [RFC 850] without making major changes.
  
!    A Draft popularly referred to as "Son of 1036" [Son-of-1036] was
!    written in 1994 by Henry Spencer.  Much is taken directly from Son of
!    1036, and it is hoped that we have followed its spirit and
!    intentions.
  
- [It is anticipated that [Son-of-1036] will shortly be published as an
- informational RFC (for purposes of historical documentation only), in
- which case most historical information can be removed from this draft,
- including the whole of Appendix A.1 and Appendix A.2.]
- 
  2.  Definitions, Notations and Conventions
--- 213,225 ----
  
!    For an account of the earlier formats used in Netnews prior to [RFC
!    1036], see Henry Spencer's 1994 draft, popularly referred to as "Son
!    of 1036" [Son-of-1036], which has recently been republished as an
!    Informational RFC.
! [That is a tentative statement, which may need revision.]
  
!    Although never adopted as a formal standard, [Son-of-1036] had a
!    considerable effect on the development of Netnews and hence on these
!    present standards, and it is hoped that we have followed its spirit
!    and intentions.
  
  2.  Definitions, Notations and Conventions
***************
*** 249,305 ****
  
!    An "article" is the unit of news, synonymous with an [RFC 2822]
!    "message".  A "proto-article" is one that has not yet been injected
!    into the news system.  In constrast to an article, a proto- article
!    may lack some mandatory header fields
  
-    A "message identifier" (F-3.1.3) is a unique identifier for an
-    article, usually supplied by the posting agent which posted it or,
-    failing that, by the injecting agent. It distinguishes the article
-    from every other article ever posted anywhere. Articles with the same
-    message identifier are treated as if they are the same article
-    regardless of any differences in the body or header fields.
- 
-    A "newsgroup" is a single news forum, a logical bulletin board,
-    having a name and nominally intended for articles on a specific
-    topic. An article is "posted to" a single newsgroup or several
-    newsgroups. When an article is posted to more than one newsgroup, it
-    is said to be "crossposted"; note that this differs from posting the
-    same text as part of each of several articles, one per newsgroup.
- 
-    A newsgroup may be "moderated", in which case submissions are not
-    posted directly, but mailed to a "moderator" for consideration and
-    possible posting.  Moderators are typically human but may be
-    implemented partially or entirely in software.
- 
     A "hierarchy" is the set of all newsgroups whose names share a first
!    component (as defined in F-3.1.5).  The term "sub-hierarchy" is also
!    used where several initial components are shared.
  
-    A "poster" is the person or software that composes and submits a
-    possibly compliant article for submission to a posting agent. The
-    poster is analogous to [RFC 2822]'s author.
- 
-    A "reader" is the person or software reading news articles.
- 
-    A "followup" is an article containing a response to the contents of
-    an earlier article, its "precursor". Every followup includes a
-    References header field identifying that precursor (but note that
-    non-followup articles may also use a References header field).
- 
-    An (email) "address" is the mailbox [RFC 2822] (or more particularly
-    the addr-spec within that mailbox) which directs the delivery of an
-    email to its intended recipient, who is said to "own" that address.
- 
-    A "sender" is the person or software (usually, but not always, the
-    same as the poster) responsible for the operation of the posting
-    agent or, which amounts to the same thing, for passing the article to
-    the injecting agent.
- [Is the definition in RFC 2822 sufficient?]
- 
-    A "control message" is an article which is marked as containing
-    control information; a "serving agent" (and in some cases a "relaying
-    agent") receiving such an article may (subject to the policies
-    observed at that site) take actions beyond just filing and passing on
-    the article.
- 
     The "semantic content" (often abbreviated to just "content" when the
--- 228,237 ----
  
!    All the technical terms defined in F-1.5 are to be considered as
!    defined also, with the same meaning, in this standard.  In addition,
!    some further terms are defined here, and in the following section.
  
     A "hierarchy" is the set of all newsgroups whose names share a first
!    <component> (as defined in F-3.1.5).  The term "sub-hierarchy" is
!    also used where several initial components are shared.
  
     The "semantic content" (often abbreviated to just "content" when the
***************
*** 314,316 ****
  
!    A Netnews system is a distributed database composed of "agents" of
     various types which, acting together according to the protocols
--- 246,248 ----
  
!    A Netnews system is a distributed database composed of agents of
     various types which, acting together according to the protocols
***************
*** 320,323 ****
     wherever stored, are identical apart from those header fields defined
!    as variant (2.4).
  
     A "posting agent" is the software that assists posters to prepare
--- 252,257 ----
     wherever stored, are identical apart from those header fields defined
!    as variant (2.4).  For explaining the working of the protocols, it is
!    convenient to define particular sub-categories of agent as follows:
  
     A "posting agent" is the software that assists posters to prepare
***************
*** 346,347 ****
--- 280,283 ----
     agents to access the news database.
+ [There is a suggestion that "serving agent" should be changed to
+ "storage agent" throughout.]
  
***************
*** 361,405 ****
     Posting, reading and followup agents (which are usually just
!    different services provided by the same piece of software) are known
!    collectively as "user agents".
  
!    Injecting, relaying and serving agents (which are often just
!    different services provided by the same piece of software) are known
!    collectively as "news-servers".
  
! 2.3.  Identification of news-servers
  
!    News-servers need to identify themselves by inserting their public
!    name, in the form of a <path-identity> (F-3.1.6), into Path,
!    Injection-Info and Xref header fields. An injecting agent MUST
!    identify itself with the same <path-identity> in both Path and
!    Injection-Info header fields, and a serving agent SHOULD use the same
!    <path-identity> in both Path and Xref header fields.
  
!    The following possibilities are available when choosing a <path-
!    identity>, but some of them are less suited to providing a unique
!    identity for the news-server concerned and are NOT RECOMMENDED, as
!    shown:
  
!    1. A fully qualified domain name (FQDN) associated with an "A" or
!       "AAAA" record (or an equivalent "CNAME"), which SHOULD identify
!       the actual host inserting this <path-identity> and, ideally,
!       should also be "mailable" (see below).
  
!    2. An encoding of an IP address - <IPv4address> or <IPv6address> [RFC
!       3986] - which SHOULD be a publicly recognized address [RFC 1918]
!       for the actual host, as above. This option SHOULD NOT be used if
!       an FQDN for that host is available.
  
!    3. A fully qualified domain name (FQDN) associated with an "MX"
!       record, which MUST be "mailable".
  
!    4. Some other (arbitrary) name believed to be unique and registered
        at least with all other news-servers sending articles directly to
!       the given one. The news-server administrator is responsible for
!       choosing an appropriate name (and will bear the consequences of an
!       inappropriate choice). This option SHOULD NOT be used unless the
!       earlier options are unavailable (e.g. because the host in question
!       is not connected to the Internet), or unless it is of longstanding
!       usage and cessation would be unduly disruptive.
  
          NOTE: The syntax permits the colon character (which, prior to
--- 297,373 ----
     Posting, reading and followup agents (which are usually just
!    different services provided by the same piece of software) together
!    comprise the "user agents" defined in F-1.5.
  
!    Likewise, injecting, relaying and serving agents (which are often
!    just different services provided by the same piece of software)
!    together comprise the "news servers".
  
! 2.3.  Identification of news servers
  
! [The format of the Path header is still under discussion (ticket #1047).
! Hence the following texts are tentative, and will need to be changed (as
! will the associated protocols in 7.3).  Moreover, there are two
! alternative texts which have been proposed:]
  
!    In order to record the passage of articles through the network, news
!    servers need to identify themselves by means of a <path-identity>
!    (F-3.1.6), which can appear in Path, Injection-Info and Xref header
!    fields. Whatever <path-identity> is used in the Path header field
!    SHOULD be used also in any Injection-Info header field (and it would
!    be normal to use it in any Xref header field also).
! [Maybe that last sentence moves elsewhere.]
  
!         NOTE: Such <path-identity>s may also be suitable for sending
!         email to news server administrators (see [USEAGE]).
  
! [1st alternative]
  
!    <Path-identity>s can take the following forms (in decreasing order of
!    preference):
  
!    1. 1. A fully qualified domain name (FQDN) that SHOULD be resolvable
!       in the DNS (whether via an A, AAAA or MX record or an equivalent
!       CNAME), thus guaranteeing a unique identity. Ideally, it will also
!       provide a means to contact the administrators by email (according
!       to [RFC 2142], the forms "usenet@server" and "news@server" are
!       common addresses for a news server administrator).
! 
!    2. Some other (arbitrary) name believed to be unique and registered
!       at least with all other news servers sending articles directly to
!       the given one. This option SHOULD NOT be used unless the earlier
!       option is unavailable (e.g. because the server in question is not
!       connected to the Internet), or unless it is of longstanding usage
!       and cessation would be unduly disruptive, or unless the earlier
!       option is provided as well.
! 
! [2nd alternative]
! 
!    <Path-identity>s can take the following forms (in decreasing order of
!    preference):
! 
!    1. A fully qualified domain name (FQDN) that can be resolved to an
!       email server via an MX, A or AAAA record according to the
!       procedures of [RFC 2821]; this guarantees that the name is unique,
!       and makes it easy to contact the administrators if needed.
! 
!    2. A fully qualified domain name (FQDN) that is guaranteed to be
!       unique by the administrators of the domain; for instance, the
!       uniqueness of "server.example.org" could be guaranteed by the
!       administrator of "example.org" even if nothing is stored in the
!       DNS for that name.
! 
!    3. Some other (arbitrary) name believed to be unique and registered
        at least with all other news-servers sending articles directly to
!       the given one. This option SHOULD NOT be used unless the earlier
!       options are unavailable, or unless the name is of longstanding
!       usage and cessation would be unduly disruptive, or unless one of
!       the earlier options is provided as well.
  
+    According to [RFC 2142]], the forms "usenet@server" and "news@server"
+    are common addresses for a news server administrator.
+ [end of alternatives]
+ 
+         NOTE: A news server administrator who chooses a name which turns
+         out not to be unique will have to bear the consequences.
+ 
          NOTE: The syntax permits the colon character (which, prior to
***************
*** 407,409 ****
          identity> which is in the form of an <IPv6address>.  It would
!         therefor be unwise to choose, as such a name, anything composed
          solely from four (or less) hexadecimal digits.
--- 375,377 ----
          identity> which is in the form of an <IPv6address>.  It would
!         therefore be unwise to choose, as such a name, anything composed
          solely from four (or less) hexadecimal digits.
***************
*** 410,422 ****
  
-    The FQDN of a news-server is "mailable" if its administrators can be
-    reached by email using both of the forms "usenet@" that FQDN and
-    "news@" that FQDN, in conformity with [RFC 2142].
- 
-    For an injecting agent prepending to a Path header field (7.2.2), the
-    <path-identity> MUST be option 1 or 3 and the FQDN MUST be mailable,
-    and if the agent offers its services to the general public the form
-    "abuse@" that FQDN MUST also be available, unless a more specific
-    complaints address has been provided in a <complainto-param> of an
-    Injection-Info header field (F-3.2.14).
- 
  2.4.  Variant Header Fields
--- 378,379 ----
***************
*** 471,476 ****
  
-         NOTE: The use of "MUST" or "SHOULD" implies a requirement that
-         would or could lead to interoperability problems if not
-         followed.
- 
          NOTE: A requirement imposed on a relaying or serving agent
--- 426,427 ----
***************
*** 498,519 ****
     1036].  It is the intention that they can be assimilated into Usenet
!    as it presently operates without major interruption to the service,
!    though some of the new features may not begin to show benefit until
!    they become widely implemented. This section summarizes the main
!    changes, and comments on some features of the transition.
  
! 3.1.  Principal Changes
  
-      o The [RFC 2822] conventions for parenthesis-enclosed <comment>s in
-        header fields are supported in all newly defined header fields
-        and in header fields inherited from [RFC 2822].  They are,
-        however, still disallowed for performance and/or compatibility
-        reasons in the Message-ID, Newsgroups, Path, Followup-To,
-        Control, Supersedes, Distribution, Xref and Lines header fields.
-      o Whitespace is permitted in Newsgroups header fields, permitting
-        folding of such header fields. Indeed, all header fields can now
-        be folded.
-      o An enhanced syntax for the Path header field enables the
-        injection point of and the route taken by an article to be
-        determined with certainty.
-      o MIME is recognized as an integral part of Netnews.
       o There is a new Control message 'mvgroup' to facilitate moving a
--- 449,458 ----
     1036].  It is the intention that they can be assimilated into Usenet
!    as it presently operates without major interruption to the service
!    (3.2), though some of the new features may not begin to show benefit
!    until they become widely implemented.  Changes in the syntax and
!    format are documented in F-Appendix B and changes to control messages
!    and the protocols are documented below.
  
! 3.1.  Protocol Changes
  
       o There is a new Control message 'mvgroup' to facilitate moving a
***************
*** 520,528 ****
         group to a different place (name) in a hierarchy.
!      o There is a new mandatory Injection-Date header field to
!        facilitate the rejection of stale articles.
!      o There are new optional header fields defined, Archive,
!        Injection-Info and User-Agent, leading to increased
!        functionality.
!      o Certain header fields and Control messages (F-3.3 and Appendix
!        A.3) have been made obsolete.
       o Distributions are expected to be checked at the receiving end, as
--- 459,465 ----
         group to a different place (name) in a hierarchy.
!      o Certain Control messages (Appendix A) have been made obsolete,
!        and the special significance of "cmsg" when at the start of a
!        Subject header field has been removed (section 6).
!      o Additional media types are defined for better structuring of
!        control messages (5.3 and 5.4).
       o Distributions are expected to be checked at the receiving end, as
***************
*** 534,549 ****
  
!    An important distinction must be made between serving and relaying
!    agents, which are responsible for the distribution and storage of
!    news articles, and user agents, which are responsible for
!    interactions with users. It is important that the former should be
!    upgraded to conform to this standard as soon as possible to provide
!    the benefit of the enhanced facilities.  Fortunately, the number of
!    distinct implementations of such agents is rather small, at least so
!    far as the main "backbone" of Usenet is concerned, and many of the
!    new features are already supported. Contrariwise, there are a great
!    number of implementations of user agents, installed on a vastly
!    greater number of small sites. Therefore, the new functionality has
!    been designed so that existing user agents may continue to be used,
!    although the full benefits may not be realised until a substantial
!    proportion of them have been upgraded.
  
--- 471,486 ----
  
!    An important distinction must be made between news servers, which are
!    responsible for the distribution and storage of news articles, and
!    user agents, which are responsible for interactions with users. It is
!    important that the former should be upgraded to conform to this
!    standard as soon as possible to provide the benefit of the enhanced
!    facilities.  Fortunately, the number of distinct implementations of
!    such servers is rather small, at least so far as the main "backbone"
!    of Usenet is concerned, and many of the new features are already
!    supported. Contrariwise, there are a great number of implementations
!    of user agents, installed on a vastly greater number of small sites.
!    Therefore, the new functionality has been designed so that existing
!    user agents may continue to be used, although the full benefits may
!    not be realised until a substantial proportion of them have been
!    upgraded.
  
***************
*** 554,566 ****
  
!      o [RFC 2822] style <comment>s in header fields do not affect
!        serving and relaying agents.  They are unlikely to hinder their
!        proper display in existing reading agents except in the case of
!        the References header field in agents which thread articles.
!        Therefore, it is provided that they SHOULD NOT be generated in
!        that case.
!      o Because of its importance to all serving agents, the newly
!        permitted whitespace and folding in Newsgroups header fields
!        SHOULD NOT be generated (though it MUST be accepted); this
!        restriction may well be removed in a future version of this
         standard.
       o The new style of Path header field, using "!!" as a <path-
--- 491,505 ----
  
!      o [RFC 2822] style <comment>s have been prohibited in the case of
!        those header fields of particular concern to news servers. They
!        are unlikely to hinder their proper display in existing reading
!        agents except in the case of the References header field in
!        agents which thread articles. [USEFOR] therefore provides that
!        they SHOULD NOT be generated in that case.
!      o Because of its importance to all serving agents, the whitespace
!        and folding in Newsgroups header fields newly permitted by
!        [USEFOR] SHOULD NOT be generated (though it MUST be accepted);
!        this restriction may well be removed in a future version of this
         standard.
+ [That last bit needs discussion. It should probably be moved to USEFOR
+ if it is to be retained.]
       o The new style of Path header field, using "!!" as a <path-
***************
*** 571,581 ****
         agents are unaffected.
!      o The introduction of MIME reflects a practice that is already
!        widespread.  Articles in strict compliance with the previous
!        standards (using strict US-ASCII) will be unaffected. Many user
!        agents already support it, at least to the extent of widely used
!        charsets such as ISO-8859-1. Users expecting to read articles
!        using other charsets will need to acquire suitable reading
!        agents. It is not intended, in general, that any single user
!        agent will be able to display every charset known to IANA, but
!        all such agents MUST support US-ASCII. Serving and relaying
         agents are not affected.
--- 510,520 ----
         agents are unaffected.
!      o The introduction by [USEFOR] of MIME reflects a practice that is
!        already widespread.  Articles in strict compliance with the
!        previous standards (using strict US-ASCII) will be unaffected.
!        Many user agents already support it, at least to the extent of
!        widely used charsets such as ISO-8859-1. Users expecting to read
!        articles using other charsets will need to acquire suitable
!        reading agents. It is not intended, in general, that any single
!        user agent will be able to display every charset known to IANA,
!        but all such agents MUST support US-ASCII. Serving and relaying
         agents are not affected.
***************
*** 592,596 ****
         agents which do not yet provide an Injection-Date header field.
!      o All the header fields newly introduced by this standard can
!        safely be ignored by existing software, albeit with loss of the
!        new functionality.
  
--- 531,535 ----
         agents which do not yet provide an Injection-Date header field.
!      o All the header fields newly introduced by [USEFOR] can safely be
!        ignored by existing software, albeit with loss of the new
!        functionality.
  
***************
*** 1296,1298 ****
     line. [RFC 1036] also permitted the list of <msg-id>s to appear in
!    the Ihave- or Sendme-argument with the syntax
        Ihave-argument      = [FWS] *( msg-id FWS ) [relayer-name]
--- 1239,1243 ----
     line. [RFC 1036] also permitted the list of <msg-id>s to appear in
!    the <Ihave-> or <Sendme-argument> with the syntax
! 
! 
        Ihave-argument      = [FWS] *( msg-id FWS ) [relayer-name]
***************
*** 1355,1357 ****
  
!    The following control messages (as described in Appendix A.3) are
     declared obsolete by this standard:
--- 1302,1304 ----
  
!    The following control messages (as described in Appendix A) are
     declared obsolete by this standard:
***************
*** 1414,1417 ****
     is also expected to bear some responsibility towards the rest of the
!    network for the behaviour of its posters (and provision is therefore
!    made for it to be easily contactable by email).
  
--- 1361,1363 ----
     is also expected to bear some responsibility towards the rest of the
!    network for the behaviour of its posters.
  
***************
*** 1436,1439 ****
     of the network in these special cases, this standard does set out the
!    correct way to reinject (see special provisions in 7.2.2 Steps 3, 4,
!    7 and 9).
  
--- 1382,1385 ----
     of the network in these special cases, this standard does set out the
!    correct way to reinject (see special provisions in 7.2.2 Steps 3, 7
!    and 9).
  
***************
*** 1501,1504 ****
     4. It MUST reject any article that does not have the proper mandatory
!       header fields for a proto-article (except, when reinjecting, for
!       the Injection-Date header field), or which contains any header
        field that does not have legal contents.  It SHOULD reject any
--- 1448,1450 ----
     4. It MUST reject any article that does not have the proper mandatory
!       header fields for a proto-article or which contains any header
        field that does not have legal contents.  It SHOULD reject any
***************
*** 1531,1532 ****
--- 1479,1482 ----
        "INN/1.7.2") used by the injecting agent.
+ [That last sentence may need to be reconsidered (in which case see
+ consequential change in 7.3).]
  
***************
*** 1558,1561 ****
        this header field SHOULD then be folded if it would otherwise
!       result in a header line of excessive length.  The prepended
!       <path-identity> MUST be an FQDN mailable address (2.3).
  
--- 1506,1510 ----
        this header field SHOULD then be folded if it would otherwise
!       result in a header line of excessive length.
! [This may need further changes depending on the resolution of ticket
! #1047.]
  
***************
*** 1698,1699 ****
--- 1647,1651 ----
        length.
+ [This may need further changes depending on the resolution of ticket
+ #1047.]
  [It has been suggested that relaying agents should be permitted to
***************
*** 1700,1701 ****
--- 1652,1658 ----
  prepend more than the one or two entries permitted above.]
+ [something like the following from Diablo might also be useful:
+ >>> NOTE <<< you should grep through newly created spool directories
+ every so often looking for .MISMATCH in the spool files to locate
+ incoming feeds with improperly configured I found that four of my 80+
+ feeds were misconfigured. ]
  
***************
*** 2350,2351 ****
--- 2309,2314 ----
  
+    See also F-5 for further security considerations related to the
+    format of articles.
+ [And a similar pointer from there to here might be in order.]
+ 
  8.1.  Leakage
***************
*** 2497,2503 ****
     parts of the same article.
- [Frank wants
- MIME security considerations are discussed in [RFC2046].  Note that
- applying some [RFC2231] extensions for parameters like multi-line
- paramters on a boundary parameter as defined in [RFC2046] might be
- abused to bypass simple algorithms trying to analyze MIME parts.]
  
--- 2465,2466 ----
***************
*** 2556,2558 ****
          Coded Character Sets - 7-Bit American National Standard Code for
!         1996.
  
--- 2523,2525 ----
          Coded Character Sets - 7-Bit American National Standard Code for
!         Information Interchange (7-Bit ASCII)", ANSI X3.4, 1986.
  
***************
*** 2563,2565 ****
     [RFC 2119] S. Bradner, "Key words for use in RFCs to Indicate
!         Disposition Notifications", RFC 2298, March 1998.
  
--- 2530,2532 ----
     [RFC 2119] S. Bradner, "Key words for use in RFCs to Indicate
!         Requirement Levels", RFC 2119, March 1997.
  
***************
*** 2566,2568 ****
     [RFC 2606] D. Eastlake and A. Panitz, "Reserved Top Level DNS Names",
!         HTTP/1.1", RFC 2616, June 1999.
  
--- 2533,2535 ----
     [RFC 2606] D. Eastlake and A. Panitz, "Reserved Top Level DNS Names",
!         RFC 2606, June 1999.
  
***************
*** 2572,2574 ****
     [RFC 3864] G. Klyne, M. Nottingham, and J. Mogul, "Registration
!         <ftp://ftp.zoo.toronto.edu/pub/news.txt.Z>, June 1994.
  
--- 2539,2541 ----
     [RFC 3864] G. Klyne, M. Nottingham, and J. Mogul, "Registration
!         procedures for message header fields", RFC 3864.
  
***************
*** 2583,2589 ****
  10.2.  Informative References
-         Information Interchange (7-Bit ASCII)", ANSI X3.4, 1986.
  
-    [ARTICLE] Charles H. Lindsey, "News Article Format and Transmission",
-         draft-ietf-usefor-article-format-*.txt.
- 
     [NNTP] Clive D.W. Feather, "Network News Transport Protocol", draft-
--- 2550,2552 ----
***************
*** 2597,2602 ****
  
-    [RFC 1918] Y. Rekhter, B. Moskowitz, D. Karrenberg, G. J. de Groot,
-         and E. Lear, "Address Allocation for Private Internets", RFC
-         1918, February 1996.
- 
     [RFC 2045] N. Freed and N. Borenstein, "Multipurpose Internet Mail
--- 2560,2561 ----
***************
*** 2607,2609 ****
          Extensions (MIME) Part Two: Media Types", RFC 2046, November
!         Requirement Levels", RFC 2119, March 1997.
  
--- 2566,2568 ----
          Extensions (MIME) Part Two: Media Types", RFC 2046, November
!         1996.
  
***************
*** 2613,2615 ****
     [RFC 2298] R. Fajman, "An Extensible Message Format for Message
!         RFC 2606, June 1999.
  
--- 2572,2574 ----
     [RFC 2298] R. Fajman, "An Extensible Message Format for Message
!         Disposition Notifications", RFC 2298, March 1998.
  
***************
*** 2617,2627 ****
          P. Leach, and T. Berners-Lee, "Hypertext Transfer Protocol --
!         procedures for message header fields", RFC 3864.
  
!    [RFC 3986] T. Berners-Lee, R. Fielding, and L. Masinter, "Uniform
!         Resource Identifier (URI): Generic Syntax", STD 66, January
!         2005.
  
-    [RFC 850] Mark R. Horton, "Standard for interchange of Usenet
-         messages", RFC 850, June 1983.
- 
     [RFC 976] Mark R. Horton, "UUCP mail interchange format standard",
--- 2576,2582 ----
          P. Leach, and T. Berners-Lee, "Hypertext Transfer Protocol --
!         HTTP/1.1", RFC 2616, June 1999.
  
!    [RFC 2821] John C. Klensin and Dawn P. Mann, "Simple Mail Transfer
!         Protocol", RFC 2821, April 2001.
  
     [RFC 976] Mark R. Horton, "UUCP mail interchange format standard",
***************
*** 2630,2631 ****
--- 2585,2587 ----
     [Son-of-1036] Henry Spencer, "News article format and transmission",
+         <ftp://ftp.zoo.toronto.edu/pub/news.txt.Z>, June 1994.
  
***************
*** 2633,2640 ****
  
!    TBD
  
  12.  Contact Address
--- 2589,2596 ----
  
!    As this document is the result of an eight year effort, the number of
!    people that have contributed to its content are too numerous to
!    mention individually.  Many thanks go out to all past and present
!    members of the USEFOR Working Group of the Internet Engineering Task
!    Force (IETF) and the accompanying mailing list.
  
  12.  Contact Address
***************
*** 2650,2652 ****
          Phone: +44 161 436 6131
!         Email: [email-masked]
  
--- 2606,2608 ----
          Phone: +44 161 436 6131
!         Email: [email-masked]
  
***************
*** 2665,2727 ****
  
! Appendix A.1 - A-News Article Format
  
-    The obsolete "A News" article format consisted of exactly five lines
-    of header field information, followed by the body. For example:
- 
-       Aeagle.642
-       news.misc
-       cbosgd!mhuxj!mhuxt!eagle!jerry
-       Fri Nov 19 16:14:55 1982
-       Usenet Etiquette - Please Read
-       body
-       body
-       body
- 
-    The first line consisted of an "A" followed by an article ID
-    (analogous to a message identifier and used for similar purposes).
-    The second line was the list of newsgroups. The third line was the
-    path. The fourth was the date, in the format above (all fields fixed
-    width), resembling an Internet date but not quite the same. The fifth
-    was the subject.
- 
-    This format is documented for archaeological purposes only.  Articles
-    MUST NOT be generated in this format.
- 
- Appendix A.2 - Early B-News Article Format
- 
-    The obsolete pseudo-Internet article format, used briefly during the
-    transition between the A News format and the modern format, followed
-    the general outline of a MAIL message but with some non-standard
-    header fields. For example:
- 
-       From: cbosgd!mhuxj!mhuxt!eagle!jerry (Jerry Schwarz)
-       Newsgroups: news.misc
-       Title: Usenet Etiquette -- Please Read
-       Article-I.D.: eagle.642
-       Posted: Fri Nov 19 16:14:55 1982
-       Received: Fri Nov 19 16:59:30 1982
-       Expires: Mon Jan 1 00:00:00 1990
- 
-       body
-       body
-       body
- 
-    The From header field contained the information now found in the Path
-    header field, plus possibly the full name now typically found in the
-    From header field. The Title header field contained what is now the
-    content of the Subject header field. The Posted header field
-    contained what is now the content of the Date header field. The
-    Article-I.D. header field contained an article ID, analogous to a
-    message identifier and used for similar purposes. The Newsgroups and
-    Expires header fields were approximately as now. The Received header
-    field contained the date when the latest relaying agent to process
-    the article first saw it. All dates were in the above format, with
-    all fields fixed width, resembling an Internet date but not quite the
-    same.
- 
-    This format is documented for archaeological purposes only.  Articles
-    MUST NOT be generated in this format.
- 
- Appendix A.3 - Obsolete Control Messages
- 
     This present standard obsoletes certain control messages defined in
--- 2621,2624 ----
  
! Appendix A - Obsolete Control Messages
  
     This present standard obsoletes certain control messages defined in
***************
*** 2787,2789 ****
  
!    Copyright (C) The Internet Society (2005).  This document is subject
     to the rights, licenses and restrictions contained in BCP 78, and
--- 2684,2686 ----
  
!    Copyright (C) The Internet Society (2006).  This document is subject
     to the rights, licenses and restrictions contained in BCP 78, and
***************
*** 2941 ****
--- 2839,2856 ----
          6.2
+ 
+    For version 05
+ 
+    1    Historical Appendices A.1 and A.2 removed, in anticipation of
+         republication of Son-of-1036 as an Informational RFC.
+         1.3
+ 
+    2    Definitions of technical terms adopted from USEFOR rather than
+         defining them separately.
+ 
+    3    Discussion of <path-identity> rewritten to reflect recent
+         developments (but still awaiting further work in USEFOR).
+         2.3
+ 
+    4    Items now included in Appendix A of USEFOR have been removed
+         from section 3, and the "Transitional Arrangements" (which still
+         cover the USEFOR changes) have been modified to reflect this.

To: [email-masked]
Xref: clerew local.usefor:22882
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Distribution (was: Control: cancel ABNF bug)
Date: Fri, 6 Jan 2006 17:10:35 GMT
Lines: 46
Sender: [email-masked]

>Charles Lindsey wrote:

[from Son-of-1036]
>| Posting agents SHOULD not break headers unnecessarily.
>| Relayers SHOULD preserve existing header breaks, and SHOULD
>| not introduce new breaks.  Breaking headers SHOULD be a last
>| resort

Yes, I agree with that, and SHOULD[2] and SHOULD[3] are in fact MUSTs in
USEPRO. And if you want wording that discourages breaking when it serves
no useful purpose, then by all means let us have such wording.

But nevertheless, all software SHOULD/MUST be written to accept such
breaks as occur (whether useful or not). In the case of Control messages,
it is hard to imagine a case where breaking would be useful, and the sky
is unlikely to fall in if Russ leaves implementing that feature until he
has time to do it (which may be a long time off, but not "never"). And the
worst that can happen if some control messages _does_ get folded is that
some sites may not create/remove the group, and if their users care it
will soon get reported and fixed.

>...for some of our eight critical "no-CFWS" header fields it's
>_never_ necessary.

And for some it is, which is why we reduced CFWS to FWS (but no lower),
just to keep to a simple policy throughout.

>BTW, what you said elsewhere about WSP in a "Newsgroups" also
>affects WSP in "Distribution", s-o-1036 uses the same wording.
>IMHO we should keep this stuff consistent, dishonest or not.

Yes. A remark in the bit where it says SHOULD NOT use FWS in Newsgroups to
the effect "this also applies to Distribution" would be in order.

To: [email-masked]
Xref: clerew local.usefor:22879
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1158 Re: ISSUE: host-value
Date: Thu, 5 Jan 2006 17:16:40 GMT
Lines: 34
Sender: [email-masked]

>> Scared of changing stuff... despite my love of square
>> brackets...

>...then let's take the 1st "4234-beautified" variant and add
>a note "any <host-value> with a colon has to be quoted as per
>[RFC2045]".  It's too easy to get this wrong, and it's the
>main difference from the NNTP-Posting-Host header field body.

I am not sure that we need to draw attention in detail to ALL the
situations where quoting might be required, given the NOTE which warns you
of the general case.

BTW, here is a niggle for KEN that I just spotted. The example that is
currently given

   sender = "\"Joe Q. Public\" <[email-masked]>"

is actually a part of that NOTE, and therefore ought to be indented
accordingly.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Distribution (was: Control: cancel ABNF bug)
Date:  Fri, 06 Jan 2006 01:29:51 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 43
Sender: [email-masked]

Charles Lindsey wrote:

 [space in s-o-1036 is 1*WSP in 4234 or FWS in 2822]
> that was clearly not the intention, as should be clear
> from the last paragraph of 4.2.1 and from 4.2.3.

If that would be clear we wouldn't discuss it for ages:

| In MAIL, and arguably in RFC 1036 (although  the wording
| is vague), it is technically legitimate for the white
| space to be part of a continuation line rather than the
| start-line, but not all existing software will accept this.

That's about our "magic SP", and all my attempts to replace
our ":" SP [FWS] by a simple ":" FWS spectacularly failed...

| Posting agents SHOULD not break headers unnecessarily.
| Relayers SHOULD preserve existing header breaks, and SHOULD
| not introduce new breaks.  Breaking headers SHOULD be a last
| resort

...for some of our eight critical "no-CFWS" header fields it's
_never_ necessary.  That "SHOULD not" deteriorated into some
kind of "MUST NOT" for Control, Message-ID, and a few others,
Certainly not my fault, and I don't care _how_ we say it, but
saying only "FWS" where a folding will never work as expected
can't be a good idea.

BTW, what you said elsewhere about WSP in a "Newsgroups" also
affects WSP in "Distribution", s-o-1036 uses the same wording.
IMHO we should keep this stuff consistent, dishonest or not.

> Would you be upset to see
>
>    Reply-To: First <[email-masked]>,
>         Second <[email-masked]>             ?

Not at all, but Reply-To is none of or eight critical "no CFWS"
cases:  No news server or gateway is supposed to do anything
with it (ignoring the general issue of our "magic SP").

                              Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Diffs from USEFOR-06 as of 1/4/06
Date:  Thu, 05 Jan 2006 23:44:26 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 56
Sender: [email-masked]

Forrest J. Cavalier III wrote:

> The hoped for effect of implied endorsement through name
> dropping is not going to make a difference.

For me it did in some cases, looking at the credits I can
see if anybody I've heard of contributed.  E.g. something
about MIME or mail without JohnK, Keith, or Ned is highly
suspicious.  The sieve vacation draft can't be completely
wrong, it mentions 3834, and Ned is a co-author.

Or 3066bis:  It mentions almost everybody I've ever heard
of in conjunction with I18N, anything I18N without Martin,
JohnK, or Harald would be suspicious.

> It is notably dishonest to try to pass these documents
> off as products of Henry, Russ, et al, without mentioning
> the cooks who spoiled the broth.

If they don't like it they'd say NO, and that would be that.

I didn't propose Bruce (who fixed numerous ABNF issues) or
Ralph (who "terminated" the old -07 WG last call), because
I guess that they'd be indeed not happy with the result as
it is.  In the case of Ralph I hope that he likes it a bit
better than the old -07.

All controversial points are compromises between "common
message format first" and "common practice first", and I've
always supported the latter as stated in s-o-1036   If you
would prefer the former with CFWS and NO-WS-CTL everywhere
it's okay, but that would be very different from...

> If these documents had been left to Henry, Ned, and Russ,
> I'm sure they would have turned out much differently, and
> be relevant.

...I've always supported Henry and Russ whereever they agreed,
or only one of them offered an opinion.  IIRC the complaints-
URL (instead of an address) was the only exception.

It's excessively rude to say that this was "dishonest, sly,
manipulative, and spoiling the broth".  Actually I think that
the "broth" is rather good.  If the "Injection-Info" is a bit
"gibbous" then that's a necessary side-effect of 2231.  There
just was no complete better proposal, "Bruce and Frank tried"
doesn't count, and if anybody else tried I missed it, because
it was before tho old -07.

No idea who the cooks were before this time, but from my POV
they didn't get the Message-ID right, and that's fixed now -
compatible with NNTP, 2822, 1738, 1036 - as far as possible.

> this one just seemed too sly and manipulative to leave
> without comment.

To: [email-masked]
Xref: clerew local.usefor:22873
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Old ticket status - USEFOR - Jan 3, 2006
Date: Thu, 5 Jan 2006 16:40:17 GMT
Lines: 63
Sender: [email-masked]

In <[email-masked]> Ken Murchison <[email-masked]> writes:

>Charles Lindsey wrote:
>> 
>> And on the subject of 3.1.5, I think I have already mentioned that the
>> wording about "harming propagation" is not strictly correct as it is
>> written (was that one of the things Ken agreed to fix?).

>It doesn't look like I've made any changes.  What was your suggested text?

It was on Dec 19 when commenting on your list of diffs in 06, when I said:

>>    Folding the Newsgroups header field over several lines has been shown
>>    to harm propagation significantly......
> 
>Actually, it is worse than that. Even WSP will break existing usage, and
>it is more than "significantly". I think you have to make it clear that
>this is a newly introduced feature, hence "MUST accept" but "MUST NOT
>generate" (yet, for some value of "yet" :-) ).

[though I should have said "...SHOULD NOT generate (yet...)..."]

The present text says

   Folding the Newsgroups header field over several lines has been shown
   to harm propagation significantly.  Folded Newsgroups header fields
   SHOULD NOT be generated, but MUST be accepted.

and there are two things wrong with that:

1. It is not just folding that doesn't work, it is any whitespace anywhere
amongst the newsgroups.

2. It has not been "shown to harm propagation", but rather it never
propagated at all in strictly conforming software, because RFC 1036 just
did not permit it (and the decision to introduce it was one of the first
things the WG decided all those years ago).

So it would be more accurate to say something like:

   The [FWS] within the list of <newsgroup-name>s is newly introduced in
   this standard and will not be accepted by most current implementations.
   Therefore, although it MUST now be accepted, it SHOULD NOT be generated
   [though this restriction may well be removed in a future version of
   this standard].

where the words in [...] is an addition that I would like to see, just to
encourage implementors to take the MUST accept seriously (there has been
such wording in USEPRO for some time, but USEPRO is not the place for it).

And the restriction on <comment>s in References headers might say
something similar.

To: [email-masked]
Xref: clerew local.usefor:22874
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Control: cancel ABNF bug
Date: Thu, 5 Jan 2006 17:00:56 GMT
Lines: 51
Sender: [email-masked]

>> The sky will not fall in if we correct this anomaly (or,
>> rather, fail to introduce it).
>          ^^^^^^^^^^^^^^^^^^^^
>S-o-1036 sections 6.6 and 4:

>| Control-content  = verb *( space argument )
>                             ^^^^^
>| space            = 1*( <HT (ASCII 9)> / <blank (ASCII 32)> )

If you follow that argument, then Son-of-1036 would not be allowing you to
fold anything anywhere (because it uses "space" in that way in the syntax
of all headers). But that was clearly not the intention, as should be
clear from the last paragraph of 4.2.1 and from 4.2.3.

>RFC 1036 has it at the begin of section 3.  It's not straight
>forward to find and interpret the convoluted 1036 section 2:

>| Each header line consist of a keyword, a colon, a blank, and
>| some additional information.  This is a subset of the Internet
>| standard, simplified to allow simpler software to handle it.
>[...]
>| The Internet convention of continuation header lines
>| (beginning with a blank or tab) is allowed.

RFC 1036 has clearly contradicted itself there, in which case you can
argue that the rule which resolves conflicts in favour of RFC 822 should
be applied.

>If the whole world interpreted this as "no FWS in control" ...

But it is clear that the world HAS interpreted this as allowing FWS in
some places (In Subjects for sure and in References for sure and with
varying degrees of assurance in other places). Would you be upset to see

   Reply-To: First <[email-masked]>,
        Second <[email-masked]>             ?

To: [email-masked]
Xref: clerew local.usefor:22875
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: 3.2.14 : <sender>
Date: Thu, 5 Jan 2006 17:11:56 GMT
Lines: 31
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>It just so happens that I strongly disagree; "posting-account" (if done 
>like suggested here) preserves my anonymity to exactly the degree I've 
>chosen when picking my From: field and agreed with my sysadmin. "Sender" 
>does not.

>So if stating a preference, I'd do it the other way round.

>> It might also be useful to add something like:
>>
>> "It is up to the administrators of a news server to choose which of these
>> <parameter>(s) to include. This is discussed further in [USEAGE]."

>I agree that it fits better in [USEAGE]. Deleting the preference here and 
>adding the Note would satisfy me.

That is certainly a proper course if we cannot agree otherwise.

To: [email-masked]
Xref: clerew local.usefor:22871
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1032
Date: Thu, 5 Jan 2006 12:35:40 GMT
Lines: 23
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Reposted, apparently the string "cmsg cancel" anywhere in the
>subject, even as "Subject: #1032 (was: cmsg cancel)", does not
>work.  That's the third attempt to send this article after
>http://mid.gmane.org/[email-masked] and
>http://mid.gmane.org/[email-masked]

I received <[email-masked]> as well as
<[email-masked]> (which is the article I am replying to).

I did not see <[email-masked]>

To: [email-masked]
Xref: clerew local.usefor:22872
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Diffs from USEFOR-06 as of 1/4/06
Date: Thu, 5 Jan 2006 14:24:50 GMT
Lines: 118
Sender: [email-masked]

In <[email-masked]> Ken Murchison <[email-masked]> writes:

> 3.2.5.  Control
> 
>    The Control header field marks the article as a control message, and
>    specifies the desired actions (additional to the usual ones of
>    storing and/or relaying the article).
> 
>    control         =  "Control:" SP [FWS] control-command [FWS] CRLF
> 
>-   control-command =  verb *( [FWS] argument )
>+   control-command =  verb *( 1*WSP argument )

Yes, but we haven't agreed that WSP yet.

> 
>    verb            =  token
> 
>-   argument        =  value
>+   argument        =  1*( %x21-7E )

That seems OK

> 3.2.14.  Injection-Info
> 
>       NOTE: Since any such <host-value>>, <sender-value> or <address-
>       list> has also to be a syntactically correct <value>, it will
>-      usually be necessary to encapsulate is as a <quoted-string>, for
>+      usually be necessary to encapsulate it as a <quoted-string>, for
>       example:
> 
>+   posting-host = "an.example.de:::ad:be:ef"

Well if we are to have an example of a quoted-string in a posting-host,
then we don't need a confusing one like that (people will be looking for
all sorts of hidden subtleties that are not there, not realising that it
is that first ':' which necessitates the quoting). We need something like

     posing-host = "[email-masked]:123.456.123.456"
(but choose an IP address guaranteed not to exist, of course)
>+
>    sender = "\"Joe Q. Public\" <[email-masked]>"

>+6.  IANA Considerations
>+
>+   IANA is requested to register the following header fields in the
>+   Permanent Message Header Field Repository, in accordance with the
>+   procedures set out in [RFC3864].
>+
>+      Header field name: Also-Control
>+      Applicable protocol: netnews
>+      Status: historic
>+      Author/change controller: IETF ([email-masked])
>+      Specification document(s): [Son-of-1036] (Section 6.15)

I think I slightly prefer "obsolete" (or is it "obsoleted") to "historic"
for these.

Also, you need to say something about NNTP-Posting-Host and
NNTP-Posting-Date which definitely need to be recorded in the IANA
Registry as something that is no longer supposed to be used (whichever
of obsolete/historic you choose to use). The fact that they have no
Specification Document make it even more important to clarify their status
here (you would say "Specification document(s): None", of course).

>+      Header field name: Expires
>+      Applicable protocol: netnews
>+      Status: standard
>+      Author/change controller: IETF ([email-masked])
>+      Specification document(s): This document (Section 3.2.4),
>+      [RFC2156] (Section 5.3.4)

No, I don't think that is right. I would expect some other template
registered with "Applicable protocol: email" to mention RFC2156. but it
should not be mentioned here. The situation with regard to Supersedes and
User-Agent is similar, but you made (quite rightly) no additional
references there.

Note that the cases where you mention RFC2822 as well as "This Document"
are fine, because in those cases our definition relies (to varying
extents) on what is said in 2822.

>+      Header field name: Lines
>+      Applicable protocol: netnews
>+      Status: obsoleted
>+      Author/change controller: IETF ([email-masked])
>+      Specification document(s): This document (Section 3.3.1)
>+      Related information: [NNTP] (Section 8.1)

s/obsoleted/obsolescent/ (it is only "SHOULD NOT generate")

or s/obsoleted/deprecated/ if you prefer.

> Appendix A.  Acknowledgements
> 
>-   Comments and/or text were provided by Mark Crispin, Claus Faerber,
>-   Ned Freed, Andrew Gierth, Tony Hansen, Paul Hoffman, Simon Josefsson,
>-   Bruce Lilly, Pete Resnick, Henry Spencer, Russ Allbery, and Alexey
>-   Melnikov.
>+   As this document is the result of an eight year effort, the number of
>+   people that have contributed to its content are too numerous to
>+   mention individually.  Many thanks go out to all past and present
>+   members of the USEFOR Working Group of the Internet Engineering Task
>+   Force (IETF) and the accompanying mailing list.

Yes, I agree (though you might decide to list the various chairmen who
have watched over us). I have adopted that same text for USEPRO.

Date: Thu, 05 Jan 2006 09:17:38 -0500
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: [email-masked]
Subject: Re: Diffs from USEFOR-06 as of 1/4/06
Sender: [email-masked]

Forrest J. Cavalier III wrote:

> I've stopped reading and participating, (as have many others if the
> list traffic is accurate indication.)  But this one just seemed too
> sly and manipulative to leave without comment.

Hopefully those people that have stopped participating on the list will 
at least look over the document during WG/IETF last call.  If we produce 
a specification that the community will ignore (again?), then we have 
wasted everyone's time.

-- 
Kenneth Murchison
Systems Programmer
Carnegie Mellon University

Date: Thu, 05 Jan 2006 08:03:13 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: Control: cancel ABNF bug
Sender: [email-masked]

Russ Allbery wrote:
 > My goal here is to publish a *real* standard that people can actually
 > implement so that hopefully in the future we won't continue to have this
 > problem.
 >
> Anyway, I think Charles and I are probably not going to agree on this
> ever, and I don't want to prolong the argument on the list.  Please don't
> take future silence on my part on this point to be agreement.
> 

Ah, confirmation that the fantasy-land cooks have worn out Russ, too.

Strike up the band!  There is no reason that everyone need be troubled
by the reality of a sinking ship.

Date: Thu, 05 Jan 2006 07:56:31 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: Diffs from USEFOR-06 as of 1/4/06
Sender: [email-masked]

 >>-   Comments and/or text were provided by Mark Crispin, Claus Faerber,
 >>-   Ned Freed, Andrew Gierth, Tony Hansen, Paul Hoffman, Simon Josefsson,
 >>-   Bruce Lilly, Pete Resnick, Henry Spencer, Russ Allbery, and Alexey
 >>-   Melnikov.
>>+   the number of people that have contributed to its content are too
>>+   numerous to mention individually.
> 
> 

Frank Ellermann wrote:

> True, but I kind of like the huge 2822 "IETF who is who", and if you
> mention Henry, Ned, and Russ it will be noted in the Usenet community
> as no nonsense.  Otherwise they might decide to ditch it as irrelevant.
> USENET is seriously mad, worse than the IETF.  Maybe "special thanks"
> to Henry, Ned, and Russ.  After the huge IANA templates "keep it short
> and sweet" is anyway lost in space... :-(

The hoped for effect of implied endorsement through name dropping is
not going to make a difference.

It is notably dishonest to try to pass these documents off as products of Henry, 
Russ, et al, without mentioning the cooks who spoiled the broth.

If these documents had been left to Henry, Ned, and Russ, I'm sure they
would have turned out much differently, and be relevant.

I've stopped reading and participating, (as have many others if the
list traffic is accurate indication.)  But this one just seemed too
sly and manipulative to leave without comment.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Diffs from USEFOR-06 as of 1/4/06
Date:  Thu, 05 Jan 2006 03:19:27 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 218
Sender: [email-masked]

Ken Murchison wrote:

> my current diffs based on discussion on the list.

Great stuff, thanks.

> +   SP            = <see RFC4234 Appendix B.1>
> +
> +   Additionally, Section 3.1.3 updates the [RFC2822] definition of
> +   <msg-id>.

Standing out like a beacon, I like it.

>     User agents MAY accept and generate other MIME extension header
>     fields, and in particular SHOULD accept Content-Disposition [RFC2183]
>     and Content-Language [RFC3282].

Out of context I wonder why that SHOULD is no MUST, and why we talk about
it at all.  If a news server rejects articles with these header fields it
is broken, or what could be a justification to do this ?

>     The following header fields defined in this document do not allow
>     comments (CFWS):
>
>     Newsgroups
>     Path
>     Followup-to
>     Control
>     Supersedes
>     Distribution
>     Xref
>     Lines
>
>     This also applies to the following header field defined in [RFC2822]:
>
>     Message-ID

After the beacon in 1.4 another alert is unnecessary, I'd kill the lines:
-
-     This also applies to the following header field defined in [RFC2822]:
-

> +   Several of these headers are mainly of interest to news servers, and
> +   news servers often need to process these fields very rapidly.

Is 7 out of 9 "Several" or better "Most" ?  Nit:  s/headers/header fields/

>     dist-list       =  [FWS] dist-name
>                        *( [FWS] "," [FWS] dist-name ) [FWS]

Are the two internal [FWS] here approved by Russ ?  The whole distribution
concept is rather old, does it really work anywhere, except from "local" ?

>        example:
>
> +   posting-host = "an.example.de:::ad:be:ef"
> +
>     sender = "\"Joe Q. Public\" <[email-masked]>"

LOL, full disclosure, nobody can claim that we didn't tell them, nice.
Unfortunately the owner of example.de doesn't like jokes with his FQDN,
(got too much bogus mails), maybe we should pick a better example.

>     is absent, is correct).  If a news server can verify the sender of an
>     article, it SHOULD use this <parameter> in favor of the "posting-
>     account" <parameter>.

Yes, ticket #1159 is noted as new point in the "Issues to be addressed".

>     Relay-Version: version B 2.10 2/13/83; site cbosgd.UUCP
>     Posting-Version: version B 2.10 2/13/83; site eagle.UUCP
>     Date-Received: Friday, 19-Nov-82 16:59:30 EST
>
>     Relay-Version contained version information about the news server
>     that last processed the article.  Posting-Version contained version
>     information about the user agent that posted the article.  Date-
>     Received contained the date when the last news server to process the
>     article first saw it (in a slightly nonstandard format).
>
> +   These header fields are mentioned for historical purposes only.
> +   Agents SHOULD NOT generate articles containing these header fields.
> +

Actually I thought that we better get rid of pre-1036 history with those
three header fields:  S-o-1036 would cover this.  We reference it anyway,
as RFC or as you have it now.  The SHOULD NOT is dubious, s-o-1036 says:

"for archaeological purposes only.  Do not generate articles using them"

Some lines later you already have a MUST NOT for the complete 3.3 zoo,
adding a SHOULD NOT for the oldest and obscurest header fields wasn't
the idea.  Just delete everything from "Early" up to "In addition,"

>     In addition, this present standard obsoletes certain header fields
>     defined in [Son-of-1036]:

In other words start section 3.3 with "This present standard" etc.

>     Content-Type: multipart/mixed
>     (; boundary=foo ; xyz=");bOuNdArY*=''next%20part(")
>
>     Content-Type: multipart/digest;
> -   boundary (not=me) = ("yes ;-) simple (foo;bar") ; x-foo = xyzzy
> +    boundary (not=me) = ("yes ;-) simple (foo;bar") ; x-foo = xyzzy

Yes, but we want the proper folding in both examples.

> +      Author/change controller: IETF ([email-masked])

IMNSHO we can do without an address for IETF.  When SPF went from
"experimental" to "standards track" (before it was pushed back) I
checked that, and IIRC "change controller: IETF" is good enough.

> +      Specification document(s): This document (Section 3.2.9)

xml2rfc has a trick for RFCxxxx instead of "this document", IANA
can't say "this document" in their registry.  Nothing to worry
about, the RFC-editor and IANA will get the idea.

> +      Specification document(s): This document (Section 3.1.2),
> +      [RFC2822] (Section 3.6.1)

Are you sure that they want and need the section ?  Even if you're
sure they should already know where to find it for 2822, they have
this in RFC 4021.

> +      Header field name: Lines
> +      Applicable protocol: netnews
> +      Status: obsoleted

Better say "deprecated" or "obsolescent" as stated in 3.3.1, we need
"obsoleted" for the NNTP-Posting-*.

> +      Header field name: Message-ID
> +      Applicable protocol: netnews
> +      Status: standard
> +      Author/change controller: IETF ([email-masked])
> +      Specification document(s): This document (Section 3.1.3),
> +      [RFC2822] (Section 3.6.4)

No, our Mesaage-ID isn't specified in 2822.  There are overloaded
names for similar 2822-productions unfortunately, but otherwise it
is quite different, as stated in 1.4 (see above).  RFC 4021 uses...

         Related information: ...

...for such cases, e.g. many specification: 2822, related info: 822.

> +      Header field name: Organization

Here I miss NNTP-Posting-* with status "obsoleted".  Maybe you can
say "Specification: n/a" (or "trad.") with "related info: RFCxxxx
section 3.2.1 Injection-Date" for NNTP-Posting-Date.  Dito "3.2.14
Injection-Info" for NNTP-Posting-Host.

All this "standard business" is about getting the idea somehow to
as many readers as possible, hoping that it's implemented a.s.a.p.

>     [ISO.3166.1988]
>                International Organization for Standardization, "Codes for
>                the representation of names of countries, 3rd edition",
>                ISO Standard 3166, August 1988.

For LTRU (3066bis) we used the freshest version:

   [ISO3166-1]
              International Organization for Standardization, "ISO 3166-
              1:1997. Codes for the representation of names of countries
              and their subdivisions -- Part 1: Country codes", 1997.

xml2rfc-template available in
http://www.inter-locale.com/ID/draft-ietf-ltru-registry-14.xml

BTW, I think that 3166-1 qualifies as "normative", maybe also NNTP.

>     [PGPVERIFY]
>                Lawrence, D., "PGPverify", June 1999.

Is that the best we can do, no URL, only author + title ?

>     [USEPRO]   Lindsey, C., "News Article Architecture and Protocols",
>                draft-ietf-usefor-usepro-*.txt.

Only informative, sure ?

> +   the number of people that have contributed to its content are too
> +   numerous to mention individually.

True, but I kind of like the huge 2822 "IETF who is who", and if you
mention Henry, Ned, and Russ it will be noted in the Usenet community
as no nonsense.  Otherwise they might decide to ditch it as irrelevant.
USENET is seriously mad, worse than the IETF.  Maybe "special thanks"
to Henry, Ned, and Russ.  After the huge IANA templates "keep it short
and sweet" is anyway lost in space... :-(

> +   o  Certain header fields, notably Lines, have been made obsolete
>        (Section 3.3).

Notably Lines is only "deprecated" / "obsolescent" in 3.3.1.  Based on
1036 (appendix B is about 1036) we obsoleted not a single header field.
The stuff we removed is in s-o-1036, and the two NNTP-Hosting-*.  And
appendix B is NOT about s-o-1036, let alone any pre-1036 "archaeology".

> +   o  A SP (space) is REQUIRED after the colon (':') following header
>        field name.

Nit: s/header/the header/ or s/header/a header/ or similar.

I miss "removed Subject: cmsg" (#1032) and I think we should mention
that we seriously try to support IPv6 everywhere (in addition to 1036).

> +      *  The obsolete <zone> "GMT" MUST be accepted within a <date-
>           time>.

s/date-time/date&nbhy;time/ in the XML source.  Okay, essentially only
#1047 and some minor stuff pending.
                                   Tnx & bye, Frank

Date: Wed, 04 Jan 2006 15:03:27 -0500
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: [email-masked]
Subject: Diffs from USEFOR-06 as of 1/4/06
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------060203060506000805030206
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Attached are my current diffs based on discussion on the list.

-- 
Kenneth Murchison
Project Cyrus Developer/Maintainer
Carnegie Mellon University

--------------060203060506000805030206
Content-Type: text/x-patch;
 name="draft-ietf-usefor-usefor-06-07.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="draft-ietf-usefor-usefor-06-07.diff"

--- draft-ietf-usefor-usefor-06.unpg	2005-12-16 14:44:25.000000000 -0500
+++ draft-ietf-usefor-usefor-07.unpg	2006-01-04 14:25:47.000000000 -0500
@@ -1,25 +1,25 @@

 
 Usenet Format Working Group                            K. Murchison, Ed.
 Internet-Draft                                Carnegie Mellon University
 Obsoletes: 1036 (if approved)                                 C. Lindsey
-Expires: June 19, 2006                          University of Manchester
+Expires: July 8, 2006                           University of Manchester
                                                                  D. Kohn
                                                         Skymoon Ventures
-                                                       December 16, 2005
+                                                         January 4, 2006

                          Netnews Article Format
-                      draft-ietf-usefor-usefor-06
+                      draft-ietf-usefor-usefor-07
 
 Status of this Memo
 
    By submitting this Internet-Draft, each author represents that any
    applicable patent or other IPR claims of which he or she is aware
    have been or will be disclosed, and any of which he or she becomes
    aware will be disclosed, in accordance with Section 6 of BCP 79.
 
    Internet-Drafts are working documents of the Internet Engineering
    Task Force (IETF), its areas, and its working groups.  Note that
@@ -30,40 +30,54 @@
    and may be updated, replaced, or obsoleted by other documents at any
    time.  It is inappropriate to use Internet-Drafts as reference
    material or to cite them other than as "work in progress."
 
    The list of current Internet-Drafts can be accessed at
    http://www.ietf.org/ietf/1id-abstracts.txt.
 
    The list of Internet-Draft Shadow Directories can be accessed at
    http://www.ietf.org/shadow.html.
 
-   This Internet-Draft will expire on June 19, 2006.
+   This Internet-Draft will expire on July 8, 2006.
 
 Copyright Notice
 
-   Copyright (C) The Internet Society (2005).
+   Copyright (C) The Internet Society (2006).
 
 Abstract
 
    This document specifies the syntax of network news (Netnews) articles
    in the context of the "Internet Message Format" (RFC 2822) and
    "Multipurpose Internet Mail Extensions (MIME)" (RFC 2045).  This
    document supersedes RFC 1036, updating it to reflect current practice
    and incorporating incremental changes specified in other documents.
 
-Note to the RFC Editor
+Changes since draft-ietf-usefor-usefor-06
+
+   o  Addressed ticket #1155 (ABNF imports).
+
+   o  Addressed ticket #1156 (IANA registration for header fields per
+      RFC 3864).
+
+   o  Addressed ticket #1157 (new ABNF for Control:).
+
+   o  Addressed ticket #1158 (better ABNF for host-value).
+
+   o  Using RFC 4234 as a reference instead of RFC 2234.
+
+   o  Removed reference to RFC 0977 -- using only
+      draft-ietf-nntpext-base.
+
+   o  Added note about Expires being defined by RFC 2156.
+
+   o  Miscellaneous editorial changes.
 
-   The normative reference to RFC 2234 and the informative reference to
-   RFC 0977 may be replaced by draft-crocker-abnf-rfc2234bis and
-   draft-ietf-nntpext-base respectively should one or both of those of
-   those documents reach RFC status before this one.
 
 Changes since draft-ietf-usefor-usefor-05
 
    o  Resolved ticket #1003 (msg-id).
 
    o  Resolved ticket #1021 (Newsgroups description and exceptions).
 
    o  Resolved ticket #1028 (fixed ABNF for orig-date).
 
    o  Began adding text for ticket #1032 (diff from RFC 1036).
@@ -80,21 +94,21 @@
    o  Resolved ticket #1079 (header fields which don't permit CFWS).
 
    o  Applied text from ticket #1080 (Injection-Info MIME params).
 
    o  Resolved ticket #1082 (Approved header field semantics).
 
    o  Resolved ticket #1088 (Injection-Date mandatory/optional).
 
    o  Resolved ticket #1102 (Definition of "agents", etc).
 
-   o  Removed RFC 0821 as a normative reference.
+   o  Removed RFC 2821 as a normative reference.
 
    o  Miscellaneous editorial changes.

 Changes since draft-ietf-usefor-usefor-04
 
    o  Resolved ticket #1002 (updated references).
 
    o  Applied text from ticket #1003 (ABNF cleanup for msg-id).
 
@@ -203,28 +217,21 @@
 
    o  More exact ABNF for Archive and Injection-Info parameters.
 
    o  Complaints-To header is now an Injection-Info parameter.

 Issues to be addressed
 
    o  Ticket #1047: Path header field delimiters and components.
 
-   o  IANA considerations (the Klyne message header registry is now
-      official as RFC 3864)?.
-
-   o  Collected Syntax?
-
-   o  Merge more security issues?
-
-   o  Merge acknowledgments?
+   o  Ticket #1159: Advice on sender vs. posting account.

 Table of Contents
 
    1.  Introduction
      1.1.  Basic Concepts
      1.2.  Scope
      1.3.  Requirements Notation
      1.4.  Syntax Notation
      1.5.  Definitions
@@ -253,23 +260,24 @@
        3.2.9.  Approved
        3.2.10. Organization
        3.2.11. Xref
        3.2.12. Archive
        3.2.13. User-Agent
        3.2.14. Injection-Info
      3.3.  Obsolete Header Fields
        3.3.1.  Lines
    4.  Internationalization Considerations
    5.  Security Considerations
-   6.  References
-     6.1.  Normative References
-     6.2.  Informative References
+   6.  IANA Considerations
+   7.  References
+     7.1.  Normative References
+     7.2.  Informative References
    Appendix A.  Acknowledgements
    Appendix B.  Differences from RFC 1036 and its derivatives
    Appendix C.  Differences from RFC 2822
    Authors' Addresses
    Intellectual Property and Copyright Statements

 1.  Introduction
 
 1.1.  Basic Concepts
@@ -313,30 +321,52 @@
 
    This document uses terms that appear in capital letters.  The key
    words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
    "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document
    are to be interpreted as described in [RFC2119].
 
 1.4.  Syntax Notation
 
    Header fields defined in this specification use the Augmented Backus-
    Naur Form (ABNF) notation (including the Core Rules) specified in
-   [RFC2234] and many constructs defined in [RFC2822] and [RFC2045].
-   Section 3.1.3 updates the [RFC2822] definition of <msg-id>.
+   [RFC4234] and many constructs defined in [RFC2822], [RFC2045] as
+   amended by [RFC2231], and [RFC3986], specifically:
+
+   token         = <see RFC2045 Section 5.1>
+   dot-atom-text = <see RFC2045 Section 5.1>
 
-   The following constructs referenced by this document are defined in
-   [RFC2822]: <mailbox-list>, <mailbox>, <date-time>, <phrase>,
-   <address-list>, <utext>, <dot-atom-text>, <dot-atom>, <FWS>, <CFWS>
-   and <CRLF>.
+   parameter     = <see RFC2231 Section 7>
+   attribute     = <see RFC2231 Section 7>
 
-   The following constructs referenced by this document are defined in
-   [RFC2045] (as amended by [RFC2231]): <token>, <parameter>, <value>.
+   FWS           = <see RFC2822 Section 3.2.3>
+   CFWS          = <see RFC2822 Section 3.2.3>
+   atext         = <see RFC2822 Section 3.2.4>
+   dot-atom-text = <see RFC2822 Section 3.2.4>
+   phrase        = <see RFC2822 Section 3.2.6>
+   utext         = <see RFC2822 Section 3.2.6>
+   date-time     = <see RFC2822 Section 3.3>
+   mailbox       = <see RFC2822 Section 3.4>
+   mailbox-list  = <see RFC2822 Section 3.4>
+   address-list  = <see RFC2822 Section 3.4>
+   fields        = <see RFC2822 Section 3.6>
+
+   IPv6address   = <see RFC3986 Section 3.2.2>
+   IPv4address   = <see RFC3986 Section 3.2.2>
+
+   ALPHA         = <see RFC4234 Appendix B.1>
+   CRLF          = <see RFC4234 Appendix B.1>
+   DIGIT         = <see RFC4234 Appendix B.1>
+   DQUOTE        = <see RFC4234 Appendix B.1>
+   SP            = <see RFC4234 Appendix B.1>
+
+   Additionally, Section 3.1.3 updates the [RFC2822] definition of
+   <msg-id>.
 
 1.5.  Definitions
 
    An "article" is the unit of news, synonymous with an [RFC2822]
    "message".  A "proto-article" is one that has not yet been injected
    into the news system.  In constrast to an "article", a "proto-
    article" may lack some mandatory header fields.
 
    A "message identifier" (Section 3.1.3) is a unique identifier for an
    article, usually supplied by the "user agent" which posted it or,
@@ -397,30 +427,30 @@
    Section 2 defines the format of news articles.  Section 3 details the
    header fields necessary to make an article suitable for the Netnews
    environment.

 2.  Format
 
 2.1.  Base
 
    An article is said to be conformant to this specification if it
-   conforms to the format specified in [RFC2822] section 3 and to the
+   conforms to the format specified in [RFC2822] Section 3 and to the
    additional requirements of this specification.
 
-   An article that uses the obsolete syntax specified in section 4 of
+   An article that uses the obsolete syntax specified in Section 4 of
    [RFC2822], except for the two exceptions mentioned below, is NOT
    conformant to this specification.
 
    Articles are conformant if they use the <obs-phrase> construct (use
    of a phrase like "John Q. Public" without the use of quotes, see
-   [RFC2822] section 4.1) but agents MUST NOT generate productions of
+   [RFC2822] Section 4.1) but agents MUST NOT generate productions of
    such syntax.
 
    Articles are conformant if they use the "GMT" <zone>, as specified in
    Section 3.1.2.
 
    This document, and specifications that build upon it, specifies how
    to handle conformant articles.  Handling of non-conformant articles
    is outside the scope of this specification.
 
    Agents conforming to this specification MUST generate only conformant
@@ -428,27 +458,27 @@
 
    The text below uses ABNF to specify restrictions on the syntax
    specified in [RFC2822]; this grammar is intended to be more
    restrictive than the [RFC2822] grammar.  Articles must conform to the
    ABNF specified in [RFC2822].
 
    Articles must also conform to the restrictions specified here, both
    those that are expressed as text and those that are expressed as
    ABNF.
 
-      NOTE: Older Netnews specifications used the term "header" as a
-      synonym for what [RFC2822] calls "header field".  This document
-      follows the terminology in Section 2 of [RFC2822] in using the
-      terms "line", "header field", "header field name", "header field
-      body", and "folding", based on a belief that consistent
-      terminology among specifications that depend on each other makes
-      the specifications easier to use in the long run.
+      NOTE: Other specifications use the term "header" as a synonym for
+      what [RFC2822] calls "header field".  This document follows the
+      terminology in Section 2 of [RFC2822] in using the terms "line",
+      "header field", "header field name", "header field body", and
+      "folding", based on a belief that consistent terminology among
+      specifications that depend on each other makes the specifications
+      easier to use in the long run.
 
 2.2.  Header Fields
 
    All header fields in a news article are compliant with [RFC2822],
    however this specification is less permissive in what can be
    generated and accepted by news agents.  The syntax allowed for news
    articles is a strict subset of the "Internet Message Format", making
    all messages compliant with this specification inherently compliant
    with [RFC2822].  Note however that the converse is not guaranteed to
    be true in all cases.
@@ -509,59 +539,60 @@
    encodings within any <unstructured> header field, or within any
    <comment> or <phrase> permittted within any structured header field.
 
    User agents MAY accept and generate other MIME extension header
    fields, and in particular SHOULD accept Content-Disposition [RFC2183]
    and Content-Language [RFC3282].

 3.  News Header Fields
 
-   The following news header fields extend those defined in section 3.6
+   The following news header fields extend those defined in Section 3.6
    of [RFC2822]:
 
    fields          =/ *( newsgroups /
                          path /
                          injection-date /
                          followup-to /
                          expires /
                          control /
                          supersedes /
                          distribution /
                          summary /
                          approved /
                          organization /
                          xref /
                          archive /
                          user-agent /
-                         injection-info )
+                         injection-info /
+                         lines )
 
    Each of these header fields may occur at most once in a news article.
 
    The following header fields defined in this document do not allow
    comments (CFWS):
 
    Newsgroups
    Path
    Followup-to
    Control
    Supersedes
    Distribution
    Xref
    Lines
 
    This also applies to the following header field defined in [RFC2822]:
 
    Message-ID
 
-   Several of these headers are mainly of interest to servers, and
-   servers often need to process these fields very rapidly.
+   Several of these headers are mainly of interest to news servers, and
+   news servers often need to process these fields very rapidly.
 
 3.1.  Mandatory Header Fields
 
    Each news article conformant with this specification MUST have
    exactly one of each of the following header fields: From, Date,
    Message-ID, Subject, Newsgroups, Path.
 
 3.1.1.  From
 
    The From header field is the same as that specified in Section 3.6.2
@@ -579,32 +610,32 @@
    "GMT" zone.
 
    orig-date       =  "Date:" SP date-time CRLF
 
       NOTE: This specification does not change [RFC2822], which says
       that agents MUST NOT generate <date-time> constructs which include
       any zone names defined by <obs-zone>.
 
    Software that accepts dates with unknown timezones SHOULD treat such
    timezones as equivalent to "-0000" when comparing dates, as specified
-   in [RFC2822] section 4.3.
+   in [RFC2822] Section 4.3.
 
    Also note that these requirements apply wherever <date-time> is used,
    including Injection-Date and Expires in Section 3.2.1 and
    Section 3.2.4 respectively.
 
 3.1.3.  Message-ID
 
    The Message-ID header field contains a single unique message
    identifier.  Netnews is more dependent on message identifier
    uniqueness and fast comparison than Email is, and some news software
-   and standards [RFC0977] might have trouble with the full range of
+   and standards [NNTP] might have trouble with the full range of
    possible <msg-id>s permitted by [RFC2822]; this section therefore
    restricts the syntax of <msg-id> as compared to Section 3.6.4 of
    [RFC2822].  The global uniqueness requirement for <msg-id> in
    [RFC2822] is to be understood as applying across all protocols using
    such message identifiers, and across both Email and Netnews in
    particular.
 
    message-id      =  "Message-ID:" SP [FWS] msg-id [FWS] CRLF
 
    msg-id          =  "<" id-left "@" id-right ">"
@@ -745,53 +776,53 @@
    private prior agreement to do so.  However, such names MUST be
    accepted by news servers.
 
    The following <newsgroup-name>s are reserved, and MUST NOT be used as
    the name of a newsgroup:
 
    o  Groups whose first (or only) component is "example"
 
    o  The group "poster"
 
-   The following <newsgroup-name&gts have been used for specific
-   purposes in various implementations and protocols, and therefore MUST
-   NOT be used for the names of normal newsgroups.  They MAY be used for
-   their specific purpose, or by local agreement.
+   The following <newsgroup-name>s have been used for specific purposes
+   in various implementations and protocols, and therefore MUST NOT be
+   used for the names of normal newsgroups.  They MAY be used for their
+   specific purpose, or by local agreement.
 
-      Groups whose first (or only) component is "to"
+   o  Groups whose first (or only) component is "to"
 
-      Groups whose first (or only) component is "control"
+   o  Groups whose first (or only) component is "control"
 
-      Groups which contain (or consist only of) the component "all"
+   o  Groups which contain (or consist only of) the component "all"
 
-      Groups which contain (or consist only of) the component "ctl"
+   o  Groups which contain (or consist only of) the component "ctl"
 
-      The group "junk"
+   o  The group "junk"
 
       NOTE: "example.*" is reserved for examples in this and other
       standards; "poster" has a special meaning in the Followup-To
       header field; "to.*" is reserved for certain point-to-point
       communications in conjunction with the "ihave" control message
       [USEPRO]; "control.*" and "junk" have special meanings in some
       news-servers; "all" is used as a wildcard in some implementations;
       and "ctl" was formerly used to indicate a <control-command> within
       the Subject header field.
 
 3.1.6.  Path
 
    The Path header field indicates the route taken by an article since
    its injection into the Netnews system.  Each agent that processes an
    article is required to prepend one (or more) identities to this
    header field body.  This is primarily to enable news servers to avoid
    sending articles to sites already known to have them, in particular
    the site they came from, and additionally to permit tracing the route
-   articles take in moving over the network, and for gathering Usenet
+   articles take in moving over the network, and for gathering
    statistics.
 
    path            =  "Path:" SP path-list CRLF
 
    path-list       =  [FWS]
                       *( ( path-identity / path-keyword /
                            path-diagnostic ) [FWS]
                          path-delimiter [FWS] )
                       tail-entry [FWS]
 
@@ -808,25 +839,30 @@
    path-delimiter  =  "!" / "!!"
 
    A <path-identity> is a name identifying a site.  It takes the form of
    a domain name having one or more components separated by dots.
 
    Each <path-identity> in the <path-list> (excluding the one in the
    <tail-entry>) indicates, from right to left, the successive agents
    through which the article has passed.  The <path-keyword> "POSTED"
    indicates that the agent to its left injected the article.  The use
    of the <path-delimiter> "!!" indicates that the agent to its left
-   claims that the agent to its right was the verified source of the
+   verified the identity of the agent to its right before accepting the
    article (whereas the <path-delimiter> "!" implies no such claim).
    The <path-keyword> "MISMATCH" indicates that the agent to its right
    failed to be so verified.
 
+      NOTE: Historically, the <tail-entry> indicated the name of the
+      sender.  If not used for this purpose, the string "not-for-mail"
+      is often used instead (since at one time the whole path could be
+      used as a mail address for the sender).
+
       NOTE: Although case-insensitive, it is intended that the <path-
       keyword>s "POSTED" and "MISMATCH" should be in upper case, to
       distinguish them from the <path-identity>s which are traditionally
       in lower case.
 
    A <path-diagnostic> is an item inserted into the Path header for
    purposes other than to indicate the name of a site.  One commonly
    observed usage is to insert an IP address.  The colon (":") is
    permitted in order to allow IPv6 addresses to be inserted; note that
    this will cause interoperability problems at older sites that regard
@@ -870,22 +906,22 @@
    already expired by the time they arrive at some news server.
 
    This header field MUST be inserted whenever an article is injected.
    However, software that predates this standard does not use this
    header, and therefore agents MUST accept articles without the
    Injection-Date header field.
 
    injection-date  =  "Injection-Date:" SP date-time CRLF

-   See the remarks under Section 3.1.2 regarding the syntax of <date-
-   time> and the requirements and recommendations to which it is
+   See the remarks under Section 3.1.2 regarding the syntax of
+   <date-time> and the requirements and recommendations to which it is
    subject.
 
       NOTE: The <date-time> in this header field would normally be
       expected to be later than the <date-time> in the Date header
       field, but differences between the clocks on the various agents
       and other special circumstances might vitiate that; no provision
       is made for any such discrepancy to be corrected - better that the
       news server should just insert the correct time as it sees it.
 
    This header field is intended to replace the currently-used but
@@ -932,63 +968,66 @@
    insensitive forms such as "Poster".
 
 3.2.4.  Expires
 
    The Expires header field specifies a date and time when the article
    is deemed to be no longer relevant and could usefully be removed
    ("expired").
 
    expires         =  "Expires:" SP date-time CRLF
 
-   See the remarks under Section 3.1.2 regarding the syntax of <date-
-   time> and the requirements and recommendations to which it is
+   See the remarks under Section 3.1.2 regarding the syntax of
+   <date-time> and the requirements and recommendations to which it is
    subject.
 
+      NOTE: The Expires header field is also sometimes used in Email
+      with a similar meaning [RFC2156].
+
 3.2.5.  Control
 
    The Control header field marks the article as a control message, and
    specifies the desired actions (additional to the usual ones of
    storing and/or relaying the article).
 
    control         =  "Control:" SP [FWS] control-command [FWS] CRLF
 
-   control-command =  verb *( [FWS] argument )
+   control-command =  verb *( 1*WSP argument )
 
    verb            =  token
 
-   argument        =  value
+   argument        =  1*( %x21-7E )
 
    The verb indicates what action should be taken, and the argument(s)
    (if any) supply details.  In some cases, the body of the article may
    also contain details.  The legal verbs and respective arguments are
    discussed in the companion document, [USEPRO].
 
    An article with a Control header field MUST NOT also have a
    Supersedes header field.
 
 3.2.6.  Supersedes
 
    The Supersedes header field contains a message identifier specifying
    an article to be superseded upon the arrival of this one.  An article
    containing a Supersedes header field is equivalent to a "cancel"
    [USEPRO] control message for the specified article, followed
    immediately by the new article without the Supersedes header field.
 
    supersedes      =  "Supersedes:" SP [FWS] msg-id [FWS] CRLF
 
-   NOTE: There is no "c" in Supersedes.
+      NOTE: There is no "c" in Supersedes.
 
-   NOTE: The Supersedes header field defined here has no connection with
-   the Supersedes header field that sometimes appears in Email messages
-   converted from X.400 according to [RFC2156]; in particular, the
-   syntax here permits only one <msg-id> in contrast to the multiple
-   <msg-id>s in that Email version.
+      NOTE: The Supersedes header field defined here has no connection
+      with the Supersedes header field that sometimes appears in Email
+      messages converted from X.400 according to [RFC2156]; in
+      particular, the syntax here permits only one <msg-id> in contrast
+      to the multiple <msg-id>s in that Email version.
 
 3.2.7.  Distribution
 
    The Distribution header field specifies geographic or organizational
    limits on an article's propagation.
 
    distribution    =  "Distribution:" SP dist-list CRLF
 
    dist-list       =  [FWS] dist-name
                       *( [FWS] "," [FWS] dist-name ) [FWS]
@@ -1117,51 +1156,53 @@
 
 3.2.14.  Injection-Info
 
    The Injection-Info header field contains information provided by the
    injecting news server as to how an article entered the Netnews system
    and to assist in tracing its true origin.  It can also specify one or
    more addresses where complaints concerning the poster of the article
    may be sent.
 
    injection-info  =  "Injection-Info:" SP [CFWS] path-identity
-                      [CFWS] *(';' parameter) CRLF
+                      [CFWS] *( ";" parameter ) CRLF
 
       NOTE: The syntax of <parameter> ([RFC2045] as amended by
       [RFC2231]), taken in conjunction with the folding rules of
       [RFC0822], effectively allows [CFWS] to occur both before and
       after the <parameter>, and also on either side of its "=".
 
    The following table gives the <attribute> and the format of the
    <value> for each <parameter> defined for use with this header field.
    At most one occurrence of each such <parameter> is allowed.
 
    <attribute>              format of <value>
    --------------------     -----------------
    "posting-host"           a <host-value>
    "posting-account"        any <value>
    "sender"                 a <sender-value>
    "logging-data"           any <value>
    "mail-complaints-to"     an <address-list>
 
    where
 
-   host-value      =  dot-atom-text / [ dot-atom-text ":" ]
-                      ( IPv4address / IPv6address ) ;  see [RFC 3986]
+   host-value      =  dot-atom-text / IPv4address / IPv6address /
+                      (dot-atom-text ":" ( IPv4address / IPv6address ))
 
    sender-value    =  mailbox / "verified"
 
       NOTE: Since any such <host-value>>, <sender-value> or <address-
       list> has also to be a syntactically correct <value>, it will
-      usually be necessary to encapsulate is as a <quoted-string>, for
+      usually be necessary to encapsulate it as a <quoted-string>, for
       example:
 
+   posting-host = "an.example.de:::ad:be:ef"
+
    sender = "\"Joe Q. Public\" <[email-masked]>"
 
    Additionally, any other <parameter> whose <attribute> starts with
    "x-" MAY be used where the defined ones appear to be unsuitable, but
    other unlisted <parameter>s SHOULD NOT be used unless defined in
    extensions to this standard.
 
    Although comments and folding of white space are permitted throughout
    the Injection-Info header field, it is RECOMMENDED that folding is
    not used within any <parameter> (but only before or after the ";"
@@ -1173,31 +1214,31 @@
       X-Complaints-To, and others.  Once a news server uses Injection-
       Info, it should have no need to send these non-standard header
       fields.
 
    The "posting-host" <parameter> specifies the FQDN and/or IP address
    (IPv4address or IPv6address) of the host from which the news server
    received the article.
 
       NOTE: If the "posting-host" <parameter> identifies a dial-up
       point-of-presence, the "posting-account" or the "logging-data"
-      <parameter> may provide additional information about true origin
-      of the article.
+      <parameter> may provide additional information about the true
+      origin of the article.
 
    The "posting-account" <parameter> identifies the source from which
-   that news server received the article.  For security reasons, it
-   SHOULD be in a cryptic notation understandable only by the
-   administrator of the news server.
+   that news server received the article.  For security reasons and
+   other reasons, it SHOULD be in a cryptic notation understandable only
+   by the administrator of the news server.
 
    The "sender" <parameter> identifies the mailbox of the verified
    sender of the article (alternatively, it uses the token "verified" to
-   indicate that at least any addr-spec in the Sender header field of
+   indicate that at least any <addr-spec> in the Sender header field of
    the article, or in the From header field if the Sender header field
    is absent, is correct).  If a news server can verify the sender of an
    article, it SHOULD use this <parameter> in favor of the "posting-
    account" <parameter>.
 
    The "logging-data" <parameter> contains information (typically a
    session number or other non-persistent means of identifying a posting
    account) which will enable the true origin of the article to be
    determined by reference to logging information kept by the news
    server.
@@ -1214,20 +1255,23 @@
    Relay-Version: version B 2.10 2/13/83; site cbosgd.UUCP
    Posting-Version: version B 2.10 2/13/83; site eagle.UUCP
    Date-Received: Friday, 19-Nov-82 16:59:30 EST
 
    Relay-Version contained version information about the news server
    that last processed the article.  Posting-Version contained version
    information about the user agent that posted the article.  Date-
    Received contained the date when the last news server to process the
    article first saw it (in a slightly nonstandard format).
 
+   These header fields are mentioned for historical purposes only.
+   Agents SHOULD NOT generate articles containing these header fields.
+
    In addition, this present standard obsoletes certain header fields
    defined in [Son-of-1036]:
 
    Also-Control: cancel <[email-masked]>
    See-Also: <[email-masked]> <[email-masked]>
    Article-Names: comp.foo:charter
    Article-Updates: <[email-masked]>
 
    Also-Control indicated a control message that was also intended to be
    filed as a normal article.  See-Also listed related articles, but
@@ -1240,42 +1284,42 @@
    The header fields listed above are documented for historical purposes
    only.  Articles containing these header fields MUST NOT be generated.
    Persons writing new agents SHOULD ignore any former meanings of these
    header fields.
 
 3.3.1.  Lines
 
    The Lines header field indicates the number of lines in the body of
    the article.
 
-   lines           =  "Lines" ":" SP [FWS] 1*DIGIT [FWS] CRLF
+   lines           =  "Lines:" SP [FWS] 1*DIGIT [FWS] CRLF
 
    The line count includes all body lines, including the signature if
    any, including empty lines (if any) at the beginning or end of the
    body, and including the whole of all MIME message and multipart parts
    contained in the body (the single empty separator line between the
    header fields and the body is not part of the body).  The "body" here
    is the body as found in the posted article as transmitted by the user
    agent.
 
-   Historically, this header field was used by the [NNTP] overview
-   extension, but its use for this purpose is now deprecated.  As a
+   Historically, this header field was used by the [NNTP] Overview
+   facility, but its use for this purpose is now deprecated.  As a
    result, this header field is to be regarded as obsolescent, and it is
    likely to be removed entirely in a future version of this standard.
    Servers and clients SHOULD ignore it, and SHOULD NOT generate it.

 4.  Internationalization Considerations
 
    Internationalization of news article header fields and bodies is
    provided using MIME mechanisms discussed in Section 2.3.  Note that
-   the generation of internationalized <newsgroup name>s for use in
+   the generation of internationalized <newsgroup-name>s for use in
    header fields is not addressed in this document.

 5.  Security Considerations
 
    The news article format specified in this document does not provide
    any security services, such as confidentiality, authentication of
    sender, or non-repudiation.  Instead, such services need to be
    layered above, using such protocols as S/MIME [RFC3851] or PGP/MIME
    [RFC3156], or below, using secure versions of news transport
@@ -1294,28 +1338,220 @@
 
    MIME security considerations are discussed in [RFC2046].  Note that
    the full range of encodings allowed for parameters in [RFC2046] and
    [RFC2231] permits constructs that simple parsers may fail to parse
    correctly; examples of hard-to-parse constructs are:
 
    Content-Type: multipart/mixed
    (; boundary=foo ; xyz=");bOuNdArY*=''next%20part(")
 
    Content-Type: multipart/digest;
-   boundary (not=me) = ("yes ;-) simple (foo;bar") ; x-foo = xyzzy
+    boundary (not=me) = ("yes ;-) simple (foo;bar") ; x-foo = xyzzy
 
    Such differences in parsing may be used as part of an attack.

-6.  References
+6.  IANA Considerations
+
+   IANA is requested to register the following header fields in the
+   Permanent Message Header Field Repository, in accordance with the
+   procedures set out in [RFC3864].
+
+      Header field name: Also-Control
+      Applicable protocol: netnews
+      Status: historic
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): [Son-of-1036] (Section 6.15)
+
+      Header field name: Approved
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.9)
+
+      Header field name: Archive
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.12)
+
+      Header field name: Article-Names
+      Applicable protocol: netnews
+      Status: historic
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): [Son-of-1036] (Section 6.17)
+
+      Header field name: Article-Updates
+      Applicable protocol: netnews
+      Status: historic
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): [Son-of-1036] (Section 6.18)
+
+      Header field name: Comments
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2),
+      [RFC2822] (Section 3.6.5)
+
+      Header field name: Control
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.5)
+
+      Header field name: Date
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.1.2),
+      [RFC2822] (Section 3.6.1)
+
+      Header field name: Distribution
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.7)
+
+      Header field name: Expires
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.4),
+      [RFC2156] (Section 5.3.4)
+
+      Header field name: Followup-To
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.3)
+
+      Header field name: From
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.1.1),
+      [RFC2822] (Section 3.6.2)
+
+      Header field name: Injection-Date
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.1)
+
+      Header field name: Injection-Info
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.14)
+
+      Header field name: Keywords
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2),
+      [RFC2822] (Section 3.6.5)
+
+      Header field name: Lines
+      Applicable protocol: netnews
+      Status: obsoleted
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.3.1)
+      Related information: [NNTP] (Section 8.1)
+
+      Header field name: Message-ID
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.1.3),
+      [RFC2822] (Section 3.6.4)
+
+      Header field name: Newsgroups
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.1.5)
+
+      Header field name: Organization
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.10)
+
+      Header field name: Path
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.1.6)
+
+      Header field name: References
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.2),
+      [RFC2822] (Section 3.6.4)
+
+      Header field name: Reply-To
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2),
+      [RFC2822] (Section 3.6.2)
+
+      Header field name: See-Also
+      Applicable protocol: netnews
+      Status: historic
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): [Son-of-1036] (Section 6.16)
+
+      Header field name: Sender
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2),
+      [RFC2822] (Section 3.6.2)
+
+      Header field name: Subject
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.1.4),
+      [RFC2822] (Section 3.6.5)
+
+      Header field name: Summary
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.8)
+
+      Header field name: Supersedes
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.6)
+
+      Header field name: User-Agent
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.13)
+
+      Header field name: Xref
+      Applicable protocol: netnews
+      Status: standard
+      Author/change controller: IETF ([email-masked])
+      Specification document(s): This document (Section 3.2.11)
+
 
-6.1.  Normative References
+7.  References
+
+7.1.  Normative References
 
    [Errata]   "RFC Editor Errata".
 
    [RFC2045]  Freed, N. and N. Borenstein, "Multipurpose Internet Mail
               Extensions (MIME) Part One: Format of Internet Message
               Bodies", RFC 2045, November 1996.
 
    [RFC2046]  Freed, N. and N. Borenstein, "Multipurpose Internet Mail
               Extensions (MIME) Part Two: Media Types", RFC 2046,
               November 1996.
@@ -1332,157 +1568,158 @@
               Requirement Levels", BCP 14, RFC 2119, March 1997.
 
    [RFC2183]  Troost, R., Dorner, S., and K. Moore, "Communicating
               Presentation Information in Internet Messages: The
               Content-Disposition Header Field", RFC 2183, August 1997.
 
    [RFC2231]  Freed, N. and K. Moore, "MIME Parameter Value and Encoded
               Word Extensions: Character Sets, Languages, and
               Continuations", RFC 2231, November 1997.
 
-   [RFC2234]  Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax
-              Specifications: ABNF", RFC 2234, November 1997.
-
    [RFC2822]  Resnick, P., "Internet Message Format", RFC 2822,
               April 2001.
 
    [RFC3282]  Alvestrand, H., "Content Language Headers", RFC 3282,
               May 2002.
 
-6.2.  Informative References
+   [RFC4234]  Crocker, D. and P. Overell, "Augmented BNF for Syntax
+              Specifications: ABNF", RFC 4234, October 2005.
+
+7.2.  Informative References
 
    [ISO.3166.1988]
               International Organization for Standardization, "Codes for
               the representation of names of countries, 3rd edition",
               ISO Standard 3166, August 1988.
 
    [NNTP]     Feather, C., "Network News Transfer Protocol",
               draft-ietf-nntpext-base-*.txt.
 
    [PGPVERIFY]
               Lawrence, D., "PGPverify", June 1999.
 
    [RFC0822]  Crocker, D., "Standard for the format of ARPA Internet
               text messages", STD 11, RFC 822, August 1982.
 
-   [RFC0977]  Kantor, B. and P. Lapsley, "Network News Transfer
-              Protocol", RFC 977, February 1986.
-
    [RFC1036]  Horton, M. and R. Adams, "Standard for interchange of
               USENET messages", RFC 1036, December 1987.
 
    [RFC2156]  Kille, S., "MIXER (Mime Internet X.400 Enhanced Relay):
               Mapping between X.400 and RFC 822/MIME", RFC 2156,
               January 1998.
 
    [RFC2616]  Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
               Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
               Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
 
    [RFC3156]  Elkins, M., Del Torto, D., Levien, R., and T. Roessler,
               "MIME Security with OpenPGP", RFC 3156, August 2001.
 
    [RFC3851]  Ramsdell, B., "Secure/Multipurpose Internet Mail
               Extensions (S/MIME) Version 3.1 Message Specification",
               RFC 3851, July 2004.
 
+   [RFC3864]  Klyne, G., Nottingham, M., and J. Mogul, "Registration
+              Procedures for Message Header Fields", BCP 90, RFC 3864,
+              September 2004.
+
    [RFC3986]  Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
               Resource Identifier (URI): Generic Syntax", STD 66,
               RFC 3986, January 2005.
 
    [Son-of-1036]
               Spencer, H., "News Article Format and Transmission",
               June 1994.
 
    [USEAGE]   Lindsey, C., "Usenet Best Practice",
               draft-ietf-usefor-useage-*.txt.
 
    [USEPRO]   Lindsey, C., "News Article Architecture and Protocols",
               draft-ietf-usefor-usepro-*.txt.

 Appendix A.  Acknowledgements
 
-   Comments and/or text were provided by Mark Crispin, Claus Faerber,
-   Ned Freed, Andrew Gierth, Tony Hansen, Paul Hoffman, Simon Josefsson,
-   Bruce Lilly, Pete Resnick, Henry Spencer, Russ Allbery, and Alexey
-   Melnikov.
+   As this document is the result of an eight year effort, the number of
+   people that have contributed to its content are too numerous to
+   mention individually.  Many thanks go out to all past and present
+   members of the USEFOR Working Group of the Internet Engineering Task
+   Force (IETF) and the accompanying mailing list.

 Appendix B.  Differences from RFC 1036 and its derivatives
 
    This appendix contains a list of changes that have been made in the
    Netnews Article Format from earlier standards, specifically
    [RFC1036].
 
-      The [RFC2822] conventions for parenthesis-enclosed <comment>s in
+   o  The [RFC2822] conventions for parenthesis-enclosed <comment>s in
       header fields are supported in all newly defined header fields and
       in header fields inherited from [RFC2822].  They are, however,
       still disallowed for performance and/or compatibility reasons in
       the Message-ID, Newsgroups, Path, Followup-To, Control,
       Supersedes, Distribution, Xref and Lines header fields.
 
-      Whitespace is permitted in Newsgroups header fields.
+   o  Whitespace is permitted in Newsgroups header fields.
 
-      An enhanced syntax for the Path header field enables the injection
+   o  An enhanced syntax for the Path header field enables the injection
       point of, and the route taken by an article to be determined with
       more precision.
 
-      MIME is recognized as an integral part of Netnews.
+   o  MIME is recognized as an integral part of Netnews.
 
-      There is a new Injection-Date header to make the rejection of
+   o  There is a new Injection-Date header to make the rejection of
       stale articles more precise and to minimize spurious rejections.
 
-      There are several new optional header fields defined, notably
+   o  There are several new optional header fields defined, notably
       Archive, Injection-Info and User-Agent, leading to increased
       functionality.
 
-      Certain header fields, notably Lines, have been made obsolete
+   o  Certain header fields, notably Lines, have been made obsolete
       (Section 3.3).
 
-      There are numerous other small changes, clarifications and
+   o  There are numerous other small changes, clarifications and
       enhancements.

 Appendix C.  Differences from RFC 2822
 
    This appendix lists the differences between the syntax allowed by the
    Netnews Article Format (this document) as compared to the Internet
    Message Format, as specified in [RFC2822].
 
    The Netnews article format is a strict subset of the Internet Message
    Format; all Netnews articles conform to the syntax of [RFC2822].
 
    The following restrictions are important:
 
-      A SP (space) is REQUIRED after the colon (':') following header
+   o  A SP (space) is REQUIRED after the colon (':') following header
       field name.
 
-      A more restricted syntax of <msg-id> (to be used by the
+   o  A more restricted syntax of <msg-id> (to be used by the
       Message-ID, References, and Supersedes header fields) is defined.
 
-      The length of a msg-id MUST NOT exceed 250 octets.
+   o  The length of a <msg-id> MUST NOT exceed 250 octets.
 
-      Comments are not allowed in the Message-ID header field.
+   o  Comments are not allowed in the Message-ID header field.
 
-      The CFWS between <msg-id>s in the References header field is not
+   o  The CFWS between <msg-id>s in the References header field is not
       optional.
 
-      It is legal for a parser to not accept obsolete syntax, except
-      that:
+   o  It is legal for a parser to reject obsolete syntax, except that:
 
-         The <obs-phrase> construct MUST be accepted.
+      *  The <obs-phrase> construct MUST be accepted.
 
-         The obsolete <zone> "GMT" MUST be accepted within a <date-
+      *  The obsolete <zone> "GMT" MUST be accepted within a <date-
          time>.
 
-      Every line of a header field body (including the first and any
+   o  Every line of a header field body (including the first and any
       that are subsequently folded) MUST contain at least one non-
       whitespace character.  This means that an empty header field body
       is illegal.

 Authors' Addresses
 
    Kenneth Murchison (editor)
    Carnegie Mellon University
    5000 Forbes Avenue
@@ -1547,21 +1784,21 @@
    "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
    OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
    ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
    INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
    INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
    WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

 Copyright Statement
 
-   Copyright (C) The Internet Society (2005).  This document is subject
+   Copyright (C) The Internet Society (2006).  This document is subject
    to the rights, licenses and restrictions contained in BCP 78, and
    except as set forth therein, the authors retain all their rights.

 Acknowledgment
 
    Funding for the RFC Editor function is currently provided by the
    Internet Society.

--------------060203060506000805030206--

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Control: cancel ABNF bug
Organization: The Eyrie
Date: Wed, 04 Jan 2006 10:10:34 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> RFC 2822, and ALL its predecessors, established the simple convention
> that you could fold wherever whitespace could occur. It is my belief
> that RFC 1036 bought into that same convention (hence why I have always
> argued that Path was foldable under RFC 1036, as opposed to Newsgroups
> which clearly was not).

RFC 1036 is not the protocol that Usenet currently uses.  Usenet currently
uses a protocol that is a combination of RFC 1036, son-of-1036, and some
things that are in neither.  RFC 1036 was somewhat poorly written and it's
very difficult to determine in some cases exactly what it's specifying,
but it also doesn't really matter.  Backward compatibility at this point
means compatibility with the deployed software, not with what RFC 1036 and
tea leaves indicate the Usenet standard was supposed to be.

My goal here is to publish a *real* standard that people can actually
implement so that hopefully in the future we won't continue to have this
problem.

Anyway, I think Charles and I are probably not going to agree on this
ever, and I don't want to prolong the argument on the list.  Please don't
take future silence on my part on this point to be agreement.

Date: Wed, 04 Jan 2006 18:12:40 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: ISSUE: 3.2.14 : <sender>
Sender: [email-masked]

Speaking as contributor:

--On onsdag, januar 04, 2006 12:23:11 +0000 Charles Lindsey 
<[email-masked]> wrote:

> So I could accept a weaker formulation of that sentence, such as
>
> "... Use of this <parameter> is prefereable to a "posting-account"
> <parameter> in cases when a news server can verify the sender of an
> article."

It just so happens that I strongly disagree; "posting-account" (if done 
like suggested here) preserves my anonymity to exactly the degree I've 
chosen when picking my From: field and agreed with my sysadmin. "Sender" 
does not.

So if stating a preference, I'd do it the other way round.

> It might also be useful to add something like:
>
> "It is up to the administrators of a news server to choose which of these
> <parameter>(s) to include. This is discussed further in [USEAGE]."

I agree that it fits better in [USEAGE]. Deleting the preference here and 
adding the Note would satisfy me.

To: [email-masked]
Xref: clerew local.usefor:22855
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Date: Wed, 4 Jan 2006 13:29:27 GMT
Lines: 55
Sender: [email-masked]

>[splitting my attempted all-in-one-reply, ideas about #1047]

>> 1047 USEFOR 3.1.6: Path field delimiters and components
>[...]
>> Still open whether USEFOR should specify syntax that allows
>> one to tell the difference between a path-identity and a
>> diagnostic.

>As it is the syntax is ambiguous.  We've seen at least two
>more complete solutions, one by Charles, another by me, but
>it's somewhat hard to get the ABNF right while it's still
>unclear what we really want.

Exactly. The things we need to know (in decreasing order of urgency) are:

1. Do we want to be able to tell the difference betweeen a diagnostic and
a server?

1a. If so, is another keyword ("SEEN" has been suggested) the way to do
it?

2. What is the full list of keywords we want (in particular, do we omit
"MATCH")? Or do we make the list extensible as Harald proposed?

3. Which keyword notation? (!foo.bar!MISMATCH! or !foo.bar..MISMATCH!, or
whatever else).

4. Do we enforce ':' ONLY in Ipv6Addresses (whether by explicit syntax or
MUSTard)?

5. Remaining syntactic niggles, such as maybe disallowing
!diagnostic!diagnostic! and where [FWS] is allowed (certainly before
delimiters, and maybe afterwards as well).

>Could we somehow agree that that's precisely what we want ?
>Then Charles could cook up some corresponding syntax, and
>I debug it, or vice versa,

Indeed. Even answering the first two questions would enable some progress.

To: [email-masked]
Xref: clerew local.usefor:22854
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: 3.2.14 : <sender>
Date: Wed, 4 Jan 2006 12:23:11 GMT
Lines: 48
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>- If a news server can verify the sender of an article, it
>- SHOULD use this <parameter> in favor of the "posting-
>- account" <parameter>.

>Charles reported that this is an old consensus, but I think
>it's wrong.  Technically news servers should be free to use
>either method.  For privacy reasons they might prefer the
><posting-account> approach.

They ARE free to use either. The agreed paragraph mentioning the privacy
issues is now in USEAGE, which is the right place for it.

I would agree, however, that the sentence with that "SHOULD" does go
against the idea that it is an implementor's choice. And in practice
the possibility for an injecting agent to know the email address of the
sender is restricted to organizations which run their own injecting agent
(so it would not apply to an article injected with an NNTP POST command,
unless it could be picked up from a SASL authentication).

So I could accept a weaker formulation of that sentence, such as

"... Use of this <parameter> is prefereable to a "posting-account"
<parameter> in cases when a news server can verify the sender of an
article."

It might also be useful to add something like:

"It is up to the administrators of a news server to choose which of these
<parameter>(s) to include. This is discussed further in [USEAGE]."

>+ Typically a news server uses either a "posting-account"
>+ or a verified "sender" parameter, inserting the <mailbox>
>+ has implications for the privacy of users.

But no. Let us leave mention of privacy oissues to USEAGE.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Control: cancel ABNF bug
Date:  Wed, 04 Jan 2006 16:06:37 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 40
Sender: [email-masked]

Charles Lindsey wrote:

> The sky will not fall in if we correct this anomaly (or,
> rather, fail to introduce it).
          ^^^^^^^^^^^^^^^^^^^^
S-o-1036 sections 6.6 and 4:

| Control-content  = verb *( space argument )
                             ^^^^^
| space            = 1*( <HT (ASCII 9)> / <blank (ASCII 32)> )

RFC 1036 has it at the begin of section 3.  It's not straight
forward to find and interpret the convoluted 1036 section 2:

| The USENET News standard is more restrictive than the
| Internet standard, placing additional requirements on each
| message and forbidding use of certain Internet features.
[...]
| Each header line consist of a keyword, a colon, a blank, and
| some additional information.  This is a subset of the Internet
| standard, simplified to allow simpler software to handle it.
[...]
| The Internet convention of continuation header lines
| (beginning with a blank or tab) is allowed.

If the whole world interpreted this as "no FWS in control" and
we say something else in USEFOR or USEPRO, then this won't fly.

Or as Henry put it (and maybe we should copy his statement):

  Several  of  the mechanisms described in this Draft may seem
  somewhat strange or even bizarre at first reading.  As  with
  Internet  mail, there is no reasonable possibility of updat-
  ing the entire installed base of news software promptly,  so
  interoperability  with  old  software  is  crucial  and will
  remain so.  Compatibility with existing practice and robust-
  ness  in  an  imperfect world necessarily take priority over
  elegance.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  #1032
Date:  Wed, 04 Jan 2006 16:08:56 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 53
Sender: [email-masked]

Charles Lindsey wrote:

>> Add:

>>| The obsolete convention to interpret subjects starting
>>| with the word "cmsg" as control message was removed.

> No, you cannot say that in Appendix B of USEFOR, because the
> change is not mentioned in USEFOR anywhere.

It is a side effect of 3.1.4 <subject> and 2.2 <unstructured>,
and adding the proposed statement to appendix B will fix the
issue that it's not yet explicitly mentioned anywhere.

Appendix B lists all differences in the "article format" from
1036, and this is a very important difference.  I used the old
"Subject: cmsg cancel" as feature on one news server in 2002,

I got an "administrative restriction" on IIRC GMaNe about it
later, and the Google archive ignored all articles with this
form of a subject, even including "Subject: Re: cmsg cancel"
in ordinary articles (no control messages).  That's broken,
and USEFOR fixes it, it has to be mentioned in appendix B.

> It IS mentioned in USEPRO

That's good, it's important enough to mention it everywhere.

Testing to change the subject here. new subject:
cmsg cancel (was: Old ticket status - USEFOR - Jan 3, 2006)

--- new ---
Well, if I didn't screw up that simply didn't work, I don't
see that article <[email-masked]> on GMaNe:

http://mid.gmane.org/[email-masked]

Maybe you got it, maybe you didn't, the list archive doesn't
work in real time, so far I don't see it:

http://www.imc.org/ietf-usefor/mail-archive/maillist.html

Maybe it's a GMaNe bug.  But above all it's a 1036 oddity
removed by USEFOR for good.  Therefore we MUST mention this.

--- new ---

Reposted, apparently the string "cmsg cancel" anywhere in the
subject, even as "Subject: #1032 (was: cmsg cancel)", does not
work.  That's the third attempt to send this article after
http://mid.gmane.org/[email-masked] and
http://mid.gmane.org/[email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1158 colon okay 
Date:  Wed, 04 Jan 2006 15:27:42 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 11
Sender: [email-masked]

Charles Lindsey wrote:

> an Ipv6Address cannot begin with a ":"

JFTR, it can begin with two colons ::ad:be:ef resulting
in say posting-host = "an.example.de:::ad:be:ef"
 
> So let it be
 
...ACK.  Let's add a note "colon => quoted because of 2045".

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  #1032 (was: cmsg cancel)
Date:  Wed, 04 Jan 2006 14:51:40 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 45
Sender: [email-masked]

Charles Lindsey wrote:

>> Add:

>>| The obsolete convention to interpret subjects starting
>>| with the word "cmsg" as control message was removed.

> No, you cannot say that in Appendix B of USEFOR, because the
> change is not mentioned in USEFOR anywhere.

It is a side effect of 3.1.4 <subject> and 2.2 <unstructured>,
and adding the proposed statement to appendix B will fix the
issue that it's not yet explicitly mentioned anywhere.

Appendix B lists all differences in the "article format" from
1036, and this is a very important difference.  I used the old
"Subject: cmsg cancel" as feature on one news server in 2002,

I got an "administrative restriction" on IIRC GMaNe about it
later, and the Google archive ignored all articles with this
form of a subject, even including "Subject: Re: cmsg cancel"
in ordinary articles (no control messages).  That's broken,
and USEFOR fixes it, it has to be mentioned in appendix B.

> It IS mentioned in USEPRO

That's good, it's important enough to mention it everywhere.

Testing to change the subject here. new subject:
cmsg cancel (was: Old ticket status - USEFOR - Jan 3, 2006)

--- new ---
Well, if I didn't screw up that simply didn't work, I don't
see that article <[email-masked]> on GMaNe:

http://mid.gmane.org/[email-masked]

Maybe you got it, maybe you didn't, the list archive doesn't
work in real time, so far I don't see it:

http://www.imc.org/ietf-usefor/mail-archive/maillist.html

Maybe it's a GMaNe bug.  But above all it's a 1036 oddity
removed by USEFOR for good.  Therefore we MUST mention this.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  #1158 colon okay (was: ABNF)
Date:  Wed, 04 Jan 2006 14:44:20 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 16
Sender: [email-masked]

Charles Lindsey wrote:

> not ambiguous since an Ipv6Address cannot begin with a
> ":". I tried to construct an ambiguity supposing that
> ICANN might have invented a TLD "beef", but I could not
> do it even then.

Okay, I tried an.example.de:be:af:: (existing <h16> ccTLDs)

It should be clear that that's an.example.de [be:af::] and
not an.example. [de:be:af::] with a trailing dot...

> So let it be

...ACK.  Let's add a note "colon => quoted because of 2045".

Date: Wed, 04 Jan 2006 08:44:12 -0500
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: Old ticket status - USEFOR - Jan 3, 2006
Sender: [email-masked]

Charles Lindsey wrote:
> 
> And on the subject of 3.1.5, I think I have already mentioned that the
> wording about "harming propagation" is not strictly correct as it is
> written (was that one of the things Ken agreed to fix?).

It doesn't look like I've made any changes.  What was your suggested text?

-- 
Kenneth Murchison
Project Cyrus Developer/Maintainer
Carnegie Mellon University

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1158 Re: ISSUE: host-value
Date:  Wed, 04 Jan 2006 14:02:51 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 10
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> Scared of changing stuff... despite my love of square
> brackets...

...then let's take the 1st "4234-beautified" variant and add
a note "any <host-value> with a colon has to be quoted as per
[RFC2045]".  It's too easy to get this wrong, and it's the
main difference from the NNTP-Posting-Host header field body.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1159 Re: ISSUE: 3.2.14 : <sender>
Date:  Wed, 04 Jan 2006 13:56:56 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 10
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> s/If a ... SHOULD ..// (delete the entire requirement) would > make me happy.

Me too, but Charles hated it.  Maybe s/SHOULD/should/ could do.

Or s/SHOULD use/would consider to use/.  Scott prefers to avoid
lower case 2119 keywords whereever possible.

Date: Wed, 04 Jan 2006 13:48:14 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: #1159 Re: ISSUE: 3.2.14 : <sender>
Sender: [email-masked]

Added.

--On tirsdag, januar 03, 2006 18:23:20 +0100 Frank Ellermann 
<[email-masked]> wrote:

>> if anyone thinks that there are more issues that should be
>> tracked
>
> Draft -06 says:
>
> - If a news server can verify the sender of an article, it
> - SHOULD use this <parameter> in favor of the "posting-
> - account" <parameter>.
>
> Charles reported that this is an old consensus, but I think
> it's wrong.  Technically news servers should be free to use
> either method.  For privacy reasons they might prefer the
> <posting-account> approach.
>
> + Typically a news server uses either a "posting-account"
> + or a verified "sender" parameter, inserting the <mailbox>
> + has implications for the privacy of users.
>
> BTW, it's possible that some kind of old consensus included
> me, in that case I've changed my mind:  "Killfiling" is an
> important feature for those who have it, but enforcing that
> possibility with a SHOULD goes to far.

"posting-account" mentions privacy. Offhand, "sender" seems more sensitive 
than "posting-account".

s/If a ... SHOULD ..// (delete the entire requirement) would make me happy.

Date: Wed, 04 Jan 2006 13:41:13 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: #1158 Re: ISSUE: host-value
Sender: [email-masked]

Assigned ticket #1158.

--On tirsdag, januar 03, 2006 17:58:19 +0100 Frank Ellermann 
<[email-masked]> wrote:

>> please send an email to the list with ONE issue and the word
>> "ISSUE:" in the subject line.
>
> A clearer ABNF for <host-value> as recommended in 4234...
>
> -   host-value      =  dot-atom-text / [ dot-atom-text ":" ]
> -                      ( IPv4address / IPv6address ) ;  see [RFC 3986]
>
> + host-value = dot-atom-text / IPv4address / IPv6address /
> +              ( dot-atom-text ":" ( IPv4address / IPv6address ))

>
> ...or a "better" (?) ABNF in the style of 2821/3986 IP-literal:
>
> + IP-literal  = "[" ( IPv4address / IPv6address ) "]"
> +
> + host-value  = dot-atom-text / IP-literal /
> +               ( dot-atom-text IP-literal )
>

This is different from the previous version, which didn't have the square 
brackets. Scared of changing stuff... despite my love of square brackets...

Date: Wed, 04 Jan 2006 13:31:47 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: IANA considerations
Sender: [email-masked]

--On tirsdag, januar 03, 2006 17:21:25 +0000 Charles Lindsey 
<[email-masked]> wrote:

> Putting them in a table is much more user friendly. Essentially, it is a
> rule whereby you could construct each individual template if you really
> wanted to, but what is the point?
>
> So my view is that we should try to publish them in table form, and if the
> IETF tell us to rewrite them, or instruct the RFC-Editor to expand them in
> full, then we should worry about it if/when that happens.

We follow the rule. We write templates.

I like the table too, as a summary. But we follow the rule, until the rule 
changes.

                     Harald

To: [email-masked]
Xref: clerew local.usefor:22849
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Old ticket status - USEFOR - Jan 3, 2006
Date: Wed, 4 Jan 2006 11:25:08 GMT
Lines: 52
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>1032 USEFOR general: Document changes from RFC 1036

>   Text in Appendix B. Please speak up if this is not sufficient, or needs
>   modification - otherwise, I'll just close this.

It says:

      Whitespace is permitted in Newsgroups header fields.

One might say "Whitespace (and hence folding) is permitted ..."

One might also add that it SHOULD NOT be generated (yet).

BTW, USEPRO has always said that "SHOULD NOT be generated" restriction
might be lifted in some future standard. Should we say that in 3.1.5, just
to encourage implemetors to take the "MUST be accepted" seriously? Ditto
for the comments in the References header.

And on the subject of 3.1.5, I think I have already mentioned that the
wording about "harming propagation" is not strictly correct as it is
written (was that one of the things Ken agreed to fix?).

>1080 USEFOR 3.2.14 - MIME parameters for Injection-Info and Archive header 
>field need more text/updated syntax

>  was "text proposed" on Nov 28, text incorporated in -06.
>  If this text is not OK, speak up - otherwise, I'll just close this.

Yes.

BTW, some way back I asked for an extra parameter
      authentication = whatever
to indicate how the user had identified himself to the injecting agent.
Discussion went as far as concluding that it was "authentication" as
opposed to "authorization" that was required, but it never got alloated a
ticket number. But if it it too late to raise it again now, then no
matter.

All other actions mentioned seem OK to me.

To: [email-masked]
Xref: clerew local.usefor:22848
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Date: Wed, 4 Jan 2006 10:44:38 GMT
Lines: 44
Sender: [email-masked]

In <EC7149AB099F278B9A9F4141@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>In addition:
>What's out there is a mess of

>...!server!diagnostic!....

>...!server!server!....

>...!server!POSTED!.... (possibly diagnostic before the final-entry, not=20
>sure)

>I'm not sure if we ever have

>...!server!diagnostic!diagnostic!....

>Should the syntax bless, curse, grudgingly permit, or ignore these=20
>practices?

So long as you can tell which is a server and which is a diagnostic and
which is a keyword, it does not really matter whether these oddities are
allowed or not. In the interests of tidiness, one would allow each
"server!" to be followed by zero or one "diagnostic!" (maybe with
keyword), and for sure USEPRO will tell you to construct just that. But
whether you write all that into the syntax is a secondary matter (my last
proposed syntax did do it).

But for sure, we don't expect software which reads this header to waste
time parsing it in that detail - all it needs to do it to pick out the
delimiters, pick out the stuff between them (it might or might not worry
about illegal characters in the 'stuff'), and then see if the 'stuff' is
recorded in its 'sys' file as a place that it need not relay the article
to.

To: [email-masked]
Xref: clerew local.usefor:22851
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ABNF
Date: Wed, 4 Jan 2006 11:39:48 GMT
Lines: 42
Sender: [email-masked]

>> I would be happy with
>>    posting-host = news.foo.example[123.456.123.456]
>> (not lack of whitespace after "example" - you would need to
>> quote it then). But then you also have to decide whether you
>> want
> 
>>    posting-host = [123.456.123.456]
>> as well as the present
>>    posting-host = 123.456.123.456

>The former as in 2821 (IPs always in square brackets).
>Quoted, because square brackets are also <tspecials> :-(

OK. If you have to quote it anyway because of the [...], then you might as
well leave it with the ":" we have at present.

>It's a new <parameter> in a new header field, if the present
>NNTP-Hosting-Host header field is a bit different that's no
>real issue.

>The real question is, is the colon confusing in conjunction
>with IPv6 ?  If *not* let's just use the 4234-beautified ABNF
>as proposed in "ISSUE: host-value", otherwise the same as is.

It is not ambiguous since an Ipv6Address cannot begin with a ":". I tried
to construct an ambiguity supposing that ICANN might have invented a TLD
"beef", but I could not do it even then. So let it be.

To: [email-masked]
Xref: clerew local.usefor:22852
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Control: cancel ABNF bug
Date: Wed, 4 Jan 2006 11:53:15 GMT
Lines: 35
Sender: [email-masked]

>> No, We agreed not to allow <comment>s. But the simple folding rule of
>> RFC 2822 ought really to be accepted everywhere eventually, even if not
>> on day one. It is just ugly to have such special exceptions lying around
>> for ever.

>I think this objection is silly, and strenuously object to FHS in the
>Control header. ...

RFC 2822, and ALL its predecessors, established the simple convention that
you could fold wherever whitespace could occur. It is my belief that RFC
1036 bought into that same convention (hence why I have always argued that
Path was foldable under RFC 1036, as opposed to Newsgroups which clearly
was not).

Yes, implementations of 1036 have been lax on many points, and where a
great many implementations have implemented the same laxity (as with the
SP after ':') then we had to go along with it.

But that does not apply here. The sky will not fall in if we correct this
anomaly (or, rather, fail to introduce it).

To: [email-masked]
Xref: clerew local.usefor:22850
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Old ticket status - USEFOR - Jan 3, 2006
Date: Wed, 4 Jan 2006 11:28:41 GMT
Lines: 27
Sender: [email-masked]

>> 1032 USEFOR general: Document changes from RFC 1036

>Add:

>| The obsolete convention to interpret subjects starting
>| with the word "cmsg" as control message was removed.

No, you cannot say that in Appendix B of USEFOR, because the change is not
mentioned in USEFOR anywhere.

It IS mentioned in USEPRO, which has its own list of differences from
1036, and I will see that it gets mentioned in that list.

To: [email-masked]
Xref: clerew local.usefor:22853
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: IANA considerations
Date: Wed, 4 Jan 2006 11:59:25 GMT
Lines: 26
Sender: [email-masked]

>> All they do is refer you to RFC 1036

>If that's about message/news it might be old, I've
>never seen a complete message/news template.  Bye

I think Google could find it for you. And I have certainly found it on the
IANA website in the past.

Both message/news and application/news-transmission were registered
privately by Henry Spencer in the days when IANA was less fussy about
where things came from. Neither was ever in any RFC

To: [email-masked]
Xref: clerew local.usefor:22832
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: IANA considerations
Date: Tue, 3 Jan 2006 17:21:25 GMT
Lines: 57
Sender: [email-masked]

In <[email-masked]> Ken Murchison <[email-masked]> writes:

>Do we really have to use the same format as given by the template, or 
>just provide the same information as specified by the template?  If the 
>latter, then I think a table such as the one I stole from Charles is fine.

I think the first thing to understand is that the templates do not need to
be as verbose as the ones found in RFC 4229 and RFC 4021 (one should bear
in mind that those documents were separate from the RFCs where the headers
were actually desccribed, which will not be the case with USEFOR). So
Frank's worries concerning the umpteen pages it might eat up in our
document are nothing like so bad as he supposes.

The table that has been proposed contains all the information that we are
required to supply (though we have the option of saying more in particular
cases if we feel the need). Even writing them out in full in the format
prescribed by RFC 3864 would not be that bad, though it would look pretty
boring (as will always be the case when you have an RFC introducing a lot
of templates all at once).

Putting them in a table is much more user friendly. Essentially, it is a
rule whereby you could construct each individual template if you really
wanted to, but what is the point?

So my view is that we should try to publish them in table form, and if the
IETF tell us to rewrite them, or instruct the RFC-Editor to expand them in
full, then we should worry about it if/when that happens.

>> The issue of documenting obsoleted headers is problematic. We can (in 
>> theory) argue that the sentences referring to them as "obsoleted" in 
>> usefor-usefor is enough definition for an obsoleted header. It would be 
>> more proper to have son-of-1036 as the reference - but that introduces a 
>> dependency, which is bad.

>I don't believe that the sentences that refer to the obsoleted headers 
>provide enough of a definition, especially without an ABNF grammar 
>somewhere.  Referencing s-o-1036 would be good if it has a formal 
>definition of the headers.  The dependency is only bad if s-o-1036 
>doesn't get into the RFC editor queue before USEFOR.

I think the main thing to ensure is that it doesn't have to be a normative
reference (and I see no reason why it should be - from a standards point
of view, all we are saying is "Don't use this header", which does not
actually _require_ any definition of what that header might have once
meant).

Date: Tue, 03 Jan 2006 22:16:29 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Sender: [email-masked]

--==========2B31044667E156980242==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

--On 3. januar 2006 20:00 +0100 Frank Ellermann <[email-masked]>=20
wrote:

> If we could agree on the requirements for #1047 the solution
> should be clear, and we're almost ready.  I think (not sure)
> the key is to consider the four possible constructs...
>
> ...!old-path
> ...!!old-path
> ...!...!MISMATCH!old-path
> ...!POSTED!old-path
>
> ...as units.  With "diagnostics" only appearing as ... before
> a MISMATCH.  Or add a fifth construct...
>
> ...!...!SEEN!old-path
>
> ...also with a second slot for "diagnostics" (IIRC Russ' idea).

In addition:
What's out there is a mess of

...!server!diagnostic!....

...!server!server!....

...!server!POSTED!.... (possibly diagnostic before the final-entry, not=20
sure)

I'm not sure if we ever have

...!server!diagnostic!diagnostic!....

Should the syntax bless, curse, grudgingly permit, or ignore these=20
practices?

                      Harald

--==========2B31044667E156980242==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========2B31044667E156980242==========--

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: IANA considerations
Date:  Tue, 03 Jan 2006 21:25:20 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 13
Sender: [email-masked]

Charles Lindsey wrote:

> in the format specified by RFC 2048

That's now AFAIK obsoleted by 4288 + 4289.

> All they do is refer you to RFC 1036

If that's about message/news it might be old, I've
never seen a complete message/news template.  Bye

http://web.archive.org/web/*/http://www.iana.org/assignments/media-types/message/

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ABNF
Date:  Tue, 03 Jan 2006 20:44:39 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 39
Sender: [email-masked]

Charles Lindsey wrote:

> I would be happy with
>    posting-host = news.foo.example[123.456.123.456]
> (not lack of whitespace after "example" - you would need to
> quote it then). But then you also have to decide whether you
> want
 
>    posting-host = [123.456.123.456]
> as well as the present
>    posting-host = 123.456.123.456

The former as in 2821 (IPs always in square brackets).
Quoted, because square brackets are also <tspecials> :-(

It's a new <parameter> in a new header field, if the present
NNTP-Hosting-Host header field is a bit different that's no
real issue.

The real question is, is the colon confusing in conjunction
with IPv6 ?  If *not* let's just use the 4234-beautified ABNF
as proposed in "ISSUE: host-value", otherwise the same as is.

BTW, while you discuss control FWS vs. 1*WSP with Russ, the -06
syntax is actually:

    verb *( [FWS} argument )

If we'd stick to FWS that should be at least:

    verb *( FWS argument )

Or  verb *( 1*WSP argument )  without folding.  Your proposal
to add a note "detailed control commands are specified in
[USEPRO]" (or something in that direction) is a good idea.  A
real example plus pointer to USEPRO could have the same effect.

                          Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  #1047 (was: Old ticket status - USEFOR - Jan 3, 2006)
Date:  Tue, 03 Jan 2006 20:00:35 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 60
Sender: [email-masked]

[splitting my attempted all-in-one-reply, ideas about #1047]

> 1047 USEFOR 3.1.6: Path field delimiters and components
[...]
> Still open whether USEFOR should specify syntax that allows
> one to tell the difference between a path-identity and a
> diagnostic.

As it is the syntax is ambiguous.  We've seen at least two
more complete solutions, one by Charles, another by me, but
it's somewhat hard to get the ABNF right while it's still
unclear what we really want.

Points I recall:  Compatible with existing practice, that's
AFAIK ...!...!MISMATCH!... or ...!POSTED!... or ...!... and
we add ...!!... (as shorthand for ...!SEEN!... but without
mentioning / allowing this long form).

Bareword (funny underscores) vs. FQDN with a <toplabel>.
IPv4 or IPv6 only in conjunction with ...!ip!MISMATCH!...

Minimize potential FWS, it can come after ...!! or after
...!...!MISMATCH! or after ...!POSTED! or after ...! but
not elsewhere.

No "multi-entries", they are too confusing as part of the
syntax.

Could we somehow agree that that's precisely what we want ?
Then Charles could cook up some corresponding syntax, and
I debug it, or vice versa,

[...]

If we could agree on the requirements for #1047 the solution
should be clear, and we're almost ready.  I think (not sure)
the key is to consider the four possible constructs...

...!old-path
...!!old-path
...!...!MISMATCH!old-path
...!POSTED!old-path

...as units.  With "diagnostics" only appearing as ... before
a MISMATCH.  Or add a fifth construct...

...!...!SEEN!old-path

...also with a second slot for "diagnostics" (IIRC Russ' idea).

The <path-identity> always in the first slot, no IP.  The only
[FWS] either before !old-path, or behind "!" before old-path.

It MUST be possible to get this right, assuming that it really
is what we want (the main problem, what should the Path be ?).

                          Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Old ticket status - USEFOR - Jan 3, 2006
Date:  Tue, 03 Jan 2006 19:53:01 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 42
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> 1032 USEFOR general: Document changes from RFC 1036

>    Text in Appendix B. Please speak up if this is not
>    sufficient, or needs modification - otherwise, I'll
>    just close this.

Add:

| The obsolete convention to interpret subjects starting
| with the word "cmsg" as control message was removed.

> 1078 USEFOR 3.1.5:
[...]
> do not say anything, leave this for USEAGE.

+1

> 1080 USEFOR 3.2.14
[...]
> just close this.

+1

> 1084 USEFOR 2.1, 3
[...]
> "proposed no change" on Nov 28

Without some fresh opposition supporting id-unique @ id-domain
etc. (instead of id-left @ id-right) there's nothing to do.
If somebody else doesn't like id-right I post my variant again.

> 1132 USEFOR 3.1.6:
[...]
> I regard the specific issue as closed.

+1

#1047 moved to a separate article.

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Control: cancel ABNF bug
Organization: The Eyrie
Date: Tue, 03 Jan 2006 10:02:02 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> No, We agreed not to allow <comment>s. But the simple folding rule of
> RFC 2822 ought really to be accepted everywhere eventually, even if not
> on day one. It is just ugly to have such special exceptions lying around
> for ever.

I think this objection is silly, and strenuously object to FHS in the
Control header.  It is gratuitous backwards incompatibility for no purpose
whatsoever.  Introducing backwards incompatibility that WILL NOT WORK in
practice just for the sake of textual consistency makes the protocol
*more*, not less, complex for implementors.  Now they can't trust the text
or have to worry about exception clauses and the uncertainty over whether
a historical note is actually likely to affect them or is really
historical.

Can we *please* try to spend less time unringing old bells and more time
documenting the protocol so that we can release a specification for what
Usenet actually is?

> You will rmemember that when we discussed this in connection with the
> Path header, it was pointed out that the worst that could happen in
> implementations that neglected to notice the folding was that a few
> articles might be relayed to one's peers unnecessarily.

I still don't like it in the Path header, but in the Path header nothing
really cares that much; it's a small optimization.  If you fold your
control header, you may as well just post the control message to
/dev/null.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  ISSUE: 3.2.14 : <sender>
Date:  Tue, 03 Jan 2006 18:23:20 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 27
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
 
> if anyone thinks that there are more issues that should be
> tracked

Draft -06 says:

- If a news server can verify the sender of an article, it
- SHOULD use this <parameter> in favor of the "posting-
- account" <parameter>.

Charles reported that this is an old consensus, but I think
it's wrong.  Technically news servers should be free to use
either method.  For privacy reasons they might prefer the
<posting-account> approach.

+ Typically a news server uses either a "posting-account"
+ or a verified "sender" parameter, inserting the <mailbox>
+ has implications for the privacy of users.

BTW, it's possible that some kind of old consensus included
me, in that case I've changed my mind:  "Killfiling" is an
important feature for those who have it, but enforcing that
possibility with a SHOULD goes to far.

                          Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  ISSUE: host-value
Date:  Tue, 03 Jan 2006 17:58:19 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> please send an email to the list with ONE issue and the word
> "ISSUE:" in the subject line.

A clearer ABNF for <host-value> as recommended in 4234...

-   host-value      =  dot-atom-text / [ dot-atom-text ":" ]
-                      ( IPv4address / IPv6address ) ;  see [RFC 3986]

+ host-value = dot-atom-text / IPv4address / IPv6address /
+              ( dot-atom-text ":" ( IPv4address / IPv6address ))

...or a "better" (?) ABNF in the style of 2821/3986 IP-literal:

+ IP-literal  = "[" ( IPv4address / IPv6address ) "]"
+
+ host-value  = dot-atom-text / IP-literal /
+               ( dot-atom-text IP-literal )

To: [email-masked]
Xref: clerew local.usefor:22831
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: IANA considerations
Date: Tue, 3 Jan 2006 17:05:45 GMT
Lines: 54
Sender: [email-masked]

In <A82C113B94EC10F039562273@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>Note that the registry points to the documents that contain the complete=20
>registration templates. I don't think we can break the rules by not=20
>submitting the templates.

Aaaarrrrrgh! I had not noticed that, and it is absolutely not what RFC
3864 was intended to bring about, as I understood it at the time, and
absolutely not the way things used to be done.

Hmmmmm! Last time I looked in anger at an IANA Registry, it contained what
was in the template. For example, I once looked at the template for
news/transmission (which was in the format specified by RFC 2048). From
the IANA site, following various links, I found the template (which AIAIR
was still somewhere under www.iana.org), and in due course used it to
construct the revised template which you will now find in USEPRO. Likewise
the template for message/news, which we propose to declare obsolete in
USEPRO.

I have just looked for those two templates again, and they are not to be
found. All they do is refer you to RFC 1036 (and they are not there as you
well know) and to Henry Spencer's no-longer-working email address.

And yet some templates are there (for example, I can find
application/msword). Indeed, most of the templates not defined in RFCs
seem to be there.

Google eventually found me a copy of the current
application/news-transmission template, but it wasn't on any IANA site, or
any site controled by Henry Spencer. Indeed, it is totally unclear who
owns the site I found.

>(The question of how IANA maintains registries is a completely different=20
>subject.)

Indeed so. What is the point of having an IANA registry if it manages to
lose specifications entrusted to it?

I think we need to understand what the role of IANA really is before
proceeding any futher with this.

To: [email-masked]
Xref: clerew local.usefor:22829
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Control: cancel ABNF bug
Date: Tue, 3 Jan 2006 15:25:00 GMT
Lines: 53
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Hi, there's apparently a subtle bug in the -06 ABNF.
>We want (USEPRO 6.3):

>      control-command     =/ Cancel-command
>      Cancel-command      = "cancel" Cancel-arguments
>      Cancel-arguments    = FWS msg-id [FWS]

>That's your  msg-id = "<" id-left "@" id-right ">"
>(same as my  msg-id = "<" id-unique "@" id-domain ">" ).

>But USEFOR and 2045 (with Ned's pending erratum) say:

>      control-command =  verb *( [FWS] argument )
>      verb            =  token
>      argument        =  value

Aargghh! Well spotted. My fault I think, because I persuaded Ken to put
that in, just so that people could parse a <control-command> into its
components before handing over to the code for the specific command, if
they preferred to do it that way. OTOH, it _should_ also make some nention
that further syntax rules will be provided in USEPRO, or else that USEPOR
will provide the requisite verbs and arguments, in the same way as we did
for Injection-Info. Whatever way we write it here in USEFOR, I can make
USEPRO fit in (though I think my preference would be to leave the USEPRO
syntax as-is and simnple write in here that the commands to be added will
be of the form of "verb *( [FWS] argument )".

For <verb>, <token> is fine, or course, but we need some other definition
of <argument>.

>               Control: cancel "<whatever@example>"

>Substituting <word> for <value> has the same problem.
>1*utext also can't solve it, we don't want NO-WS-CTL.
>Taking 1*VCHAR should do the trick, for section 1.4:

>      VCHAR           =  <see RFC 4234 appendix B.1>

Yes, I would be happy with that.

To: [email-masked]
Xref: clerew local.usefor:22828
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ABNF
Date: Tue, 3 Jan 2006 15:04:05 GMT
Lines: 49
Sender: [email-masked]

>> if you look at draft-ietf-usefor-article-13, you will see
>> that is exactly what was done (non-normatively, of course),

>Yes, I liked that, but a later consensus (?) was to drop it.

I don't recall that it was ever discussed. Anyway it is there if we want
to continue to use it (or something like it). There should certainly be a
collected syntax of some sort, but it needn't go to that extent if people
prefer it shorter.

> <host-value> 
>> The syntax using the ":" was taken from the presently used
>> NNTP-Posting-Host header, which <host-value> is intended to
>> replace.

>Okay.  But note that colon is a 2045 <tspecial>, so there will
>a major difference for "FQDN or IPv4 only" vs. "FQDN:IPv4", in
>the latter case the <host-value> MUST be in a <quoted-string>.

>With square brackets for all IP literals (as in 2821) or at the
>very minimum for IPv6 and beyond (as in STD 66) we could avoid
>all potential IPv6 trouble:

OK, that is a good point. I would be happy with

   posting-host = news.foo.example[123.456.123.456]

(not lack of whitespace after "example" - you would need to quote it
then). But then you also have to decide whether you want

   posting-host = [123.456.123.456]
as well as the present
   posting-host = 123.456.123.456

To: [email-masked]
Xref: clerew local.usefor:22830
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Control: cancel ABNF bug
Date: Tue, 3 Jan 2006 15:37:31 GMT
Lines: 42
Sender: [email-masked]

>> Control: newgroup 
>>  new.group.example
>>  moderated

>I would be surprised if that would work anywhere.  It certainly wouldn't
>work with any version of INN.  Sorry, I missed that -- I thought we'd
>already discussed this before and agreed on WSP.

No, We agreed not to allow <comment>s. But the simple folding rule of RFC
2822 ought really to be accepted everywhere eventually, even if not on day
one. It is just ugly to have such special exceptions lying around for ever.

Now any hierarchy admin who issues the control message above with the
expectation that the group will be created correctly everywhare deserves
all that he gets, but I doubt anyone is likely to try.

Therefore, although to claim 100% compliance implementations will need to
accept it and process it correctly, I fully accept that fixing that
particular bit of INN is likely to be just about at the bottom of Russ's
list of priorities, and likely to remain there for quite a long time. So
be it, but people writing new code had better get it right.

You will rmemember that when we discussed this in connection with the Path
header, it was pointed out that the worst that could happen in
implementations that neglected to notice the folding was that a few
articles might be relayed to one's peers unnecessarily.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1157 Re: Control: cancel ABNF bug
Date:  Tue, 03 Jan 2006 17:46:47 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 15
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> I've created ticket #1157 for this problem.

Thanks.  The quoted fixed ABNF did not yet addreess the
[FWS] issue as discussed with Russ, for that it should be:

        control-command =  verb *( 1*WSP argument )
        verb            =  token
        argument        =  1*(%x21-7E)

So that's s/ [FWS] / 1*WSP / plus s/ value / 1*(%x21-7E) /

                     Bye, Frank

Date: Tue, 03 Jan 2006 15:14:01 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Not tracking minor issues
Sender: [email-masked]

I've created 3 tickets (1155, 1156, 1157) with the "major" issues raised 
against -06.

There were a number of minor issues raised, to which Ken said "OK" - I'm 
planning to not track those. I'm not completely sure they were all minor.

So - if anyone thinks that there are more issues that should be tracked - 
please send an email to the list with ONE issue and the word "ISSUE:" in 
the subject line.

Yours for a document-finishing 2006....

Date: Tue, 03 Jan 2006 15:04:37 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: #1157 Re: Control: cancel ABNF bug
Sender: [email-masked]

I've created ticket #1157 for this problem.

--On s�ndag, januar 01, 2006 01:44:59 +0100 Frank Ellermann 
<[email-masked]> wrote:

>
> Hi, there's apparently a subtle bug in the -06 ABNF.
> We want (USEPRO 6.3):
>
>       control-command     =/ Cancel-command
>       Cancel-command      = "cancel" Cancel-arguments
>       Cancel-arguments    = FWS msg-id [FWS]
>
> That's your  msg-id = "<" id-left "@" id-right ">"
> (same as my  msg-id = "<" id-unique "@" id-domain ">" ).
>
> But USEFOR and 2045 (with Ned's pending erratum) say:
>
>       control-command =  verb *( [FWS] argument )
>       verb            =  token
>       argument        =  value
>
>       value := token / quoted-string
>       token := 1*<any (US-ASCII) CHAR except SPACE, CTLs,
>                   or tspecials>
>       tspecials :=  "(" / ")" / "<" / ">" / "@" /
>                     "," / ";" / ":" / "\" / <"> /
>                     "/" / "[" / "]" / "?" / "="
>                     ; Must be in quoted-string,
>                     ; to use within parameter values
>
> In other words Control: cancel <whatever@example> is not
> allowed, and we certainly don't want to quote a <msg-id>:
>
>                Control: cancel "<whatever@example>"
>
> Substituting <word> for <value> has the same problem.
> 1*utext also can't solve it, we don't want NO-WS-CTL.
> Taking 1*VCHAR should do the trick, for section 1.4:
>
>       VCHAR           =  <see RFC 4234 appendix B.1>
>
> For section 3.2.5 (changing only its <argument>):
>
>       control-command =  verb *( [FWS] argument )
>       verb            =  token
>       argument        =  1*VCHAR
>
> Or directly  argument = 1*(%x21-7E)  without the VCHAR.
>
>                         Bye, Frank
>
> P.S.:  why on earth do they need more than 10 months to
> publish an erratum submitted by one of the 2045 authors ?
>
>
>

Date: Tue, 03 Jan 2006 14:14:53 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Ken Murchison <[email-masked]>
Cc: [email-masked]
Subject: #1156 Re: IANA considerations
Sender: [email-masked]

--On mandag, januar 02, 2006 12:17:11 -0500 Ken Murchison 
<[email-masked]> wrote:

>> I don't see much leeway in RFC 3864 section 3 and 4:
>
> So, what is the decision of the chair regarding the format and location
> of the IANA header registry entries?
>

Given that the editor is willing to do it, the chair rules that the 
registration forms for the headers defined here will go into the USEFOR-07 
draft. I'll leave it to the editor's license whether it's inside the IANA 
considerations section or a new appendix pointed to by the IANA 
considerations section.

I've created #1156 to keep track of this.

                          Harald

Date: Tue, 03 Jan 2006 14:11:22 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: #1155 USEFOR 1.4 Formalize ABNF imports (Re: artwork)
Sender: [email-masked]

--On s�ndag, januar 01, 2006 00:21:17 +0100 Frank Ellermann 
<[email-masked]> wrote:

> Hi, nobody said that a collection of ABNF-prose is a bad idea.
>
> So I guess that we want this for the point "Collected ABNF?"
> in the remaining five "Issues to be Addressed" as stated on
> http://tools.ietf.org/html/draft-ietf-usefor-usefor-06#page-5
>

I've recorded this as #1155: USEFOR 1.4: Formalize ABNF Imports.

Seems good to me!

                     Harald

Date: Tue, 03 Jan 2006 13:24:37 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Old ticket status - USEFOR - Jan 3, 2006
Sender: [email-masked]

Happy new year, folks!

At the moment, I think we have 6 tickets left from "the old stuff":

1032 USEFOR general: Document changes from RFC 1036

   Text in Appendix B. Please speak up if this is not sufficient, or needs
   modification - otherwise, I'll just close this.

1047 USEFOR 3.1.6: Path field delimiters and components

   Proposed text incorporated in -06.

   Still open whether USEFOR should specify syntax that allows one to
   tell the difference between a path-identity and a diagnostic.

1078 USEFOR 3.1.5: Need to describe meaning of Newsgroups header field in 
email

   was "no consensus" Nov 28. I think "proposed no change" would be a
   reasonable current status, based on the 2 comments at that time.
   That is - do not say anything, leave this for USEAGE.

1080 USEFOR 3.2.14 - MIME parameters for Injection-Info and Archive header 
field need more text/updated syntax

  was "text proposed" on Nov 28, text incorporated in -06.
  If this text is not OK, speak up - otherwise, I'll just close this.

1084 USEFOR 2.1, 3 Names for ABNF productions redefining 822 constructs

  was "proposed no change" on Nov 28

1132 USEFOR 3.1.6: Outlaw IP address in path-identity?

  was "text accepted" on Nov 28 - this is closely enough linked to 1047
  that they should be considered together, but I regard the specific
  issue as closed.

I'll open some new tickets with the issues subsequently raised on the list.

                     Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Control: cancel ABNF bug
Date:  Tue, 03 Jan 2006 08:50:30 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

Russ Allbery wrote:

>> Simply all mentioned TLHs in the body ?
> Yup.

Okay, then the USEPRO checkgroups ABNF is fine, it could
even stick to FWS as separator if 1*WSP isn't good enough.

But if we intend to allow FWS _anywhere_ for control we
also have to keep it in the general USEFOR control ABNF.

In that case we could enforce 1*WSP in USEPRO for all old
control header fields.

Rationale:  1*WSP is a proper subset of FWS, but not the
other way around.  I'm not completely sure what's better:

1:  FWS in USEFOR, 1*WSP in USEPRO "only where necessary",
    in other words everywhere excl. the new checkgroups
2:  1*WSP in both USEFOR and USEPRO, if somebody manages
    to create a checkgroups with more than 998 char.s in
    the checkgroups header field this can't be serious.

                           Bye, Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Control: cancel ABNF bug
Organization: The Eyrie
Date: Mon, 02 Jan 2006 21:59:00 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> If no <chkscope> at all is state of the art, how do servers know what
> the checkgroups is about ?  Simply all mentioned TLHs in the body ?

Yup.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Control: cancel ABNF bug
Date:  Tue, 03 Jan 2006 06:16:37 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 19
Sender: [email-masked]

Russ Allbery wrote:
 
>> The <chkscope> can't be optional, or can it ?

> I dunno, all that chkscope and chksernr stuff is new
> innovation that no one's ever implemented, so I suppose
> it can be just about anything we want.  :)  I think the
> assumption was that the leading # in chksern would
> disambiguate even if chkscope wasn't provided.

Okay, I didn't know that a <chkscope> is completely new,
I only knew that checkgroups de !de.alt would be new.

If no <chkscope> at all is state of the art, how do
servers know what the checkgroups is about ?  Simply
all mentioned TLHs in the body ?

                      Bye, Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Control: cancel ABNF bug
Organization: The Eyrie
Date: Mon, 02 Jan 2006 11:55:52 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> The control: checkgroups might have an additional problem:

>       Checkgroup-command  = "checkgroups" Checkgroup-arguments
>       Checkgroup-arguments= [ chkscope ] [ chksernr ]
>       chkscope            = 1*( FWS ["!"] newsgroup-name )
>       chksernr            = FWS "#" 1*DIGIT

> The <chkscope> can't be optional, or can it ?  Bye, Frank

I dunno, all that chkscope and chksernr stuff is new innovation that no
one's ever implemented, so I suppose it can be just about anything we
want.  :)  I think the assumption was that the leading # in chksernr would
disambiguate even if chkscope wasn't provided.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Control: cancel ABNF bug
Date:  Mon, 02 Jan 2006 20:44:57 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 28
Sender: [email-masked]

Russ Allbery wrote:

> I missed that -- I thought we'd already discussed this before
> and agreed on WSP.

We discussed "[FWS] => *WSP almost everywhere", but IIRC Bruce
said it would remove 2822 obs-FWS, and then we visited any
rathole in sight, I forgot how that general debate ended...

Here it's actually a case of FWS => 1*WSP, not [FWS] => *WSP,
the [FWS] as separator is of course also wrong.  Fixed ABNF:

|       control-command =  verb *( 1*WSP argument )
|       verb            =  token
|       argument        =  1*(%x21-7E)

Maybe there are more obscure [FWS].  In USEPRO it should be
FWS => 1*WSP for all control header field arguments.

The control: checkgroups might have an additional problem:

      Checkgroup-command  = "checkgroups" Checkgroup-arguments
      Checkgroup-arguments= [ chkscope ] [ chksernr ]
      chkscope            = 1*( FWS ["!"] newsgroup-name )
      chksernr            = FWS "#" 1*DIGIT

The <chkscope> can't be optional, or can it ?  Bye, Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Control: cancel ABNF bug
Organization: The Eyrie
Date: Mon, 02 Jan 2006 10:51:04 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> Yes, not FWS between arguments, WSP within an argument.  BTW,
> is this [FWS] as you want it, or should it be 1*WSP ?  Would...

> Control: newgroup 
>  new.group.example
>  moderated

> ...work anywhere ?  Everywhere ?  Bye, Frank

I would be surprised if that would work anywhere.  It certainly wouldn't
work with any version of INN.  Sorry, I missed that -- I thought we'd
already discussed this before and agreed on WSP.

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Control: cancel ABNF bug
Organization: The Eyrie
Date: Mon, 02 Jan 2006 10:41:54 -0800
Sender: [email-masked]

>>>       control-command =  verb *( [FWS] argument )
>>>       verb            =  token
> [...]
>>>              argument = 1*(%x21-7E)
> {...]

>> Is there no case where a control: message needs a space?

> That would be ambiguous with the [FWS].  For new/rm/mvgroup
> it's <newsgroup-name> (like FQDN plus underscore, no space).

Oh, I understand now.  Yes, no individual argument to the Control header
may ever contain a space since software does a simple split on spaces to
parse it.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Control: cancel ABNF bug
Date:  Mon, 02 Jan 2006 19:38:39 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 13
Sender: [email-masked]

Russ Allbery wrote:
 
> Or am I misunderstanding?

Yes, not FWS between arguments, WSP within an argument.  BTW,
is this [FWS] as you want it, or should it be 1*WSP ?  Would...

Control: newgroup 
 new.group.example
 moderated

...work anywhere ?  Everywhere ?  Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Control: cancel ABNF bug
Date:  Mon, 02 Jan 2006 19:16:15 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 56
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

>>       control-command =  verb *( [FWS] argument )
>>       verb            =  token
[...]
>>              argument = 1*(%x21-7E)
{...]

> Is there no case where a control: message needs a space?

That would be ambiguous with the [FWS].  For new/rm/mvgroup
it's <newsgroup-name> (like FQDN plus underscore, no space).

Dito for checkgroups, adding "!" for excluded scopes, and a
serial number "#" 1*DIGIT

For cancel it's <msg-id> (no space even for 2822 NO-WS-CTL).

For ihave / sendme it's <relayer-name> (= <path-identity> ).
Whatever we do with barewords and #1047, we shouldn't allow
a space within a <path-identity> :-)

The obsolete 1036 commands version and sendsys have no
argument.  The "archaeological" whogets and senduuname are
unspecified, apparently no argument for senduuname and a
<newsgrup-name> for whogets.  No spaces within arguments.

>> P.S.:  why on earth do they need more than 10 months to
>> publish an erratum submitted by one of the 2045 authors ?

> I sure don't know - when it was my problem, every time they
> asked, they said that the IETF had told them publishing RFCs
> was more important.....

For the obvious 2045 "bug" that's true.  OTOH it's so obvious
that any delay is more work than just publishing the erratum.

Generally I'm not so sure, e.g. I'm unable to reproduce most
DIGEST MD5 examples published in various RFCs starting with
2069 (no problem for 2617 and 2831), so I'd say "stay away
from this stuff and use CRAM-MD5 or OTP" to anybody who asks.
For 2069 it's most probably only a stupid error on my side.

For the strange claim that UTF-8 needs %x00-FF in a fresh RFC
I didn't try anymore, and just posted it on the rfc-interest
list, that's a place where GoogleBot might find it.

More examples where I didn't bother anymore skipped, 2368bis
is one of the potential victims - won't fly without erratum
for 3986 or reverse engineering of a proper <uric>.

Okay, 2368bis has also several other problems, not only this.
And at least John's 3696 erratum made it.

                         Bye, Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Control: cancel ABNF bug
Organization: The Eyrie
Date: Mon, 02 Jan 2006 10:07:04 -0800
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> Personal opinion: I like the last version - simple and self-contained.

> Q: Is there no case where a control: message needs a space?

Like:

    Control: newgroup example.moderated moderated

?  Or am I misunderstanding?

Date: Mon, 02 Jan 2006 12:17:11 -0500
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: Harald Tveit Alvestrand <[email-masked]>
CC: [email-masked]
Subject: Re: IANA considerations
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
> 
> 
> --On mandag, januar 02, 2006 09:46:58 -0500 Ken Murchison 
> <[email-masked]> wrote:
> 
>>> Grumble. The HTTP and Mail documents both give full templates per
>>> header, and as you say, this makes for a very long document (easy to
>>> produce, just tedious). I like the idea of not making -usefor depend on
>>> this.
>>
>> Do we really have to use the same format as given by the template, or
>> just provide the same information as specified by the template?
> 
> I don't see much leeway in RFC 3864 section 3 and 4:

So, what is the decision of the chair regarding the format and location 
of the IANA header registry entries?

-- 
Kenneth Murchison
Systems Programmer
Carnegie Mellon University

Date: Mon, 02 Jan 2006 17:25:15 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: Control: cancel ABNF bug
Sender: [email-masked]

--On s�ndag, januar 01, 2006 01:44:59 +0100 Frank Ellermann 
<[email-masked]> wrote:

> For section 3.2.5 (changing only its <argument>):
>
>       control-command =  verb *( [FWS] argument )
>       verb            =  token
>       argument        =  1*VCHAR
>
> Or directly  argument = 1*(%x21-7E)  without the VCHAR.

Personal opinion: I like the last version - simple and self-contained.

Q: Is there no case where a control: message needs a space?

>
>                         Bye, Frank
>
> P.S.:  why on earth do they need more than 10 months to
> publish an erratum submitted by one of the 2045 authors ?

I sure don't know - when it was my problem, every time they asked, they 
said that the IETF had told them publishing RFCs was more important.....

Date: Mon, 02 Jan 2006 17:18:11 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Ken Murchison <[email-masked]>, [email-masked]
Subject: Re: IANA considerations
Sender: [email-masked]

--On mandag, januar 02, 2006 09:46:58 -0500 Ken Murchison 
<[email-masked]> wrote:

>> Grumble. The HTTP and Mail documents both give full templates per
>> header, and as you say, this makes for a very long document (easy to
>> produce, just tedious). I like the idea of not making -usefor depend on
>> this.
>
> Do we really have to use the same format as given by the template, or
> just provide the same information as specified by the template?

I don't see much leeway in RFC 3864 section 3 and 4:

3.  Registry Usage Requirements

   RFCs defining new header fields for Internet mail, HTTP, or MIME MUST
   include appropriate header registration template(s) (as given in
   Section 4.2) for all headers defined in the document in their IANA
   considerations section.  Use of the header regi stry MAY be mandated
   by other protocol specifications, however, in the absence of such a
   mandate use of the registry is not required.

4.  Registration Procedure

   The procedure for registering a message header field is:

   1.  Construct a header field specification

   2.  Prepare a registration template

   3.  Submit the registration template

Perusing the Mail header registry document, RFC 4021, clarifies that the 
words PERMANENT MESSAGE HEADER FIELD REGISTRATION TEMPLATE are not part of 
the template, luckily :-)

                  Harald

Date: Mon, 02 Jan 2006 09:46:58 -0500
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: [email-masked]
Subject: Re: IANA considerations
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
> 
> 
> --On 22. desember 2005 08:17 -0500 Ken Murchison <[email-masked]> 
> wrote:
> 
>>>    NNTP-Posting-Date netnews deprecated IETF
>>>    Also-Control netnews obsoleted IETF
>>>    See-Also netnews obsoleted IETF
>>>    Article-Names netnews obsoleted IETF
>>>    Article-Updates netnews obsoleted IETF
>>>
>>> A proper reference for the last 4 would be Son-of-1036, which we propose
>>> to refer to in any case. The 1st two have no published definition that I
>>> know of apart from the source code of INN. However, they are both much
>>> used, and we have explicitly deprecated their continued use in USEFOR. I
>>> think it would still be proper to describe IETF as the "Change
>>> Controller" (but not as Author, obviously).
>>
>> If s-o-1036 actually defines a header, then I think we can probably
>> register it.  If there is no definition anywhere that we can reference,
>> then how can we register it?  I certainly don't want to add ABNF to our
>> doc for headers that are dead.
>>
>> Harald, any thoughts?
> 
> Grumble. The HTTP and Mail documents both give full templates per 
> header, and as you say, this makes for a very long document (easy to 
> produce, just tedious). I like the idea of not making -usefor depend on 
> this.

Do we really have to use the same format as given by the template, or 
just provide the same information as specified by the template?  If the 
latter, then I think a table such as the one I stole from Charles is fine.

> The issue of documenting obsoleted headers is problematic. We can (in 
> theory) argue that the sentences referring to them as "obsoleted" in 
> usefor-usefor is enough definition for an obsoleted header. It would be 
> more proper to have son-of-1036 as the reference - but that introduces a 
> dependency, which is bad.

I don't believe that the sentences that refer to the obsoleted headers 
provide enough of a definition, especially without an ABNF grammar 
somewhere.  Referencing s-o-1036 would be good if it has a formal 
definition of the headers.  The dependency is only bad if s-o-1036 
doesn't get into the RFC editor queue before USEFOR.

-- 
Kenneth Murchison
Systems Programmer
Carnegie Mellon University

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: IANA considerations
Date:  Mon, 02 Jan 2006 13:22:59 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 31
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> the registry points to the documents that contain the complete
> registration templates. I don't think we can break the rules
> by not submitting the templates.

Okay, so either a separate document (my preference, same style
as for mail and http), or complete templates in the core spec.
(ugh).  It's a piece of cake to create the former, but I don't
want to waste time for it if we then pick the latter strategy.

With split documents an outline for the core spec. would be:

IANA Considerations

  The news header fields specified or obsoleted by this memo
  are listed separately in [draft-ietf-usefor-hdrreg-00] with
  the registration templates for the IANA registry of header
  fields.
[...]

Informative References
[...]
  [draft-ietf-usefor-hdrreg-00] Registration of news header
  fields specified or obsoleted by [RFC XXXX] (this memo).

xml2rfc has some magic supporting self-references [RFC XXXX],
and draft-ietf-usefor-hdrreg-00 would be an informational RFC.

                             Bye, Frank

Date: Sun, 01 Jan 2006 14:57:54 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: IANA considerations
Sender: [email-masked]

--==========45FE29AC6D08DFC5E696==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

--On 28. desember 2005 19:41 +0000 Charles Lindsey <[email-masked]>=20
wrote:

>
> In <4E1927EA4BCAE0EFDB80BE29@B50854F0A9192E8EC6CDA126> Harald Tveit
> Alvestrand <[email-masked]> writes:
>
>> Grumble. The HTTP and Mail documents both give full templates per
>> header,=3D20 and as you say, this makes for a very long document (easy =
to
>> produce, just=3D20 tedious). I like the idea of not making -usefor =
depend
>> on this.
>
> Well looking at the registries IANA have actually constructed
> (http://www.iana.org/assignments/message-headers/perm-headers.html and
> http://www.iana.org/assignments/message-headers/prov-headers.html), they
> have only taken the bare bones of those documents - essentially what =
Ken's
> table proposed), so I think that is all we are obliged to provide. In
> fact, they haven't even included the 'Status' and 'Author/Change
> controller' fields at all, which I find odd. And for sure, if we want any
> "related information" to be attached to some header, we will need to say
> so explicitly in out IANA Considerations.

Note that the registry points to the documents that contain the complete=20
registration templates. I don't think we can break the rules by not=20
submitting the templates.

(The question of how IANA maintains registries is a completely different=20
subject.)

--==========45FE29AC6D08DFC5E696==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========45FE29AC6D08DFC5E696==========--