Subject: #1047 full circle

From: Frank Ellermann <[email-masked]>
Date: Tue, 21 Feb 2006 09:37:49 +0100

Hi, a curious effect, I tried to simplify the <path>-ABNF 
with a mandatory <tail-entry> as in s-o-1036 and optional
<diag-posted> as proposed by Harald, and arrived at the
same syntax already proposed and discussed some weeks ago.

It's now only clearer (my POV) why <diag-posted> is never
combined with other diagnostics.  Two possibilities:

     diag-posted = "!.POSTED." diag-identity

Simple, but it doesn't allow a terse ...!.POSTED!...

     diag-posted = "!.POSTED" [ "." diag-identity ]

Could the folks supporting POSTED please state what they
like better ?  For a <tail-entry> I think it's now clear
why allowing dots would be a bad idea, we need a "SHOULD
be not-for-mail" or a decent hint in this direction.

JFTR I add the complete ABNF below, you know it already
from an earlier attempt.  Can we declare "#1047 solved" ?

                          Bye, Frank

; ------------------------------------------------------------------#

path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF
path-list       =  *( path-identity [FWS] [path-diagnostic] "!" )

path-diagnostic =  diag-match / diag-mismatch / diag-seen /
                   diag-posted / diag-deprecated

diag-match      =  "!"                           ; an additional "!"
diag-seen       =  "!.SEEN."           diag-identity
diag-mismatch   =  "!.MISMATCH."       diag-identity
diag-posted     =  "!.POSTED"    [ "." diag-identity ]
diag-deprecated =  "!" 1*( path-nodot "." ) path-nodot

diag-identity   =  path-identity / IPv4address / IPv6address
tail-entry      =  path-nodot                    ; not-for-mail

path-identity   =  ( 1*( label "." ) toplabel ) / path-nodot
path-nodot      =  1*( alphanum / "-" / "_" )    ; legacy names

label           =  alphanum [ *( alphanum / "-" ) alphanum ]
toplabel        =  ( [ label *( "-" ) ] ALPHA   *( "-" ) label   ) /
                   (   label *( "-" )   ALPHA [ *( "-" ) label ] ) /
                   (   label          1*( "-" )          label   )

alphanum        =  ALPHA / DIGIT                 ; compare RFC 3696

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 22 Feb 2006 00:34:45 +0100

I'd be happy to declare consensus on this one, so people who object should 
speak up quickly....

Implications:

- Keywords can only be added by updating -usefor-
- diag-deprecated must contain at least one dot, but is otherwise 
free-form, so not distinguishable from path-identity. Was this your 
intention?
(I think this is more like "alt B" in the poll than "alt A" - but if the WG 
now has the consensus to keep it in, that's OK with me too)

Personally, I like to allow "!.POSTED!", so would prefer your second 
alternative - YMMV.

                        Harald

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 22 Feb 2006 04:58:19 +0100

> - Keywords can only be added by updating -usefor-

Yes.  Otherwise it's too difficult to split it cleanly, we
would get diag-match and diag-obs in USEFOR, and diag-other
details in USEPRO, enumerating POSTED / SEEN / MATCH there
for a new diag-keyword in USEFOR, that's IMO too confusing:

 diag-match      = "!"                     ; another "!"
 diag-deprecated = "!" IPv4address         ; (see below)
 diag-other      = "!." diag-keyword [ "." diag-identity ]

 diag-keyword    = 1*ALPHA                 ; see USEPRO

Besides <diag-identity> isn't optional for !.SEEN or !.MATCH

> - diag-deprecated must contain at least one dot, but is
> otherwise free-form, so not distinguishable from
> path-identity. Was this your intention?

That should be Russ' intention (*), "other crap with a dot not
matched by a <path-identity>".  What we really need is IPv4.

We could say so (shown above).  Then some crappy paths where
nobody knows what it is, neither a <path-identity> nor IPv4,
would be "syntactically invalid".  Whatever that means.

For Russ' version (*) we need prose explaining the ambiguity,
and why <path-identity> should match before <diag-deprecated>
for _some_ crappy paths.  Weirder nonsense like adjacent dots
would be still invalid in this version.

So maybe  diag-deprecated = "!" IPv4address  is better.  Bye

*: of course read "as I understood what Russ said"

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 23 Feb 2006 12:24:12 GMT

>I'd be happy to declare consensus on this one, so people who object should 

Well it's certainly a cleaner syntax than some of Frank's earlier ones.

>- Keywords can only be added by updating -usefor-

Not so sure about that. See my comments in reply to Frank.

>free-form, so not distinguishable from path-identity. Was this your 

It is ambiguous, but maybe it is best left so. But I would prefer to see
it as a deprecated-identity than as a deprecated-diagnostic. Again, see my
reply to Frank.

>Personally, I like to allow "!.POSTED!", so would prefer your second 

Yes, the nakes ...!.POSTED!... is still needed.

>> path-nodot      =  1*( alphanum / "-" / "_" )    ; legacy names

<path-nodot> is what we have sometimes referred to as a <bareword>.
Whatever it is called, I would object strongly to including the word
"legacy" in here. USEPRO contains wording to describe what should and
should not be used for a <path-identity>, and a domain-name is much to be
preferred, and hence you need a good reason to be using a
<bareword/path-nodot>. But that is not to say that such "good reasons" may
continue to arise in the future, hence "legacy" is not correct.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 23 Feb 2006 12:47:46 GMT

>for a new diag-keyword in USEFOR, that's IMO too confusing:

Nevertheless, I think we should try it that way.

> diag-other      = "!." diag-keyword [ "." diag-identity ]

<diag-match> needs to be in USEFOR because it is a special case
syntactically speaking. There would be wording to explain this, along with
wording to direct you to USEPRO for that and for rules explaining how to
use <diag-keyword>s.

I would remove <diag-deprecated> to <identity-deprecated>. Syntactically,
it looks like a <path-identity>, and software will inevitably treat it as
such (i.e. software may detect and act upon a "." after a "!", but is
unlikely to detect IP addresses as such - you just never put them in your
'sys' file and so they never get recognized to be acted upon).

>Besides <diag-identity> isn't optional for !.SEEN or !.MATCH

Indeed. There will be all sorts of unsuitable things allowed
syntactically, but so what? They cause no harm. USEPRO will tell you what
you are allowed to put, and strongly discourage anything else.

>matched by a <path-identity>".  What we really need is IPv4.

Personally, I would just allow anything separated by dots in
<path-identity> (plus <path-nodot/bareword> of course) and then say that,
although that would permit IPv4addresses, that was only for legacy (and
possibly diagnostic) reasons - i.e. MUST accept, MUST NOT generate.

And I would also allow multiple diagnostics after each <path-identity> (so
people can say .SEEN-like things as well as .POSTED), and leave it to
USEPRO to restrict it to sensible usages. Again, no harm arises from a
more liberal syntax (and software is not going to bother to check it
anyway).

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 23 Feb 2006 20:31:04 +0100

Charles Lindsey wrote:

>> it's too difficult to split it cleanly, we would get
>> diag-match and diag-obs in USEFOR, and diag-other
>> details in USEPRO, enumerating POSTED / SEEN / MATCH
>> there for a new diag-keyword in USEFOR, that's IMO too
>> confusing:

> Nevertheless, I think we should try it that way.

IBTD.  It's not the same situation as for the "Control:", where
a clean split is possible and makes sense.  You've enough to do
to explain the various diagnostics in USEPRO.

We're talking about three ABNF lines (for !.s, !.m, !.p) versus
two lines "diag-other" and "diag.keywords" here.

> <diag-match> needs to be in USEFOR because it is a special
> case syntactically speaking. There would be wording to
> explain this, along with wording to direct you to USEPRO

It is OBVIOUS if the complete syntax is here, no additional
wording needed.  What can <diag-match> be if it's followed
by <diag-mismatch> ?  You only need long and irritating prose
in both documents if you split it.

> I would remove <diag-deprecated> to <identity-deprecated>.
> Syntactically, it looks like a <path-identity>, and
> software will inevitably treat it as such

But we've decided that it shouldn't do this for IPv4, and IIRC
Richard observed that IPv4 is apparently used as some kind of
diagnostics in paths today.  Not as identity.  We had a poll
and rough consensus about "no IP as identity".

We can reduce <diag-deprecated> to  "!" IPv4address  to get rid
of the ambiguity.  It's then also more obvious that it's _not_
some kind of <identity-deprecated>:

<IPv4address> doesn't match <path-identity>, and vice versa.

As you said elsewhere no harm done if a weirdo puts !oo..ps in
the path (adjacent dots).  Matching !o_o.p_s (underscore) as a
"valid" <diag-deprecated> doesn't really help if we don't catch
the similar !oo..ps,  I'd say let's take the clear IPv4address.

> syntactically, but so what? They cause no harm.

All crap in the path can be harmful, it makes it longer, that
means folding, and I'm far from sure that all news servers will
get a folded path right (both creating and reading).

> USEPRO will tell you what you are allowed to put, and
> strongly discourage anything else.

That's unnecessary for !.SEEN and !.MISMATCH if the syntax is
here and clear.  You can then focus on the important stuff in
USEPRO.  Without wasting time for pointless explanations why a
<diag-identity> is required after "!.SEEN." and "!.MISMATCH."

No SHOULD or other MUSTard needed, neither here nor in USEPRO.

> no harm arises from a more liberal syntax (and software is
> not going to bother to check it anyway

I strongly disagree that a "liberal syntax" with "MUST accept"
and "MUST NOT generate" is better than a clear syntax without
any prose, let alone MUSTard.

> <path-nodot> is what we have sometimes referred to as a

Also sometimes referred as <path-legacy>, yes.  <path-nodot>
stresses the syntactically more important point, no dot.

> Whatever it is called, I would object strongly to including
> the word "legacy" in here.

Yes, I insist on it because it's the truth.  UUCP-names _are_
legacy.  We cannot protect them in any way against collisions
in their namespace.  They might even clash with a <tail-entry>
on 1036-servers (and in theory that's _all_  servers today).

All we can do to protect them is to forbid FQDNs without dot
(= TLDs), and we've done this.  For more protection we'd need
the UUCP world map as a new IANA registry or a similar stunt.

As long as this doesn't happen I insist on saying "legacy" for
"legacy", because it's an utter dubious idea to create any new
identities in this legacy style in an uncoordinated manner.

                          Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 27 Feb 2006 11:10:58 GMT

>two lines "diag-other" and "diag.keywords" here.

It's not about saving lines. It's about allowing for future extensibility
(always a good idea when you are introducing something new, which may need
some fine tuning later). And maybe even allowing keywords beginning with
"X" for people to use experimentally.

And also it is about not making any suggestion that software should be
parsing and rejecting Path headers at that level of detail. So long as you
can recognize the delimiters and FWS, and so separate out the items, and
then maybe distinguish which items are diagnostics and which are not
(assuming the particular piece of software neds to know that distinction),
then it does not really matter what is present, syntactically speaking.

OTOH, for making the protocol work, and for providing information useful
for troubleshooting, then there need to be rules/conventions/whatever for
what goes in there, but USEPRO is the place for those.

>and rough consensus about "no IP as identity".

Yes, but the poll was for what would be allowed in future. In the present,
people DO put IPv4 addresses in the Path, so thay are MUST accept but MUST
NOT generate in future. But whether the ones in current use are there for
identity of diagnostic reasons is irrelevant to that - either way they are
still MUST accept NOT generate. Yes, we believe that most of the present
usage is diagnostic, but that is irrelevant and nothing is gained by
writing it into the syntax (and in fact doing so causes harm by suggesting
that it has some meaning that parsers should be taking note of).

>on 1036-servers (and in theory that's _all_  servers today).

Who said a bareword was necessarily UUCP? In any case, some news is still
transported by UUCP. We agreed months ago on some wording now in USEPOR on
what was acceptable for identities. It was made clear that barewords were
only to be used where there was a Good Reason (and "legacy", or rather
"longstanding usage" was certainly one Good Rreason). And it was made clear
that uniqueness could not be absolutely guaranteed. That said, it was left
to newsadmins to decide in each case whether the benefit outweighed the
disadvantage.

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 27 Feb 2006 14:37:38 +0100

Charles Lindsey wrote:

> Who said a bareword was necessarily UUCP?

s-o-1036 chapter 5.6 (excl. its obscure "cooperating subnets"):

| a relayer name MUST be either a UUCP name registered in the
| UUCP maps (without any domain suffix such as ".UUCP"), or a
| complete Internet domain name.

> It was made clear that barewords were only to be used 
> where there was a Good Reason (and "legacy", or rather
> "longstanding usage" was certainly one Good Rreason).

Yes, of course, we're talking about a comment in the ABNF line:

| path-nodot   =  1*( alphanum / "-" / "_" )    ; legacy names

Other points skipped, obviously we disagree.  If the simple set
of four diagnostics !.m, !.p, !.s, and !! is already dubious I
fail to see why future extensions incl. !.X-whatever should be
desirable.  This is the <path>, it's transported by all relays
and all readers of an article, it's not the place to allow some
kind of X-Face for relays.
                          Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 28 Feb 2006 10:34:14 GMT

>kind of X-Face for relays.

The Path header in news serves a similar purpose to the Received header in
email. In recent years, the content of the Received header has grown like
Topsy as people have crammed more and more information into it, putting
formatted stuff into comments (a gross misuse of the comment facility, but
there was no other way to do it). By and large, this increased information
is desirable and useful, in spite of the fact that the notations used are
far from consistent.

What this shows is that the need for diagnostic information tends to grow,
but that the syntax of the Received header was far too restrictive to
accomodate what turned out to be needed.

Let us not make the same mistake. I doubt that we need as much information
as currently goes into Received headers, and I cannot at the moment
foresee more than the !.m, !.p, !.s, and !! that you mention, of which the
"!!" and the "!.p" are the only ones which should regularly appear. But it
costs us nothing to allow for future extension should it be needed.

Yes, there is the possility that some relayers will abuse the
possibilities, but you may be sure that USEPRO will make it clear such
practices are a Bad Thing (probably with SHOULD NOT wording).