Subject: #1047 POLL: Path-diagnostics

From: Harald Tveit Alvestrand <[email-masked]>
Date: Mon, 16 Jan 2006 04:07:26 +0100

OK, it seems that further discussion is not bringing us much closer to a 
single point of view on this one. But I think I've seen enough to suggest 
another poll.

Please indicate whether your point of view matches:

A: A path-diagnostic must be syntactically distinguishable from a 
path-identity. Stuff that isn't a path-identity or a syntactically marked 
path-diagnostic (such as an IP address) should be syntacitcally illegal.

B: A path-diagnostic must be syntactically distinguishable from a 
path-identity. Stuff that isn't a path-identity or a syntactically marked 
path-diagnostic (such as an IP address) should be syntacitcally legal, but 
have no standardized meaning.

C: A path-diagnostic need not be syntactically distinguishable from a 
path-identity.

D: None of the above alternatives is close to my opinion.

If A or B is the rough consensus of the WG, I'll suggest that a 
path-diagnostic always start with a dot; that's a simple marker.

If B is the consensus, I think I'll propose a third category of "stuff in 
path" for the ABNF - "path-legacy-rubbish" or something similarly 
descriptive.

                  Harald

---

From: Russ Allbery <[email-masked]>
Date: Sun, 15 Jan 2006 19:15:31 -0800

Harald Tveit Alvestrand <[email-masked]> writes:

> path-identity. Stuff that isn't a path-identity or a syntactically
> marked path-diagnostic (such as an IP address) should be syntacitcally
> legal, but have no standardized meaning.

I think this is the most realistic option that still adds path diagnostics
(which is something for which there is a clear need and utility, given
that people keep independently reinventing them -- just because they're
not strongly authenticated doesn't mean they're not useful in tracking
down problems).  There's a bunch of legacy stuff in the Path header that
realistically isn't going to go away.

> C: A path-diagnostic need not be syntactically distinguishable from a 

I can live with path diagnostics going away completely as too hard to
standardize.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Seth Breidbart <[email-masked]>
Date: Sun, 15 Jan 2006 23:53:32 -0500 (EST)

> A: A path-diagnostic must be syntactically distinguishable from a=20
> path-identity. Stuff that isn't a path-identity or a syntactically marked=20

Preference.

B is also acceptable.

I don't like C.

> If A or B is the rough consensus of the WG, I'll suggest that a=20

However, that has a bad effect on existing path-diagnostics.

Seth

---

From: Russ Allbery <[email-masked]>
Date: Sun, 15 Jan 2006 21:03:07 -0800

Seth Breidbart <[email-masked]> writes:

>> illegal.

> However, that has a bad effect on existing path-diagnostics.

The only way that you're going to not have a bad effect on existing
path-diagnostics is with C.

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 16 Jan 2006 06:33:39 +0100

Harald Tveit Alvestrand wrote:
 
> I've seen enough to suggest another poll.

I could support A and D, but not C.  IFF we want or allow
<path-diagnostic> it should be syntactically distinguishable.

In both ...!x!y!SEEN!... and ...!x!y!MISMATCH!... it's IMO
already good enough:  x must be a <path-identity>, either a
FQDN with a dot, or some legacy given name not matching SEEN,
MISMATCH, or POSTED.  Definitely no IPv4address or IPv6address.

The obscure case FQDN without dot (= TLD) would be treated
like a legacy given name, if somebody would insist on TRAVEL,
CAT, JOBS, MOBI, COOP, AERO, MUSEUM etc. nothing breaks

The y in y!SEEN or y!MISMATCH is (a part of) the diagnostic,
it can be an IPv4address.  Some B-news systems interpreting
colon as delimiter might be confused by an IPv6address, as
discussed elsewhere.  

That's my POV for the state of the art assuming that we want
a diagnostic at all.  Without B-news compatible ersatz-IPv6,
for strict backwards compatibility we'd need a "structured"
Subject: cmsg and several other oddities.
 
> B:
[... starting like A, but then ...]
> Stuff that isn't a path-identity or a syntactically marked
> path-diagnostic (such as an IP address) should be
> syntacitcally legal, but have no standardized meaning.

It's already tricky to get the syntax for A right.  I'm lost
how to add "random garbage" as a third option in addition to
<path-identity> vs. <path-diagnostic>.

> C: A path-diagnostic need not be syntactically 
> distinguishable from a path-identity.

Either that makes no sense, if it's not distinguishable why
talk about it.  Or what you really wanted to propose is to
allow all kinds of random garbage not limited to FQDN, IPv4,
IPv6, legacy names, TLDs, obscure keywords, pseudo-TLDs, etc.

> If A or B is the rough consensus of the WG, I'll suggest
> that a path-diagnostic always start with a dot; that's a
> simple marker.

Is that something like the folowing ?

1:  ...!x!y!SEEN!...     => ...!x!.y!... 
2:  ...!x!y!MISMATCH!... => ...!x!.y.MISMATCH!...
3:  ...!x!POSTED!...     => ...!x!.POSTED!...
4:  ...!x!!...           => ...!x!!...

No obvious problem, you could also pick verbose variants for
(1) adding .SEEN, and for (4) using .MATCH.  Some advantages
of this idea:  

a: It's hard to misinterpret a leading dot.  With a trailing
   dot I'd be less confident, 2821bis might allow it.

b: With a mandatory leading dot .POSTED, and .y.MISMATCH can't
   be confused with a TLD.  They are not used as pseudo-TLD.

c: On that solid base we _could_ s/toblabel/label/ and delete
   <toplabel> - silently permitting IPv4 as <path-identity>.

Whatever it is, let's just get it right.  So far we have "!"
as the only permitted delimiter (as stated in s-o-1036), and
that's our excuse to allow the IPv6 colon.  

Like the removal of "Subject: cmsg" that's a very important
difference from RFC 1036, this must be stated in appendix B.

                          Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 16 Jan 2006 07:23:10 +0100

Russ Allbery wrote:

 [B] 
> I think this is the most realistic option that still adds
> path diagnostics (which is something for which there is a
> clear need and utility, given that people keep independently
> reinventing them
[...]
> There's a bunch of legacy stuff in the Path header that

And how would you add the latter to the syntax ?  For A using
Harald's "leading dot" idea I get something like the following
verbose variant without <toplabel>:

path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF
path-list       =  *( path-entry "!" [FWS] )
path-entry      =  path-identity / path-verified / path-unverified /
                   path-mismatch / path-posted

path-verified   =  path-identity "!.MATCH"
path-unverified =  path-identity "!." path-diagnostic ".SEEN"
path-mismatch   =  path-identity "!." path-diagnostic ".MISMATCH"
path-posted     =  path-identity "!.POSTED"

path-diagnostic =  path-identity / IPv4address / IPv6address
path-identity   =  ( 1*( label "." ) label ) / path-legacy

path-legacy     =  1*( ALPHA / DIGIT / "-" / "_" )
tail-entry      =  path-legacy                   ; allows underscore

label           =  letdig [ *( letdig / "-" ) letdig ]
letdig          =  ALPHA / DIGIT 

It's possible to get rid of ".MATCH" for Henry's more elegant
"!!" idea, and a terse variant could also eliminate ".SEEN".

To allow more constructs you'd have to replace <path-identity>
in <path-entry> by some <path-deprecated>, starting like this:

path-deprecated =  path-identity / path-other-TBD

Do you have an idea for <path-other-TBD> ?  Harald wrote that
he has an idea, but without seeing it I'm reluctant to support
B, it could turn out to be essentially the same as C.

                           Bye, Frank

---

From: Russ Allbery <[email-masked]>
Date: Sun, 15 Jan 2006 22:33:08 -0800

> verbose variant without <toplabel>:

Don't you already have it here?

> tail-entry      =  path-legacy                   ; allows underscore

path-legacy looks like exactly what I'm talking about.

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 16 Jan 2006 09:58:00 +0100

> path-legacy looks like exactly what I'm talking about.

That's all, formerly known as "barwword", no dots allowed,
but underscore is okay ?

Well, if what I thought "A" could be is in fact what you
think "B" could be, then Harald has some leaway for his
Chair-magic.

-2-

I forgot to mention one disadvantage / advantage of the
"leading dot" idea:  Old systems not knowing this syntax
won't "see" any <path-identity> or IP hidden behind a dot.

Is that actually a feature or a problem or irrelevant ?

-3-

And my attempt to get rid of <toplabel> might be wrong,
<IPv4address> matches <path-identity> without <toplabel>:

If we want this ambiguity we should sort the alternatives
of <path-diagnostic> from most specific to less specific:

path-diagnostic =  IPv6address / IPv4address / path-identity

With <toplabel> it's IMO cleaner, but IIRC only Charles
agreed that <toplabel> is a good idea.

Maybe Harald could start a "toplabel-task-force" with
Donald (2606bis) and JohnK (3696 + IDNA).  It's harmless,
only updating a note in RfC 1123, nothing to worry about,
no appeals announced, ICANN will love more potential TLDs,
less bogus queries for all-digit "TLDs", it helps with a
detail in 2821bis...  so why do I feel so queasy ?

                         Bye, Frank

---

From: Russ Allbery <[email-masked]>
Date: Mon, 16 Jan 2006 01:25:59 -0800

> That's all, formerly known as "barwword", no dots allowed,

Well, once you add dots, it looks just like a regular path-identity except
that it may not actually be a valid FQDN.  At that point, you can't
*syntactically* tell the difference between legacy path entries and the
path entries we're encouraging.

> Is that actually a feature or a problem or irrelevant ?

I'm not sure I understand.  I assume that we're talking about something
like:

    !.MISMATCH!some.site

Old systems will interpret both of those as path identities, with harmless
results for .MISMATCH.  I don't think anything is going to be hidden.

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 16 Jan 2006 11:28:36 +0100

Russ Allbery wrote:

> once you add dots, it looks just like a regular path-identity
> except that it may not actually be a valid FQDN.

Yes, that's why we didn't add dots so far.  We could resulting
in "like a valid host name" vs. "like an FQDN with underscores"

> At that point, you can't *syntactically* tell the difference
> between legacy path entries and the path entries we're
> encouraging.

With the "leading-dot" concept (but see below) we might win the
leaway for legacy constructs without a leading dot.  Raw idea:

path-identity  = ( 1*( label "." ) toplabel ) / path-legacy ; as before
path-legacy    = *( path-component "." ) path-component     ; MODIFIED
path-component = letdig [ *( letdig / "-" / "_" ) letdig ]  ; allows "_"

tail-entry     = path-legacy                                ; as before, NOW WITH DOT
label          = letdig [ *( letdig / "-" ) letdig ]        ; as before

If dots in <tail-entry> are wrong s/path-legacy/path-component/
If dots in <tail-entry> are okay my prior proposals were wrong.

This syntax is ambiguous, all FQDNs are also <path-legacy>,
but for one or more underscores only <path-legacy> matches.

Unfortunately <IPv4address> also matches if we allow dots.

If you want to allow a leading or trailing "-" or "_" in the
<path-component> of <path-legacy> its syntax can be simplified:

path-component = 1*( ALPHA / DIGIT / "-" / "_" )

If you also want to allow adjacent dots for the <path-legacy>
"like..this" it's more difficult, but not impossible, and we
solved a similar problem for <id-unique> of a Message-ID.

-2-

> I'm not sure I understand.  I assume that we're talking about
> something like:

>     !.MISMATCH!some.site

So far I thought that Harald's idea is "!.some.site.MISMATCH"
( or maybe "!.MISMATCH.some.site", matter of taste ) and then
old systems won't see the hidden "some.site" in this construct.

With your proposal "some.site" is again visible:

> Old systems will interpret both of those as path identities,
> with harmless results for .MISMATCH.

ACK, and apparently you prefer to have the "some.site" visible,
that was my question - I didn't have this feature in my syntax.

Below is my first "Russ' plan B" attempt reflecting this.  I'm
not sure about allowing this broad new "<path-legacy> with dot"
within <path-diagnostic>.  Just disallow it there, maybe ?

path-verified   =  path-identity "!"             ; an additional "!"
path-unverified =  path-identity "!.SEEN!"     path-diagnostic
path-mismatch   =  path-identity "!.MISMATCH!" path-diagnostic
path-posted     =  path-identity "!.POSTED"

path-diagnostic =  IPv6address / IPv4address / path-identity
path-identity   =  ( 1*( label "." ) toplabel ) / path-legacy

path-legacy     =  *( path-component "." ) path-component
path-component  =  1*( ALPHA / DIGIT / "-" / "_" )
tail-entry      =  path-legacy                   ; allows underscore

label           =  letdig [ *( letdig / "-" ) letdig ]
letdig          =  ALPHA / DIGIT                 ; compare RFC 3696

toplabel        =  ( [ label *( "-" ) ] ALPHA   *( "-" ) label   ) /
                   (   label *( "-" )   ALPHA [ *( "-" ) label ] ) /
                   (   label          1*( "-" )          label   )

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Mon, 16 Jan 2006 11:33:18 +0100

> path entries we're encouraging.

this is actually fairly common in ABNFs of various kinds.
For instance, the RFC 2822 ABNF has:

fields          =       *(trace
                          *(resent-date /
                           resent-from /
                           resent-sender /
                           resent-to /
                           resent-cc /
                           resent-bcc /
                           resent-msg-id))
                        *(orig-date /
                        from /
                        sender /
                       .....
                        keywords /
                        optional-field)

where "optional-field" is

 optional-field  =       field-name ":" unstructured CRLF

 field-name      =       1*ftext

 ftext           =       %d33-57 /               ; Any character except
                        %d59-126                ;  controls, SP, and
                                                ;  ":".

All the headers will match "optional-field"; it's a matter of giving 
priority in parsing (ugh) to parse "from" as a from-field and not as an 
optional-field.

The same type of construction can be used for legacy-junk-path-entry or 
whatever we end up calling it (and we can attach one of the MUST be 
accepted, SHOULD NOT be generated warning labels to it if we want to).

Messy, but not novel.

                    Harald

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Mon, 16 Jan 2006 11:27:26 +0100

> won't "see" any <path-identity> or IP hidden behind a dot.

don't understand this.... in the path

  x!.y!z

do you mean that the parser won't try to see y as a path-identity, or that 
it won't see z?

If it won't see y, that's a Good Thing IMHO; if it won't see z, we have a 
problem.
> Is that actually a feature or a problem or irrelevant ?

depends on what you mean.....

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 16 Jan 2006 14:40:42 +0100

> do you mean that the parser won't try to see y as a
> path-identity, or that it won't see z?
 
> If it won't see y, that's a Good Thing IMHO; if it won't
> see z, we have a problem.

Yes, I thought your idea was x!.y.z without second ! within
a <path-entry>.  For x!.y!z the z is visible, and if we want
this all is fine.

Your 2822 optional-field example is interesting.  So we _can_
have some "catch-all" rules and similar ambiguities, as long
as they come after all matching more specific alternatives.

Here <IPv4address> before <path-identity> before <path-legacy>
could help to define host names with a simple "at least two
LDH-labels" rule (ducking the <toplabel> issue if that really
is an issue), and a <path-legacy> could be dot-separated junk
of LDH or underscore.  
                          Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 16 Jan 2006 14:53:10 +0100

[...]
>                         optional-field)

Ugh, and we inherit this using fields =/ <our-stuff>.

> priority in parsing (ugh) to parse "from" as a from-field and not as
> an optional-field.

You just killed all our header fields, because they come after
the <optional-field> catch-all.

> Messy, but not novel.

Okay, proposed fix, we rename our <fields> to <news-fields>,
and replace the "fields =/ <our-stuff>" in draft -06 by a 

            news-fields = <our-stuff> / fields

With that trick <optional-field> is again the last catch-all.

Besides Bill's parser never liked the old fields =/ approach,
it always proposed to join the two parts of the <fields>.  Bye

---

From: Richard Clayton <[email-masked]>
Date: Mon, 16 Jan 2006 14:02:42 +0000

In message <EB731E6472DBE44DCEBB6C4C@B50854F0A9192E8EC6CDA126>, Harald
Tveit Alvestrand <[email-masked]> writes

>path-diagnostic always start with a dot; that's a simple marker.

As noted elsewhere, I think the diagnostics are a snare and delusion
(and although Russ says they keep being reinvented, they don't seem to
be useful enough to stick).

If we must have them [and just documenting what's out there as being a
mess would be preferable to inventing YARITWCIFD]...

... then using the existing deployed scheme would seem to be best way
forward (.MISMATCH) ... but if there's only one piece of software
creating them, then maybe that's not necessary.  Understanding that
point would seem to be useful for the next stage of decision-making.

So to the poll -- of these current alternatives, then if we must have
diagnostics, then A would be closest to my viewpoint.

However, I don't feel very strongly about it, since no-one is going to
machine process this stuff and so it will be processed semantically
instead -- and all sorts of confusion can be dealt with very easily.

I do have some sympathy with what I understand to be John Stanley's view
that sites called "dead" or "beef" can have some problems not of their
making... but since they don't seem to exist, I can live with a scheme
that disadvantages them.

- -- 
richard                                                   Richard Clayton

Those who would give up essential Liberty, to purchase a little temporary 
Safety, deserve neither Liberty nor Safety. Benjamin Franklin 11 Nov 1755

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Mon, 16 Jan 2006 15:12:55 +0100

> as they come after all matching more specific alternatives.

I think ABNF is actually silent on the subject; the running code seems to 
be "ambiguous grammars are OK, you have to look at the text to figure out 
how to resolve them".

Ugly....

---

From: Russ Allbery <[email-masked]>
Date: Mon, 16 Jan 2006 10:56:07 -0800

> but for one or more underscores only <path-legacy> matches.

Yeah, that sounds like what I was thinking about.  It doesn't assign
productions unambiguously to different elements, but it does represent
what's out there now with the addition of diagnostics.

> solved a similar problem for <id-unique> of a Message-ID.

I don't.  :)  I doubt anyone is using such things in anger, and I have no
problem with it being syntactically invalid and then falling into the
general question of how much leeway one wants to give to syntactically
invalid messages.

> that was my question - I didn't have this feature in my syntax.

Oh, yes, you do want it to be visible since the most likely cause of a
mismatch is a simple misconfiguration and you still want the path to have
an effect then.

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 17 Jan 2006 12:03:06 GMT

>mismatch is a simple misconfiguration and you still want the path to have

Are you saying that if you see

    !site-a!some.site.MISMATCH!some.sote!........

(where "some.sote" is a misconfiguration of the correct "some.site") then
you want later sites that peer with some.sight to refrain sending stuff
back to them? I am not sure I agree (sites which misconfigure themselves
deserve what they get), but if that were wanted then we would be better
with the notation in the present draft

    !site-a!some.site!MISMATCH!some.sote!........

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 17 Jan 2006 11:36:21 GMT

>>> A:

>path-diagnostics is with C.

I don't follow that. Existing diagnostics appear to be of the form

   !news.example.net.MISMATCH!

which is, syntactically speaking, an FQDN.

Unless you are meaning existing diagnostic usages such as

   !123.123.234.234!

which maybe we need to allow as a 'legacy case'.

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 17 Jan 2006 11:31:11 GMT

>path-diagnostic (such as an IP address) should be syntacitcally illegal.

I prefer A - assuming that <path-identity> will cover FQDNs and barewords
but not IP addresses.

There might also be some further legacy cases to consider, though I doubt
there is any existing usage of things starting with a dot, or with 2 dots
consecutive. So that might be moving a little closer to B.

>path-diagnostic always start with a dot; that's a simple marker.

I think an intial dot is a bit too inconspicuous. If you want something
that is clearly not an FQDN, then Forrest published a list of several
alternatives a while back, from which I selected

       !news.example.com..MISMATCH!

as the best (two dots in succession is the marker, but it is still
recognizable to people used to the current DNEWS method). Doesn't work so
well for POSTED, though (how about !..POSTED!).

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 17 Jan 2006 11:49:09 GMT

>point would seem to be useful for the next stage of decision-making.

There are two problems with the existing diagnostics DNEWS-style:

1. "alleged.example.net.MISMATCH" is an FQDN (hence my proposal, taken
from a suggestion by Forrest, for "alleged.example.net..MISMATCH"). But I
doubt ICANN would be so stupid to invent "MISMATCH" as a TLD (or maybe
they wouldn't :-( ).

2. "news.example.net.POSTED: is always added by the genuine site
"news.example.net" (which is a true <path-identity>, not a <diagnostic>).
Nevertheless, present systems seeing then will not realise that they are
not supposed to be sending the article back to news.example.net (unless
they have carefully configured the full news.example.net.POSTED into their
'sys' files as an alias). That is the reason we invented the
"!news.example.net!POSTED!" notation as it appears in the present draft.

I think #2 is an important argument against the present deployed usage. I
agree that #1 is more of a red herring.

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 17 Jan 2006 11:55:14 GMT

><IPv4address> matches <path-identity> without <toplabel>:

I agree. If we don't want <IPaddress>es (of either flavour) as Path
identities, then either we have to forbid it by syntax (using <toplabel>),
or else we have to forbid it by MUSTard.

OTOH, is we need to allow !123.234.123.234! as a legacy case, then the
MUSTard might be a better way to go about it

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 17 Jan 2006 12:43:37 GMT

>> That's all, formerly known as "barwword", no dots allowed,

I think Frank's use of the term "path-legacy" to cover the "bareword" case
is unfortunate. I think we have already decided that we need barewords.
Richard was quite insistent that "demon" was necessary in addition to
"news.demon.net" because so many sites had "demon" configured into then -
we even wrote some special wording into USEPRO to that effect.

So please can we reserve the word "legacy" for cases which we do not want
to promote for future use, but which are still likely to turn up and
therefore may need special treatment in our draft.

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 17 Jan 2006 18:03:48 +0100

Charles Lindsey wrote:
 
> "alleged.example.net.MISMATCH" is an FQDN

Actually it's a potential hostname (LDH-rule as in 3696).

FQDNs might be anything consisting of lables separated
by dots with certain size limits, each label consisting
of arbitrary octets %x00-FF  (Sorry, I had an overdose
of RfC 4343 recently, but it's really interesting... ;-)

> (hence my proposal, taken from a suggestion by Forrest,
> for "alleged.example.net..MISMATCH").

Harald's proposal "!.MISMATCH" is clearer and more robust.
He and Russ both said they want an "alleged.example.net"
visible, in other words "!.MISMATCH!alleged.example.net!".

> I doubt ICANN would be so stupid to invent "MISMATCH" as
> a TLD (or maybe they wouldn't :-( ).

The latter, it's no question of stupidity,  If they adopt
a policy allowing random TLD registrations by everybody,
then I'm free to register MISMATCH if I feel like it.

For example.de (exists) see the invalid.de (exists) test.de,
the latter is a domain of "Stiftung Warentest", and their
main publication is "test".  This makes perfect sense for
them.  A TLD "MISMATCH" could also make sense for somebody.

We could update 2606 and claim it for our purposes, but just
squat a potential TLD without following "proper" procedures
(after inventing what "proper" is :-) strikes me as bad idea.

                       Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 17 Jan 2006 18:39:38 +0100

Charles Lindsey wrote:

>> my attempt to get rid of <toplabel> might be wrong,

> I agree. If we don't want <IPaddress>es (of either flavour)
> as Path identities, then either we have to forbid it by
> syntax (using <toplabel>), or else we have to forbid it by
> MUSTard.

No MUSTard necessary, just some prose explaining why there's
an ambiguity in the ABNF reflecting Russ' proposal:  Each
potential host name is also a potential <path-legacy>.

The stuff with (1) underscores, or (2) leading / trailing
hyphens, or (3) only digits is <path-legacy> "junk" incl.
<tail-entry>, but not a host name.  That explains it for...

...without MUSTard, maybe "SHOULD NOT generate <path-legacy>"
for readers needing a clue.

For <path-diagnostic> we could be more restrictive and forbid
<path-legacy>.  That would also forbid <path-legacy> without
dots, i.e. bare words.  But I don't see the point of this ABNF
exercise, one general SHOULD NOT is clear.

For Harald's proposal "the first match depth first left to
right can resolve ambiguities" all we need is <IPv4address>
before <path-identity> in <path-diagnostic>, that's trivial:

Complete proposal apparently satisfying most of Russ' points
in <news://news.gmane.org/[email-masked]>
and <http://mid.gmane.org/[email-masked]>

Don't shift the [FWS] before the "!", it doesn't work for the
"!!" case.  With "!.MATCH!" instead of "!!" it's possible but
IMHO ugly, a single <path-entry> should stay in one line.

                                 Bye, Frank

---

From: Russ Allbery <[email-masked]>
Date: Tue, 17 Jan 2006 09:57:34 -0800

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> mismatch is a simple misconfiguration and you still want the path to
>> have an effect then.

> (where "some.sote" is a misconfiguration of the correct "some.site")
> then you want later sites that peer with some.sight to refrain sending
> stuff back to them?  I am not sure I agree (sites which misconfigure
> themselves deserve what they get), but if that were wanted then we would
> be better with the notation in the present draft

>     !site-a!some.site!MISMATCH!some.sote!........

Ah, hm.  Good question.

You're right, the visibility question isn't as obvious as I thought it
was.  I'm not sure which of those would be better.

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 17 Jan 2006 19:47:38 +0100

> You're right, the visibility question isn't as obvious as I
> thought it was.  I'm not sure which of those would be better.

Toss a coin.  

We can hide some.site by ...!.MISMATCH.some.site!some.sote!...

We might be also able to hide some.sote by something like
...!some.site!.MISMATCH.some.sote!... but that would result
in some syntactical assymetries for the SEEN / MATCH cases.

And we can "hide" nothing, after all any "obscurity" affects
only old sites, new sites could extract the "hidden" info if
they insist on it.
                          Bye, Frank

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 18 Jan 2006 08:40:55 +0100

> was.  I'm not sure which of those would be better.

hmm... if I have an attack scenario:

  A-------->B
            ^
    M-------

where M is trying to inject articles to B pretending to be A, then putting 
M's identifier into the path (B!M!MISMATCH!A) is preventing articles from 
going to M, but doing nothing about the fact that the attck articles won't 
be going to A.

B!.MISMATCH.M!A will allow distribution to M. Not sure we care.

On the other hand, in a misconfiguration scenario:

  A----->B---->C
  ^            !
  +------------+
(A misconfigured, calls himself AA without telling B first)

the two paths will be

 B!A!MISMATCH!AA
 B!.MISMATCH.A!AA

If C has been told that A is now AA, he will not send A either article.
If C thinks that A is still called A, he will send him the second, but not 
the first.

But since I don't have much belief in "true verified identity", I don't 
think it matters that much - what B puts next to the MISMATCH is unlikely 
to look anything like a valid name for A.

My NOK 0.50.

                     Harald

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 18 Jan 2006 11:27:43 GMT

>Actually it's a potential hostname (LDH-rule as in 3696).

Indeed, but a simple-minded checker just looking at the syntax would
accept it as a FQDN. Which is why I included the words "syntactically an
FQDN".

>visible, in other words "!.MISMATCH!alleged.example.net!".

That is not Harald's proposal as I understood it. I thought he wanted
something like !.alleged.example.net! with or without a "MISMATCH" tagged
on somewhere.

But in any case, your example should have been
"alleged.example.net!.MISMATCH!"  if you want to leave
"alleged.example.net" visible (but Russ seems to be having second thoughts
on that after my "sote" example).

So I think the question of whether to indicate diagnostics by including
the keyword inside that same path-item, or by its presence as the
immediately following path-item will be decided by whether or not we want
it to be "visible" (to a relaying agent deciding whether to send the
article there). Clearly, we do want it to be so "visible" in the case of
POSTED, so the case to be decided is MISMATCH and SEEN (and MATCH if we
have it).

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 18 Jan 2006 14:38:55 +0100

> something like !.alleged.example.net! with or without a "MISMATCH" tagged

To clarify:
I haven't made a specific proposal so far.
If I were to make a proposal, I'd like to suggest ABNF like this (fragment):

  path-element = path-identity / path-diagnostic / old-path-diagnostic

  path-diagnostic = "." (ALPHA / NUMBER / "-" / "." / "_" / ":")*

  old-path-diagnostic = (ALPHA / NUMBER / "-" / "." / "_" )*

with text saying

  Rules for construction of a path-diagnostic are given in [USEPRO]

Personally, I'd like to embed !MISMATCH! in the diagnostic too, mostly 
because it reduces the number of special cases for the grammar, and makes 
the question of "do we have enough keywords" a problem that doesn't block 
USEFOR.

But it's no showstopper for me if people want to keep MISMATCH a separate 
entry in the path - I just don't see the point of doing so.

                     Harald

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 18 Jan 2006 15:48:28 +0100

Harald Tveit Alvestrand wrote:

> To clarify:  I haven't made a specific proposal so far.

You said that "leading dot" might be a part of it, no problem
with that idea from my POV - iff "we" want a <path-diagnostic>.

> path-diagnostic = "." (ALPHA / NUMBER / "-" / "." / "_" / ":")*

That removes MATCH (aka !!), MISMATCH, POSTED, and SEEN from
the picture.  It allows colons outside of an IPv6address.

It removes the <path-entry> concept limiting [FWS} to the end
of "<path-identity> plus optional <path-disgnostic>" construct.

And it offers no syntactical hint about the semantics of this
<path-diagnostic>, a better name might be <path-opaque-junk>.

> old-path-diagnostic = (ALPHA / NUMBER / "-" / "." / "_" )*

With <path-legacy> (incl. bareword) as part of <path-identity>
<old-path-diagnostic> would be almost never reached by a depth
first left to right parser.  (Only "almost", because you allow
<o-p-d> to be empty, killing the "!!" idea also here).

If you remove <path-legacy> from <path-identity> the bareword
and IPv4address cases would both reach <o-p-d>.

But at least bareword is not necessarily any diagnostic, it's
just a machine name.  So better use <path-legacy> instead of
<o-p-d> for that catch-all construct.

You've dots anywhere in <o-p-d>, the parser must really try to
match "leading dot" first.

I don't like the "path-opaque-junk" <path-diagnostic> aspect of
your proposal, it loses the clear distinction MATCH, MISMATCH,
POSTED, and SEEN with the accompanying restrictions for colons
and [FWS].
                          Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 18 Jan 2006 16:38:34 GMT

>was.  I'm not sure which of those would be better.

OK, then I think we need to discuss that and decide, because it will
determine the notation that we use.

My view is that things before MISMATCH or SEEN should not be visible in
that sense, but things before POSTED most definitely should.

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 18 Jan 2006 18:16:10 GMT

>for readers needing a clue.

No! No! No! No!

What you have called <path-component> (and I would call <bareword>) is NOT
a legacy item. It is a perfectly respectable <path-identity> which people
can and should generate whenever a domain-name is not appropriate, for
whatever reason (for example 'demon', which Richard Clayton insisted was
essential on account of the large number of sites configured to use it).
Use of the name 'legacy' is quite wrong in such situations.

I agree that stuff with leading/trailing oddities (notably '.') would be
legacy, and maybe continuing use of IpV4addresses.

As to what legacy oddities exist out there, I did a trawl through my own
news spool, and found comparatively little.

....!masternews.telia.net.!newsb.telia.net.POSTED!not-for-mail
....!bnewsoutpeer00.bru.ops.eu.uu.nat!$3ef82b3c!133.256.1.103.MISMATCH!not-for-mail
....!demon!pcserviceselectronics.co.uk!paul$
....!easynews!news.alt.net!·×·Mogadon-John×·!

So we have a domain with a spurious '.' at the end from telia.net, a weird
'$3ef82b3c' from bnewsoutpeer00.bru.ops.eu.uu.nat (which itself has a
strange TLD), and a couple of tail-entries, of which one has an illegal '!'
at the end.

So I think we need to be liberal in tail entries.

But my news spool is small. Someone ought to look through a bigger spool.
Here is my Perl script in case others want to use it (it needs to be
embedded in a big 'find').

#!/usr/local/bin/perl -w

use English;
use File::CheckTree;

my $path_entry = "([\\w-]+(\\.[\\w-]+)*|[_\\w-]+)";
my $fws = "(\\n?[ \\t]+)";

$INPUT_RECORD_SEPARATOR = ""; # to get the whole article header
  foreach $arg (@ARGV) {
    if (!validate("$arg -f")) {
      open FILE, "< $arg";
      $head = <FILE>; 
      if ($head =~ /Path:[ \t]+(([^\n]*(\n[ \t]+)?)*)\n/o) {$path = $1;}
      close FILE;
      unless ($path =~ /^($path_entry$fws?!)*$path_entry$/o) {
        print $arg," ",$path,"\n"
      }
    }
  }

>IMHO ugly, a single <path-entry> should stay in one line.

Then fix the syntax so that it is allowed. We certainly need the [FWS]
before the "!". The only issue is whether we need it after as well. BTW,
my Perl script will catch any cases of people using it after, so we shall
see.

---

From: Russ Allbery <[email-masked]>
Date: Wed, 18 Jan 2006 15:08:46 -0800

Charles Lindsey <[email-masked]> writes:

> What you have called <path-component> (and I would call <bareword>) is
> NOT a legacy item. It is a perfectly respectable <path-identity> which
> people can and should generate whenever a domain-name is not
> appropriate, for whatever reason (for example 'demon', which Richard
> Clayton insisted was essential on account of the large number of sites
> configured to use it).  Use of the name 'legacy' is quite wrong in such
> situations.

Why?  I have no problem at all with calling demon legacy.  If they were
setting up their service from scratch right now, they shouldn't use that
Path identity.

Legacy is not the same thing as deprecated syntax.  Legacy is "people who
have been doing this may want to continue for various reasons, but if you
have a choice, don't do this in the future."

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 19 Jan 2006 07:38:19 +0100

Charles Lindsey wrote:
 
>> without MUSTard, maybe "SHOULD NOT generate <path-legacy>"

> What you have called <path-component> (and I would call
> <bareword>) is NOT a legacy item. It is a perfectly
> respectable <path-identity>

IBTD.  All "demon", "wolf", "dead", etc. are legacy when
they occur outside of <tail-entry>.  S-o-1036 5.6 Path says:

| a relayer name MUST be either a UUCP name registered in the
| UUCP maps (without any  domain suffix such as ".UUCP"), or
| a complete Internet domain name.

There are no more UUCP maps in this millennium, and therefore
we get "MUST be" "a complete Internet domain name".  That's
not the case for "demon", it violates a MUST in s-o-1036.  We
are perfectly entitled to say SHOULD NOT and call this legacy.

> I think we need to be liberal in tail entries.

<path-legacy> or Harald'd <o-p-d> _are_ already very liberal.

>> Don't shift the [FWS] before the "!", it doesn't work for
>> the "!!" case.  With "!.MATCH!" instead of "!!" it's
>> possible but IMHO ugly, a single <path-entry> should stay in
>> one line.
 
> Then fix the syntax so that it is allowed. We certainly need
> the [FWS] before the "!".

I certainly do NOT need our ersatz-comma "!" at the begin of
a folded line.  If a server wishes to fold the Path it can do
this after his <path-entry> terminated by ersatz-comma "!".

If you insist on folding before "!" I take it you've given up
on the "!!" shorthand for a "!.MATCH!".  Then shifting [FWS]
resulting in "!.MATCH" [FWS] "!" is no big issue, see above.

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 19 Jan 2006 14:12:36 GMT

>setting up their service from scratch right now, they shouldn't use that

Who knows? There may well be situations in the future where use of a
bareword is right and proper - usually when the site in question does not
have a domain name because, for example, it is using UUCP, or carrier
pigeons, or whatever else (note that Usenet is not tied to the Internet,
it just happens that the Internet is currently the most convenient medium
for transporting it).

OTOH, there is already wording in USEPRO that makes it clear that, when a
site is choosing an identity for itself, a domain-name is the preferred
choice and barewords should only be used where there is a good reason.

>have a choice, don't do this in the future."

No, my understanding of "legacy" is that it is only for continuation of
some existing usage, and is not to be used in new situations. That is
going too far in this case where we actually mean, at most, "SHOULD NOT use
in future".

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 19 Jan 2006 14:01:23 GMT

>going to M, but doing nothing about the fact that the attck articles won't 

Yes, but that problem has always been around, with not much to be done
about it. And none of the proposed notations will prevent it.

>If C thinks that A is still called A, he will send him the second, but not 

OTOH, if B was mistaken in supposing it truly (for some value of "truly")
came from A, then C will not be misled into witholding the article from A
(who in reality had nothing to do with AA). Therefore, the 2nd notation is
safer. Observe that sending an article to A unnecessarily does no more
than waste a little bandwidth, which is no problem provided the extent of
such mistakes is small.

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 19 Jan 2006 14:41:24 GMT

>  Rules for construction of a path-diagnostic are given in [USEPRO]

No, that is not right because the basic idea that we have been discussing
is that each diagnostic should not only be distinguishable from a
<path-identity>, but should also always be attached to an explanation (i.e. a
keyword) of what it is trying to diagnose (though the exact syntactic
sugar of the attachment is under debate). I see that Frank also seems to
be holding this view.

So if you want to stick to the minimum is USEFOR and leave the actual
mechanisms to USEPRO, then I would suggest something like the following:

path-element = path-identity / path-diagnostic

path-identity = label 1*( "." label ) ; which allows IpV4 addresses for
                                      ; legacy use only with MUSTard
                                      ; against future use
                / bareword ; with suitable syntax for bareword
path-diagnostic = [ path-identity / IpV4address / IpV6address ]
                    ".." keyword
keyword = 1*( ALPHA / DIGIT ) ; or somesuch

and leave it to USEPRO to define whatever <keyword>s were deemed
necessary, with instructions on how/when to generate them, and with a
clear statement of whether/how other keywords could be introduced (e.g.
free use of <keyword>s beginning with "X").

That assumes we don't want diagnostics to be "visible" (we seem to be
coming round to that view). So that would allow stuff like:

Path: site-a!!site-b!site-m..MISMATCH!site-x!123.234.123.234..SEEN!
      site-c!..POSTED!xyzzy!not-for-mail

So anything consisting of or ending with "..<keyword>" is a diagnostic,
everything else is a <path-identity>, and the dead:beef situation only
arises on account of IpV6addresses, and so all-hex <bareword>s are the
only ones that need be avoided.

---

From: Richard Clayton <[email-masked]>
Date: Fri, 20 Jan 2006 13:56:35 +0000

In message <[email-masked]>, Russ Allbery
<[email-masked]> writes
>setting up their service from scratch right now, they shouldn't use that

I have no problem with that view ... and the wording about such things
supports that [which I thought was settled, despite Frank wanting to
reopen the issue in another message :(]

What I don't want to see happening is the users of UUCP names (and there
are a number besides "demon") ending up infringing a SHOULD NOT.

Describing what they are doing as "legacy" doesn't put them in a
position of arguing with barrack-room lawyers about "not conforming to
standards" -- and it is precisely that which I wish to see avoided.

>have a choice, don't do this in the future."

me too :)

- -- 
richard                                                   Richard Clayton

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 20 Jan 2006 17:03:49 +0100

Charles Lindsey wrote:

> There may well be situations in the future where use of a
> bareword is right and proper - usually when the site in
> question does not have a domain name because, for example,
> it is using UUCP, or carrier pigeons

Then it will solve this hypothetical problem in a proper way,
like publish an RfC how its namespace is supposed to work in
the case of conflicts, and use say "what.ever.invalid" until
that's clear.  What it can't do is simply use "demon.cat" in
TLD cat against the will of the owner of domain "demon.cat".

> note that Usenet is not tied to the Internet

Other nets have their own rules.  A site name "2:24/0" IIRC
won't fly in FTSC nets if it's not related to the region 24
coordinator in zone 2 of FidoNet.

> it just happens that the Internet is currently the most
> convenient medium for transporting it

IBTD, that's not only happenstance.  You could use "proper"
names in in the e164.arpa zone for a notation of phone numbers
compatibile with DNS.  But you MUST NOT abuse phone numbers
without the consent of the owner, that's just what "right and
proper" means.

>> legacy is "people who have been doing this may want to
>> continue for various reasons, but if you have a choice,
>> don't do this in the future."

> No, my understanding of "legacy" is that it is only for
> continuation of some existing usage, and is not to be used
> in new situations

For me "SHOULD NOT" means that you need a _good_ reason if you
violate it, it can have harmful and undesirable side-effects.

As an example it might be no good idea if a major site decides
to pick "demon" as its name.  OTOH another site that already
uses this name has plausible reasons to continue this practice.

We had this perfectly clear for some months:  Anything with a
dot is FQDN or at worst IPv4, and all names without dots are
legacy.  Potential conflicts with new TLDs avoided by adopting
the 2821 "at least one dot" rule.

But apparently that's not good enough, and we have to allow
legacy names with dots.  Destroying our masterplan to avoid
conflicts between legal and legacy names by looking for a dot.

There's no way I know of how "demon" could produce a world map
entry for this name in 2006, and without this it hits a MUST
in s-o-1036.  You said "add MUSTard", I didn't want this, as
far as I'm concerned "path-legacy" is clear enough.

But if you want it in prose then yes, "legacy" means obviously
SHOULD NOT.  That's no question of "barrack lawyers", that's a
question of name spaces, potential abuse, conflicts, and harm:

| a relayer name MUST be either a UUCP name registered in the
| UUCP maps (without any domain suffix such as ".UUCP"), or a
| complete Internet domain name.
                                Bye

---

From: Russ Allbery <[email-masked]>
Date: Fri, 20 Jan 2006 11:26:59 -0800

> are a number besides "demon") ending up infringing a SHOULD NOT.

Right, I agree with this.  "demon" shouldn't be a general SHOULD NOT.  At
most, the document could say something like "SHOULD NOT be used unless was
already in use prior to this standard" (in some way that's less clunky).

---

From: Russ Allbery <[email-masked]>
Date: Fri, 20 Jan 2006 11:47:06 -0800

Charles Lindsey <[email-masked]> writes:

> bareword is right and proper - usually when the site in question does
> not have a domain name because, for example, it is using UUCP, or
> carrier pigeons, or whatever else (note that Usenet is not tied to the
> Internet, it just happens that the Internet is currently the most
> convenient medium for transporting it).

I'm quite comfortable with requiring domain names going forward, and I
think that's the right thing to do for any new site.  I think it's highly
unlikely that it will ever again be worthwhile to use an undifferentiated
flat namespace for names.

> going too far in this case where we actually mean, at most, "SHOULD NOT
> use in future".

Whatever distinction you're drawing here is too subtle for me, as both
sentences above sound to me like they're saying the same thing.  If
anything, the second sentence is harsher and too strong; what we want to
say is "SHOULD NOT use in new installations".

---

From: Russ Allbery <[email-masked]>
Date: Fri, 20 Jan 2006 11:48:32 -0800

Charles Lindsey <[email-masked]> writes:

> OTOH, if B was mistaken in supposing it truly (for some value of
> "truly") came from A, then C will not be misled into witholding the
> article from A (who in reality had nothing to do with AA). Therefore,
> the 2nd notation is safer. Observe that sending an article to A
> unnecessarily does no more than waste a little bandwidth, which is no
> problem provided the extent of such mistakes is small.

Yeah, after thinking this over, I agree with Charles.

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 21 Jan 2006 00:02:44 +0100

Russ Allbery wrote:

> after thinking this over, I agree with Charles.

Just to be sure, that's "invisible", as in my first guess what
Harald's dots are supposed to do, hide the complete diagnostics
in B!.MISMATCH.A!AA

Or do you here only agree for MISMATCH, but not B!.SEEN.A!AA ?

For B!.POSTED!A!AA I hope we have it clear that A!AA is either
what B got, or B got nothing constructing whatever!not-for-mail
(i.e. A = whatever path-identity, AA = not-for-mail).

In other words no case of B adding !.POSTED!A! to a given AA,
also no !.POSTED.A! added to a given AA.

In that line of the discussion the following ABNF should work,
it has among others these features:

1: legal "demon.cat" beats legacy "demon.cat"
2: no legal "demon", any "demon" is legacy (dito cat, aero, etc.)
3: no legal IP-literal, any IPv4 is legacy or <path-diagnostic>
4: no IPv6 outside of <path-diagnostic>
5: Charles's preferred position of the one and only [FWS]
6: remove the !.MATCH and <path-verified> if it's useless

                       Bye, Frank

path            =  "Path:" SP *WSP path-list tail-entry *WSP CRLF
path-list       =  *( path-entry [FWS] "!" )
path-entry      =  path-identity / path-verified / path-unverified /
                   path-mismatch / path-posted

path-verified   =  path-identity "!.MATCH"
path-unverified =  path-identity "!.SEEN."     path-diagnostic
path-mismatch   =  path-identity "!.MISMATCH." path-diagnostic
path-posted     =  path-identity "!.POSTED"

---

From: Russ Allbery <[email-masked]>
Date: Fri, 20 Jan 2006 15:22:38 -0800

> Or do you here only agree for MISMATCH, but not B!.SEEN.A!AA ?

Right, and I agree for all of the diagnostics, including SEEN.

> (i.e. A = whatever path-identity, AA = not-for-mail).

If B receives a post with existing Path content A!AA from host H, my
understanding is that it should be constructing a Path header saying
either:

    B!.POSTED!A!AA

or

    B!.POSTED.H!A!AA

The latter form is probably pointless since it can put that data into
Injection-Info, though.  In either case, the existing Path content should
not be hidden, I think.

> In other words no case of B adding !.POSTED!A! to a given AA,

Right.

> also no !.POSTED.A! added to a given AA.

We could do this, but I don't see a lot of point.

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 23 Jan 2006 12:11:35 GMT

>me too :)

But it seems not everyone understands the word "legacy" in that way. For
example, Frank seems to think it is equivalent to SHOULD NOT, which
appears to be further than you want to go.

Seems to me that "legacy" is a word we should avoid using, and we should
make our intentions as to future use clear by explicit verbiage (either
here or in USEPRO).

-

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 23 Jan 2006 12:11:35 GMT

-

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 23 Jan 2006 14:35:43 +0100

Charles Lindsey wrote:

>>> Legacy is not the same thing as deprecated syntax.

Yes, e.g. I'm using a "legacy" browser (pre HTML 4 + CSS) with
"legacy" charsets (not Unicode) and "legacy" SSL3 (no TLS 1.1).
So far nothing's wrong with that.

> But it seems not everyone understands the word "legacy" in
> that way. For example, Frank seems to think it is equivalent
> to SHOULD NOT, which appears to be further than you want to
> go.

New browsers SHOULD NOT only support [see above], only old
browsers have an excuse.

> Seems to me that "legacy" is a word we should avoid using,

IBTD, it's perfectly clear.  While we do and can protect a
legacy path-identity like "demon", we cannot guarantee this
protection for "demon.cat" (anything with a dot) or "feed"
(hex. digits).

Therefore it's our duty to say that this might not work as
expected, and while we're at it it's also clear that "demon"
is not protected from other systems claiming to be "demon".

All we can guarantee is that a fully qualified host name
with at least one dot can't cause trouble for the legacy
"demon".

> we should make our intentions as to future use clear by
> explicit verbiage (either here or in USEPRO).

It's no question of _future_ use, any "demon.cat" unrelated
to TLD cat (now online for a few days) would be in trouble,
no matter how old it is.

This is the same discussion as with <id-domain> in <msg-id>.

A namespace is a namespace, if you want a "better" solution
let IANA start a name-registry, if they can do dictionaries
they can also do UUCP maps.
                            Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 23 Jan 2006 12:28:02 GMT

>already in use prior to this standard" (in some way that's less clunky).

Well the text in USEPRO (which I thought we had agreed at least as far as
this particular bit is concerned, though there are other issues with the
precise wording) is:

   <Path-identity>s can take the following forms (in decreasing order of
   preference):

   1. A fully qualified domain name (FQDN)....

   2. Some other (arbitrary) name believed to be unique and registered
      at least with all other news servers sending articles directly to
      the given one. This option SHOULD NOT be used unless the earlier
      option is unavailable (e.g. because the server in question is not
      connected to the Internet), or unless it is of longstanding usage
      and cessation would be unduly disruptive, or unless the earlier
      option is provided as well.

Note that <bareword>s (which is the term I prefer to "legacy") also turn
up in various specialized usages such as 'cybercancel'. Other specialized
needs may appear in the future.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9  From [email-masked] Mon Jan 23 12:15:07 2006
To: [email-masked]
Xref: clerew local.usefor:23009
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: path-legacy (bareword) and toplabel in A ~ B
Date: Mon, 23 Jan 2006 12:28:02 GMT
Lines: 37
Sender: [email-masked]

>already in use prior to this standard" (in some way that's less clunky).

Well the text in USEPRO (which I thought we had agreed at least as far as
this particular bit is concerned, though there are other issues with the
precise wording) is:

   <Path-identity>s can take the following forms (in decreasing order of
   preference):

   1. A fully qualified domain name (FQDN)....

   2. Some other (arbitrary) name believed to be unique and registered
      at least with all other news servers sending articles directly to
      the given one. This option SHOULD NOT be used unless the earlier
      option is unavailable (e.g. because the server in question is not
      connected to the Internet), or unless it is of longstanding usage
      and cessation would be unduly disruptive, or unless the earlier
      option is provided as well.

Note that <bareword>s (which is the term I prefer to "legacy") also turn
up in various specialized usages such as 'cybercancel'. Other specialized
needs may appear in the future.

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 23 Jan 2006 15:34:33 GMT

>also no !.POSTED.A! added to a given AA.

OK, I think we are converging on a possible solution, but your syntactic
sugar is all wrong. We already have some usage of
!news.example.com.MISMATCH!, so we need a notation that is immediately
recognisable to people who are used to that. Hence my proposal for

B!A..MISMATCH!AA and B!..POSTED!A!AA

for your two examples above plus

B!A..MATCH!A!AA, or more likely just B!!A!AA.

>it has among others these features:

But there are several things I don't like about your syntax, including the
lack of "!!".

>path-legacy     =  *( path-component "." ) path-component

Why on earth do you want to permit a <path-legacy> with both a "." and a
"_" in it? Can anyone point me to a significant quantity of such beasties
out there in the real world?

Here is the syntax I would recommend if we follow this sort of route:

   path = "Path" ":" SP [FWS] *( path-entry [FWS] ( "!" / "!!" ) )
             tail-entry [FWS] CRLF
   path-entry = path-identity / path-diagnostic
   path-identity = fqdn / bareword
   fqdn = 1*( label "." ) label ; subject to restriction below
   label = alphanum / alphanum *( alphanum / "-" ) alphanum
   bareword = ALPHA / ALPHA *( alphanum / [ "-" / "_" ] ) alphanum )
   path-diagnostic = [ path-identity / IP-address ] ".." keyword
   IP-address = IPv4address / IPv6address ;  see [RFC 3986]
   keyword = ; see below
   tail-entry = bareword

   ALthough the above syntax permits an IPv4address to appear as a
   <path-identity>, this is only provided to accomodate some existing
   usage (usually of a diagnostice nature). Whilst it MUST still be
   accepted, it MUST/SHOULD not be generated in future (although an
   IPv4address remains acceptable in its own right within a
   <path-diagnostic>).

For the syntax of <keyword>, you either give an explicit list (MATCH,
MISMATCH, SEEN, POSTED, ...), or else you give a generic syntax (1*ALPHA
or somesuch). In either case, you point to USEPRO for the gory details of
when/how to generate.

You also point to USEPRO for when to use/not use <bareword>s and when to
use/not use "!!" (although the semanics of "!!" could be mentioned here,
as in the present draft).

---

From: Russ Allbery <[email-masked]>
Date: Mon, 23 Jan 2006 09:19:57 -0800

> B!A..MISMATCH!AA and B!..POSTED!A!AA

I really don't think that !.MISMATCH.news.example.com! is going to stress
anyone's brain to figure out.  I think it's a lot cleaner to put the token
in front rather than doubling the period; people are too likely to omit
the second period because it looks like a typo.

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 23 Jan 2006 20:21:49 +0100

Charles Lindsey wrote:
 
> I think we are converging on a possible solution

No, I don't think so, quite the contrary.

> your syntactic sugar is all wrong

It's my way to express what I think the concepts are.

> We already have some usage of
> !news.example.com.MISMATCH!

Yes, that's flat out, no 2606bis only for USEFOR and
obscure keyword pseudo-TLDs.

> B!A..MISMATCH!AA and B!..POSTED!A!AA

I like Harald's _leading_ dot better, because that's
more obvious (in conjunction with a leading keyword),
but maybe that's a matter of taste.

A quick reality check nslookup .a.de / nslookup a..de
has apparently the same (error) result.

> B!A..MATCH!A!AA, or more likely just B!!A!AA.

The latter won't fly for your "[FWS] before last !".
 
> several things I don't like about your syntax,
> including the lack of "!!".

That lack is a side-effect of your [FWS] preference.
With "! like comma" the "!!" is of course elegant.

If we ignore that IIRC Richard saw a few bogon "!!",
where the unclear meaning was definitely not "MATCH".

> Why on earth do you want to permit a <path-legacy>
> with both a "." and a "_" in it?

Russ wants this.  And if <tail-entry> is some kind
of syntactically restricted local part, we might
need it anyway.  <path-legacy> is a legacy catch-all.

> path = "Path" ":" SP [FWS] *( path-entry [FWS] ( "!" / "!!" ) )

The first {FWS] violates MUSTard, it should be *WSP.
Dito the last [FWS].  

> path-entry = path-identity / path-diagnostic

You've split "MATCH" from the rest of the diagnostic
zoo, that doesn't reflect the "!!" semantic.

A <path-entry> should be what a site adds to the path,
one record consisting of identity and maybe dianostic.  

It does not add arbitrary sequences of identity and
diagnostic, it adds precisely one identity followed
by zero or one diagnostic.  And if the record ends
with "!!" it's a MATCH.  You can't have "!!" behind
any other diagnostic like MISMATCH / POSTED / SEEN.
 
>    fqdn = 1*( label "." ) label ; subject to restriction below

Without <toplabel> that matches <IPv4address>, but
we have some kind of consensus for "no IP".

>    bareword = ALPHA / ALPHA *( alphanum / [ "-" / "_" ] ) alphanum )

Russ wanted "-" and "_" everywhere in <path-legacy>,
Your <bareword> also cannot start with a DIGIT.

>    IP-address = IPv4address / IPv6address ;  see [RFC 3986]

That's a <tail-entry> without dots.  This does
nowhere reflect what we talked about for weeks.

                     Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 24 Jan 2006 16:12:02 GMT

>but maybe that's a matter of taste.

Indeed a matter of taste. If the majority wants it at the beginning, it is
easily done. But I would still prefer two dots, even at the beginning,
just to make it stand out more for human readers (anyone foolish enough to
read headers in a proportional font will have trouble spotting it).

>The latter won't fly for your "[FWS] before last !".

That is just a matter of writing the correct syntax, so that [FWS] just
never happens in between the two '!'s.

>Russ wants this.

I do not recall that. Can you point me to where he said it? If it is
common in present usage, then indeed we have to accept it (but not
generate, of course). But some Real World examples would settle the
matter.

>  And if <tail-entry> is some kind
>need it anyway.

Yes, I have no problem if people want to allow more complicated things in
the <tail-entry>, such as a full <path-identity>.

>The first {FWS] violates MUSTard, it should be *WSP.

No, that is a separate issue for which there is a separate thread.

>zoo, that doesn't reflect the "!!" semantic.

Eh? "MATCH" (if we have it) is a keyword. "!!" needs to appear in the syntax
in its own right (to ensure there is no [FWS] inside it). That means it is
probably not a keyword syntactically speaking, even though its semantics
maybe the the same as a keyword called "MATCH". But USEPRO is the place to
tell you when to use or not use it.

>one record consisting of identity and maybe dianostic.

Again, that can be enforced in USEPRO. Our Chair seems to want to keep the
syntax here as simple as we can get away with (and we don't want agents to
be refusing articles just because they mistakenly have two diagnostics in
succession). That is why I removed that minor complication from what I had
in my earlier syntaxes. It can go back if people want it so.

>any other diagnostic like MISMATCH / POSTED / SEEN.

As I said, if something turns up that USEPRO says should NEVER have been
generated, then nothing in the protocol actually breaks, though someone
who inspects the line manually might well be confused.

>we have some kind of consensus for "no IP".

Yes, but you have carefully snipped my wording further down, where I said
that an IPv4address there was a "MUST accept, MUST/SHOULD NOT generate".

>Your <bareword> also cannot start with a DIGIT.

I think that is what we agreed a few weeks back when <bareword>s were last
discussed.

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 24 Jan 2006 15:39:29 GMT

>the second period because it looks like a typo.

If that is the majority view, then OK. But I still think a single "." at
the beginning is easily overlooked by the human reader. How about a double
dot, just to make it more conspicuous as in
       "!..MISMATCH.news.example.com!"?

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 24 Jan 2006 16:27:15 GMT

>IBTD, it's perfectly clear.

Not if it implies "SHOULD NOT", because Richard has specifically asked
that Demon should not be in any violation of a SHOULD NOT.

>is not protected from other systems claiming to be "demon".

Yes, we do say that, and it is made clear that people use these barewords
at their own risk. But they have been doing just that for years, and the
sky has not fallen in yet.

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Tue, 24 Jan 2006 13:10:22 -0500

>        "!..MISMATCH.news.example.com!"?

I think two dots is easily overlooked, especially in a proportional font
set to show all 900 characters on one line.  3 dots
might be good, but might be confusing because it looks like an ellipsis,
and some gateway might do inappropriate Unicoding of that into some other code 
point entirely.

Plus if there is a data channel bit error, the dot might turn into something 
else entirely.

So,

If we make it 10 dots, there is a sufficient level of redundancy that bit-error
rates on lossy channels will not confuse anyone, and future I18N of
domain names is extremely unlikely to use a sequence of 10 dots for
overloading.  And it isn't a multiple of 3, so that avoids some ellipsis
recoding problems.  Hmm, maybe 11 is better because that is 3n+2, so there
are at least 2 dots leftover.

I think we need a poll with choices of 1 to 10, and then take
the average number.  That would be most democratic, and a loud 10 is weighted
in the average 5 times better than a vote for 2.

---

From: Richard Clayton <[email-masked]>
Date: Tue, 24 Jan 2006 20:22:24 +0000

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes
>the beginning is easily overlooked by the human reader.

no-one reads this stuff unless they are paranoid about forgery...

... if so then they will pay attention

> How about a double

people pay money for their bandwidth -- it adds nothing :(

- -- 
richard @ highwayman . com                       "Nothing seems the same
                          Still you never see the change from day to day
                                And no-one notices the customs slip away"

---

From: Seth Breidbart <[email-masked]>
Date: Tue, 24 Jan 2006 16:27:44 -0500 (EST)

> that Demon should not be in any violation of a SHOULD NOT.

I think "legacy" means "It's OK to continue if you're already doing
it, but you SHOULD NOT start doing it."  So Demon isn't violating a
SHOULD NOT because they've been doing it long enough.

So I think "legacy" is the correct term.

Seth

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Tue, 24 Jan 2006 16:43:15 -0500

> So I think "legacy" is the correct term.

+1

And 'SHOULD NOT' and 'MUST NOT' are not the only words we should use
to communicate reality to a cooperative implementor.

We've already been through all the flame wars here about that.  EVERY 
implementor that reads an IETF document is a voluntary, cooperative
implementor....

Especially with Usenet, which apparently exists for the express purpose
of lawless symbiosis, IETF standards are not effective cluebats.

No imperatives needed here for legacy, if plain words will do.

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 24 Jan 2006 23:13:22 +0100

> Not if it implies "SHOULD NOT", because Richard has
> specifically asked that Demon should not be in any
> violation of a SHOULD NOT.

Then propose some text to establish a IANA registry replacing
the defunct UUCP maps for "demon", and propose ABNF reflecting
this concept.  Otherwise it's in conflict with a MUST about
relayer names in son-of-1036.

> it is made clear that people use these barewords at their
> own risk.

This is NOT clear in your ABNF.  Please accept that this is
"legacy" that won't work as expected in the case of conflicts
or trivial tasks like "try to add news@ to get an address".

With a path ...!what.ever example!demon!...  it's dubious that
an address <[email-masked]> works.  This is now
ancient history.  We all love it, that's why we are here, but
we should now also admit that it's "legacy" and move on.  Bye

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 25 Jan 2006 02:01:07 +0100

Charles Lindsey wrote:

>>> more likely just B!!A!AA.

> That is just a matter of writing the correct syntax,
> so that [FWS] just never happens in between the two '!'s.

Nobody managed to propose acceptable syntax for this detail.

You either arrive at "!!" [FWS] with a folding always at the
end of an "identity plus optional diagnostics" <path-entry>,

Or you arrive at "!.MATCH" {FWS] "!" with a folding always
before the terminating "!" of this <path-entry> record.

There's no third option to get this right if "!" [FWS] "!"
is undesirable, and if "!!" must not be used in conjunction
with other diagnostics.

We don't want ...!x!.POSTED!!y!not-for-mail (or similar).
We don't want ...!x!.MISMATCH.z!!y!... (dito for any SEEN)
We don't want ...!x!.POSTED!.POSTED!y!not-for-mail
We don't want ..,!x!.MISMATCH.a!.MISMATCH.b!y...
We don't want ...!x!.MISMATCH.a!!.POSTED!y!not-formail

Your proposals make all these cases syntactically correct.
Not only Bruce already didn't like this months ago.

>>> Why on earth do you want to permit a <path-legacy> with

> I do not recall that. Can you point me to where he said it?

<http://article.gmane.org/gmane.ietf.usenet.format/30442>
<http://article.gmane.org/gmane.ietf.usenet.format/30459>

> If it is common in present usage, then indeed we have to
> accept it (but not generate, of course). But some Real
> World examples would settle the matter.

As far as dots in the <tail-entry> are concerned all I can
offer is "I use addresses with dots in their local part".

Somebody found cases of IPv4 in the path, mostly near the
<tail-entry>, or probably some case of "old-path-diagnostic".

Any "old-path-diagnostic" is a proper subset of the catch
all <path-legacy> construct.  Harald has just confirmed that
we don't want IPs in "regular" path-identities, i.e. neither
diagnostics nor legacy.

> we don't want agents to be refusing articles just because
> they mistakenly have two diagnostics in succession

I certainly want this to be a clear syntax error for a "new"
<path-diagnostic>.  That's already dubious as it is, relax
the syntax, and it's garbage.  Everybody insisting on adding
garbage to the path can use <path-legacy> for this purpose.

If a relaying agent anywhere on the world checks the syntax
of the path before relaying articles it missed some details
in USEPRO - IIRC relaying agents are not supposed to do this.

> Yes, but you have carefully snipped my wording further down,
> where I said that an IPv4address there was a "MUST accept,
> MUST/SHOULD NOT generate".

I don't care much about most MUSTard in the prose, I want the
ABNF to be clear.  Or as somebody else put it:

 <http://permalink.gmane.org/gmane.org.w3c.public-iri/122>
| I have sufficient experience with "Grammar says this but spec
| says that"-style discussions that I avoid them as much as
| possible.

This is our best chance to get it right and avoid such useless
discussions in the future.

> I think that is what we agreed a few weeks back when
> <bareword>s were last discussed.

I don't recall that, and I found nothing about it in an old
grizzly book.  With <path-legacy> as catch-all it's no issue,
why should "123abc" be worse than say "demon" or "a__123" ?

BTW, it would be simple to integrate your idea of "multiple
path identities" into the proposed _unfoldable_ <path-entry>:

~~~ old ~~~
path-entry      =  path-identity / path-verified / path-unverified /
                   path-mismatch / path-posted
~~~ new ~~~
path-entry      =  *( path-identity "!" ) ( path-identity /
                      path-verified / path-unverified /
                      path-mismatch / path-posted )
~~~ bye ~~~

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 25 Jan 2006 11:18:54 GMT

>This is NOT clear in your ABNF.

It was not meant to be clear in the ABNF (nor is there any way to do so).
The wording is in USEPRO, because this is a matter of "how" to use the
facilities provided, not a matter of what can legitimately be present
(which is what USEFOR is supposed to be about).

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 25 Jan 2006 11:14:48 GMT

>So I think "legacy" is the correct term.

There is a bunch of stuff out there that we MUST continue to accept, so
things don't suddenly stop working, and so that parsers need not feel
obliged to spend time checking for every conceivable syntactic oddity.

Out of that, there is a sub-bunch of stuff that we are happy to see
generated in the future, both for old sites and for new sites (FQDNs are
the obvious example, but not if they have "_"s in them).

The question at issue is whether "barewords" are in this latter category,
subject to recommendations not to use them without good cause, and
warnings to choose them carefully because there is no global register to
ensure uniqueness.

IMO they _should_ be in that latter category and until this last week,
when the word "legacy" suddenly started to be used to describe them, that
was the position we had agreed in this WG. In particular, the wording
containing those recommendations and warnings, and which is now in USEPRO
(as quoted in full a couple of days ago in response to Russ) was discussed
in this WG months ago and seemed to be agreed (for example, it was still
present in a proposed text submitted by Harald, even though an issue
remained with related wording regarding FQDNs).

So by all means review our previous policy if you want to, but do not
bring in changes by the back door by introducing words such as "legacy"
for the names of syntactic objects.

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 26 Jan 2006 13:12:47 GMT

>Nobody managed to propose acceptable syntax for this detail.

I published such a syntax six or so weeks ago. I give another such syntax
below.

>is undesirable, and if "!!" must not be used in conjunction

There is a balance to be struck between putting extra complexity into the
syntax, or having a simpler syntax with MUSTard to remove the unwanted
cases.

>We don't want ...!x!.MISMATCH.a!!.POSTED!y!not-formail

All of those examples are stupid, but none of them would actually do any
harm if it slipped through. The instructions in USEPRO telling you how to
construct the Path header would not produce any of them.

><http://article.gmane.org/gmane.ietf.usenet.format/30459>

I have reread both of those, and they do not seem to say what you imply.
The first just seems to be saying that if you allowed dots into a
bareword, then that would permit some non-valid FQDNs. It does not say
that such things need to be accomodated.

The second is commenting on your syntax which introduced such dots, and
seems to be saying that at least it includes all the things currently seem
out there.

However, Russ can easily clarify what he meant.

Russ: Habe you ever seen things like !news.foo_bar.example! out there in
the real world, and often enough that we need to take account of them?

>offer is "I use addresses with dots in their local part".

In your tail entries? I agree that the syntax of the tail-entry is pretty
unimportant, but it is a minor detail compared to the rest of the syntax.

Anyway, if you want a syntax that leaves as little as possible to
verbiage, then here it is. Personally, I would prefer something shorter
that corresponded to the MUST accept stuff, and left what was to be
generated to MUSTard, either here or in USEPRO.

   path = "Path" ":" SP [FWS]
             *( path-identity [FWS] "!" path-diagnostic )
             tail-entry [FWS] CRLF 

[The 1st & last [FWS] might become *WSP if the Issue on that topic
succeeds] 
 
   path-diagnostic = [ "!" ]
                       [ "." keyword [ diagnostic-identity ] [FWS] "!" ]

   keyword = ( "SEEN" / "MATCH" / "MISMATCH" / "POSTED" ) 

[or maybe keyword = 1*ALPHA and let USEPRO introduce those that are
needed] 
 
   path-identity = fqdn / bareword / obs-path-identity 

   obs-path-identity = IPv4address ; and any other cases thought necessary

   fqdn = 1*( label "." ) toplabel

   toplabel = [ label *( "-" ) ] ALPHA [ label *( "-" ) ]
              / label 1*( "-" ) label    ; at least one non-digit

   alphanum = ALPHA / DIGIT

   tail-entry = path-identity

[how about tail-entry = 1*( alphanum / "-" / "_" / "." ) ?]

   diagnostic-identity = fqdn / IP-address / bareword

Some notes on that:

If you look at the <path-diagnostic>, you can't have more than one of them
at a time, and the syntax allows them to be completely empty (the usual
case).

They may start with a "!" (giving you a "!!" with no FWS inside it). The
notation is to use a leading "." before the <keyword> (which seems now to
be the consensus), and the keyword may or may not be followed by an
identity (that is for the benefit of ".POSTED").

I have introduced <obs-path-identity>, which can grow to accomodate any
further cases deemed necessary. We can argue as to whether <bareword>
belongs in <obs-path-identity> or not.

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 26 Jan 2006 20:02:45 +0100

Charles Lindsey wrote:

> by all means review our previous policy if you want to, but
> do not bring in changes by the back door by introducing words
> such as "legacy" for the names of syntactic objects.

If you really think that an _explicit_ SHOULD NOT in the prose
about the <bareword> in
<http://permalink.gmane.org/gmane.ietf.usenet.format/30548>
or about the <path-nodot> in
<http://permalink.gmane.org/gmane.ietf.usenet.format/30547>
is "better" than simply saying <path-legacy> without any
2119 prose, then that's a bit odd, but otherwise acceptable.

                              Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 26 Jan 2006 21:30:28 +0100

Charles Lindsey wrote:

> There is a balance to be struck between putting extra
> complexity into the syntax, or having a simpler syntax
> with MUSTard to remove the unwanted cases.

Yes, I prefer KISS without MUSTard where possible.

> the syntax of the tail-entry is pretty unimportant, but it
> is a minor detail compared to the rest of the syntax.

[... quote rearranged ...]

> tail-entry = path-identity

My last proposal - <path-nodot> is a liberal <bareword> - was:

| path-nodot      =  1*( alphanum / "-" / "_" )
|
| tail-entry      =  *( path-nodot "." ) path-nodot

Pretty similar, your version matches less obscure tail-entries.

> [how about tail-entry = 1*( alphanum / "-" / "_" / "." ) ?]

That would match more than the <path-nodot> construct, it can
have adjacent dots.  Allowing "!..." might be a dubious idea.
An unquoted "..." is even no 2822 <dot-atom-text> or <id-left>.

[...]
>                        [ "." keyword [ diagnostic-identity ] [FWS] "!" ]

That allows  y!!.SEEN.u!x  or  y!!.POSTED!x  with a bogus "!!".

And "we" want at most one [FWS] per <path-element>, you have
two.  IIRC "limit [FWS]" was originally your idea.

>    keyword = ( "SEEN" / "MATCH" / "MISMATCH" / "POSTED" )

That allows  y!.MATCH.x!x  (and  y!!.MATCH.x!x ), that's wrong,
"we" want  y!!x  for this purpose.

Or at most  y!.MATCH!x  in a scheme for "[FWS] before "!" where
"!!" doesn't work.

It also allows  y!.POSTED.u!x  (and  Y!!.POSTED.u!x ), but "we"
don't want a <diagnostic-identity> with .POSTED or .MATCH

>    obs-path-identity = IPv4address ; and any other cases thought necessary

Apparently Harald wants the IPv4 case firmly buried in <o-p-d>
and removed from <path-identity>.  I renamed his <o-p-d> to
<diag-deprecated> and used the same trick as for <tail-entry>:

| diag-deprecated =  "!" *( path-nodot "." ) path-nodot

That matches almost any crap (incl. IPv4) as <diag-deprecated>.

>    label = alphanum / alphanum *( alphanum / "-" ) alphanum

That doesn't have the 4234 recommended grouping.  My proposal:

| label           =  alphanum [ *( alphanum / "-" ) alphanum

>               / label 1*( "-" ) label    ; at least one non-digit

The same 4234 grouping issue for the hyphen-alternative.  And
it's the "old" toplabel idea allowing a singleton ALPHA as TLD.

Maybe it's ridiculous, but I fear that whatever we pick here
could sooner or later end up as 2606bis / ICANN policy for the
whole world, and in that case I'd prefer "at least two char.s".

>    bareword = ALPHA / ALPHA *( alphanum / [ "-" / "_" ] ) alphanum )

More restrictive than <path-nodot> (and missing 4234 grouping).
Can the legacy names really never begin with DIGIT / "-" / "_"
or end with "-" / "_" ?

> diagnostic-identity = fqdn / IP-address / bareword

That won't allow <obs-path-identity> unless it's <IP-address>.
In my proposal any <path-identity> or <IP-address> is allowed:

| diag-seen       =  "!.SEEN."     ( path-identity / diag-literal )
| diag-mismatch   =  "!.MISMATCH." ( path-identity / diag-literal )

> If you look at the <path-diagnostic>, you can't have more
> than one of them at a time

Almost, you still have "!!" together with other disagnostics,
and you have <diagnostic-identity> where you don't want them,
i.e. for MATCH (fixed by simply removing it) and POSTED (bug).

> the keyword may or may not be followed by an identity (that
> is for the benefit of ".POSTED").

ACK, what you call "benefit" is what I call "bug".

> I have introduced <obs-path-identity>, which can grow to
> accomodate any further cases deemed necessary.

Yes, but apparently not allowed in <diagnostic-identity>.

> We can argue as to whether <bareword> belongs in
> <obs-path-identity> or not.

This would also affect !.MISMATCH.demon! (allowed as you have
it, also in my version), but not more if we move it to <o-p-i>.

                         Bye, Frank

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Fri, 27 Jan 2006 10:44:36 -0800

I think I'm going to not file this issue.... the practice of ABNF in the 
IETF has been that ambiguous parses are permitted, with no guidance on how 
to indicate how they are resolved.

                 Harald

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 28 Jan 2006 12:41:24 +0100

Harald Tveit Alvestrand wrote:

> the practice of ABNF in the IETF has been that ambiguous
> parses are permitted, with no guidance on how to indicate
> how they are resolved.

The <fields> in -06 are apparently the only case in USEFOR
plus RfC 2822, where a "depth first left to right" parser
won't work.

And STD 66 (one of our sources) has some guidance in 3.2.2:

|    host        = IP-literal / IPv4address / reg-name
|
| The syntax rule for host is ambiguous because it does not
| completely distinguish between an IPv4address and a reg-name.
| In order to disambiguate the syntax, we apply the
| "first-match-wins" algorithm:
| If host matches the rule for IPv4address, then it should be
| considered an IPv4 address literal and not a reg-name.

With that trick they got rid of the dubious 2396 <toplabel>
and could allow other (non-DNS) "name registries".

With  news-fields = <our stuff> / fields  instead of the -06
notation  fields =/ <our stuff>  we'd also make sure that new
header fields for mail can't simply overwrite <our stuff> in
a "first-match-wins" parser.
                             Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 30 Jan 2006 14:21:03 GMT

>| tail-entry      =  *( path-nodot "." ) path-nodot

I think we leave the <tail-entry> until the rest of the syntax is agreed,
and then just build it using whatever tools are then available. Its only
_essential_ property is that there is no "!" in it.

>That allows  y!!.SEEN.u!x  or  y!!.POSTED!x  with a bogus "!!".

Yes, the "y!!.POSTED!x" was intended. If we decide we do not want it, it
is easily removed. And a more complicated syntax could clearly remove
other unintended funnies, but would not be so KISSable.

>two.  IIRC "limit [FWS]" was originally your idea.

Yes, but is preventing a [FWS] between a <path-identity> and a
<path-diagnostic> really worth the complication? Harald seems to be
pressing for a simpler syntax, which is why I removed that "feature". If
the WG really wants it tying down that hard, then it can be put back.

>"we" want  y!!x  for this purpose.

If we are to have a fixed list of keywords, then I am happy to remove
"MATCH" from it (so long as "!!" remains possible).	

>| label           =  alphanum [ *( alphanum / "-" ) alphanum

OK

>it's the "old" toplabel idea allowing a singleton ALPHA as TLD.

OK

>whole world, and in that case I'd prefer "at least two char.s".

Indeed. It's not really our job to define what domain names look like. So
I would much rather just allow the syntax to produce 123.234.345.456 and
then deprecate it with MUSTard (all we are trying to say is "don't use an
IPv4address in here, but just accept it for legacy purposes", so why not
just say that?).

>In my proposal any <path-identity> or <IP-address> is allowed:

Eh? The only thing in my <obs-path-identity> was <IPv4address>. If <o-p-i>
acquires some more stuff, then <diagnostic-identity> might have to be
reviewed.

>i.e. for MATCH (fixed by simply removing it) and POSTED (bug).

It depends whether ...!!.POSTED!... is to be allowed, and on whether you
really want every last detail to be hard coded in (KISS and all that).

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 30 Jan 2006 19:21:46 +0100

Charles Lindsey wrote:

> leave the <tail-entry> until the rest of the syntax is
> agreed, and then just build it using whatever tools are
> then available. Its only _essential_ property is that
> there is no "!" in it.

Yes, other VCHARs including ":" might be also "essential".
Not in my s-o-1036 parallel universe, but maybe for Bruce.

> the "y!!.POSTED!x" was intended

What's the difference from "y!.POSTED!x" ?  AFAIK that's
the first time that you mention this as an _intentional_
feature, no surprise if we arrive at different ABNF.

For an identical list that would be almost impossible as
soon as the list of intended Path:-features is complete.

 [limit [FWS] in the path]
> is preventing a [FWS] between a <path-identity> and a
> <path-diagnostic> really worth the complication?

It's pretty simple, not complex...

> Harald seems to be pressing for a simpler syntax

...not sure about this.  Anything where I see at first
glance what it's supposed to do would be "simple" from
my POV.  One earlier 2005 attempt with something like...

   path-sep = [FWS] 1*2( "!" ) [FWS]

...or similar was horrible, nobody liked it.  IMHO the
concept "!!" as shorthand for a non-existing "!.MATCH!"
made sense.  We could also swap !.MATCH! and !.POSTED!
if the match-case is irrelevant, and then "!!" would be
a shorthand for a non-existing "!.POSTED!".  <shrug />

> I would much rather just allow the syntax to produce
> 123.234.345.456 and then deprecate it with MUSTard

We had some "no IP" rough consensus, therefore you can't
match it as ordinary <path-identity> together with FQDNs.

In theory you could match it before FQDN like STD 66, and
then deprecate it.  Curious minds would immediately start
to [Discuss] or ask questions about IPv6 in this case.

  {diagnostic-identity = fqdn / IP-address / bareword]
[...]
> Eh? The only thing in my <obs-path-identity> was
> <IPv4address>.

| obs-path-identity = IPv4address ; and any other cases thought necessary
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I wanted to point out that it won't work for any other cases
like legacy names with dots.  No issue if we agree that other
cases either don't exist or are irrelevant.

 [general]
> It depends whether ...!!.POSTED!... is to be allowed

Yes.  I vaguely recall that Bruce and I didn't like it, but if
it has a meaning different from !.POSTED! that would be a new
argument.

We could use "!.POSTED-1!" and "!.POSTED-2!" for two versions
of "!.POSTED!", but I've no clue what this difference is, and
therefore I can't propose better names for POSTED-1 / POSTED-2.

                          Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 31 Jan 2006 15:34:44 GMT

>feature, no surprise if we arrive at different ABNF.

OK, on reading more carefully what is written in USEPRO, the case we
actually need (and which has been there all along) is

    ...!news.injector.example!.POSTED!!xyzzy.claranet.de!frank

The injector there is asserting
   a) that it is the injecting agent, and
   b) that it indeed received the article from the site xyzzy.claranet.de.

So I was trying to put the "!!" on the wrong side of the ".POSTED", but it
still amounts to two diagnostics in succession. And there is still some
existing usage of the form:

    ...!news.injector.example!123.234.345.456!not-for-mail

where the "123.234.345.456" is what the injector SAW (and not what the
posting site put there). Under our new system, that would logically be
rendered as

    ...!news.injector.example!.POSTED!.SEEN.123.234.345.456!not-for-mail

On that basis, I have now modified my syntax to:

   path = "Path" ":" SP [FWS]
             *( path-identity [FWS] "!" *( path-diagnostic ) )
             tail-entry [FWS] CRLF

   path-diagnostic = "." keyword [ "." diagnostic-identity ] [FWS] "!"
                     / "!"

which is also somewhat cleaner to follow.

But now we need to decide whether we really need to allow two diagnostics
in succession for this particular usage. One might argue that an injecting
site should be using Injection-Info if it wants to say where it got the
article from. In that case, 
   s/*( path-diagnostic )/[ path-diagnostic ]/
in the above.

>match it as ordinary <path-identity> together with FQDNs.

The only agents that will ever "parse" the Path header in anger are
relaying agents, and all they are ever going to do is to ignore any
folding and whitespace, and then identify the chunks that come between the
"!"s. If any such chunk appears in their 'sys' file, then they will not
forward the article to that site. So if some admin is stupid enough to
put ".POSTED" in his 'sys' file, he will never forward any articles
anywhere :-( .

So the question of whether some IPv4address that appears without any
keyword in front of it is to be classed syntactically as an obsolete
<path-identity> or as a <path-diagnostic> is purely academic. Nobody
cares.

>>> That won't allow <obs-path-identity> unless it's

Indeed. We don't particularly want any <obs-path-identity> in our
<diagnostic-identity>s.

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 31 Jan 2006 19:47:44 +0100

Charles Lindsey wrote:

> on reading more carefully what is written in USEPRO, the case
> we actually need (and which has been there all along) is

> ...!news.injector.example!.POSTED!!xyzzy.claranet.de!frank

Okay, ignoring the question why and since when "we" need this,
let's say that's the 1st case ...!.POSTED-1!xyzzy.claranet.de

> there is still some existing usage of the form:

Yes, any IPv4 is covered by <diag-deprecated> in my ABNF for
Harald's idea.  In an older proposal for Russ' idea it would
be simply a <path-legacy> construct with dots, but you hated
this.  That's the <o-p-d> (Harald) vs. <o-p-i> (you) conflict.

With <o-p-d> and my proposal we'd end up with ...!not-for-mail
as a <path-nodot> case of <path-identity>.  Very misleading if
we don't rename <path-nodot> to <path-legacy>.

> Under our new system, that would logically be rendered as

So that's your 2nd case of POSTED or POSTED-2, resulting in
...!.POSTED-2.123.234.345.456!not-for-mail for this bogus IPv4.

In other words POSTED-1 is a standalone keyword, and POSTED-2
comes with an argument <diagnostic-identity>.

Therefore we don't need two different POSTED-1 and POSTED-2,
one name POSTED with an optional <diagnostic-identity> will do:

diag-match      =  "!"                           ; an additional "!"
diag-posted     =  "!.POSTED" [ "." diag-identity ]
diag-seen       =  "!.SEEN."        diag-identity
diag-mismatch   =  "!.MISMATCH."    diag-identity
diag-deprecated =  "!" 1*( path-nodot "." ) path-nodot

diag-identity   =  path-identity / IPv4address / IPv6adress

Killing my <diag-literal>, your <diag-identity> is IMO clearer.

Ideas about swapping "!.POSTED!" and "!.MATCH!" for "!!" also
obsolete, we need some miracles to get the <path> right without
any "!!" optimization.

> path-diagnostic = "." keyword [ "." diagnostic-identity ] [FWS] "!"

One [FWS] too many.  No "!.POSTED!.SEEN.ip".  No "!.POSTED!!y".

Apparently you want "!.POSTED.ip!" and "!.POSTED!y" for these
two purposes, that's okay.  As far as anything is still okay -
we can always give up and kill the complete <path-diagnostic>.

> now we need to decide whether we really need to allow two
> diagnostics in succession for this particular usage.

No, see above, optional <diag-identity> would be good enough.

> The only agents that will ever "parse" the Path header in
> anger are relaying agents,

Yes, with your ABNF they match IPv4 as <o-p-i>, with my latest
ABNF they get <diag-deprecated> (= Harald's <o-p-d>), and for
Russ' _old_ proposal they would get a <path-legacy> with dots.

Harald's <o-p-d> tries to guarantee that they'd ignore IPv4.

> all they are ever going to do is to ignore any folding and
> whitespace, and then identify the chunks that come between
> the "!"s. If any such chunk appears in their 'sys' file, then
> they will not forward the article to that site. So if some
> admin is stupid enough to put ".POSTED" in his 'sys' file,
> he will never forward any articles anywhere :-( .

Educating stupid admins is the job of USEPRO, let's ignore this
for the moment.  Implementing a parser that won't consider any
<path-diagnostic> (incl. <o-p-d-> matching IPv4) as "chunk" is
the job of Russ' and others.  It's possible to get this right.

Otherwise we shouldn't waste time with this dubious diagnostics
concept.  Or as Richard put it "YARITWCIFD" = "Yet Another Rfc
Invention That Will Confuse Implementors For Decades."

> the question of whether some IPv4address that appears without
> any keyword in front of it is to be classed syntactically as
> an obsolete <path-identity> or as a <path-diagnostic> is
> purely academic. Nobody cares.

We had a poll about it, some of us do care.  They don't want to
allow IPv4.  Maybe because they'd then get in trouble trying to
rule out IPv6.  And I fear (or is that hope ?) that somebody in
the IESG won't let us get away with any "for IPv4 only" stunts.

> We don't particularly want any <obs-path-identity> in our
> <diagnostic-identity>s.

That "we" is dubious, it's the conflict between <o-p-d> and
<o-p-i>.  I've no problem with any legacy "demon" or "IPv4" as
<diag-identity>.

Well, tons of new issues, it's starting to get an interesting
job for Harald, Ken, and Alexey.

                         Bye, Frank