Subject: Path header - suggested text

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 1 Jun 2005 19:54:44 GMT

This is, I hope, the final form of the Path header. It was pretty well
agreed last time, but Alexey asked me to move the material regarding the
form of <path-identity>s from USEFOR to USEPRO, which I have now done,
resulting in a new "Identification of news-servers" section in USEPRO.

There are a few minor fixes required in USEFOR, including a slight syntax
change for <tail-entry> which I suggested here a couple of days ago, the
removal of an unnecessary reference to the <tail-entry> in USEPRO, and a
sentence about <IPv6address>s which did not get moved to USEPRO.

In USEFOR

3.1.6  Path 

   The Path header indicates the route taken by an article since its
   injection into the Netnews system.  Each agent that processes an
   article is required to prepend one (or more) identities to this
   header.  This is primarily to enable relaying agents to avoid sending
   articles to sites already known to have them, in particular the site
   they came from, and additionally to permit tracing the route articles
   take in moving over the network, and for gathering Usenet statistics.

   path            =  "Path:" SP path-list CRLF

   path-list       =  [FWS]
                      *( ( path-identity / path-keyword ) [FWS]
                      path-delimiter [FWS] )
                      tail-entry [FWS]

   path-identity   =  ( ALPHA / DIGIT )
                      *( ALPHA / DIGIT / "-" / "." / ":" / "_" )

   path-keyword    = "POSTED" / "MISMATCH"

|  tail-entry      =  ( ALPHA / DIGIT )
|                     *( ALPHA / DIGIT / "-" / "." / ":" / "_" )
[To avoid any suggestion that the <tail-entry> inherits any of the
semantic properties of a <path-identity> (see USEPRO). Any other syntax
that avoids that problem would do.]

   path-delimiter  =  "!" / "!!"

   Each <path-identity> in the <path-list> (excluding the one in the
   <tail-entry>) indicates, from right to left, the successive agents
   through which the article has passed.  The <keyword> "POSTED"
   indicates that the agent to its left injected the article.  The use
   of the <path-delimiter> "!!" indicates that the agent to its left
   claims that the agent to its right was the verified source of the
   article (whereas the <path-delimiter> "!" implies no such claim).
   The <keyword> "MISMATCH" indicates that the agent to its right failed
   to be so verified.  The full procedure for constructing a Path header
|  as well as the specific format and use of the <path-identity> are
|  discussed in [USEPRO].
[No need to refer to USEPRO for the <tail-entry>, because the following
NOTE contains all that needs to be said.]

      NOTE: Historically, the <tail-entry> indicated the name of the
      sender.  If not used for this purpose, the string "not-for-mail"
      is often used instead (since at one time the whole path could be
      used as a mail address for the sender).

      NOTE: Although case-insensitive, it is intended that the <path-
      keyword>s "POSTED" and "MISMATCH" should be in upper case, to
      distinguish them from the <path-identity>s which are traditionally
      in lower case.

|     NOTE: The character ':' is now permitted within a <path-identity>
|     to facilitate the use of an <IPv6address> therein.
[That sentence belongs here, rather than in USEPRO.]

In USEPRO

2.1.  Defining the Architecture

....................

   Posting, reading and followup agents (which are usually just
   different services provided by the same piece of software) are known
   collectively as "user agents".

|  Injecting, relaying and serving agents (which are often just
|  different services provided by the same piece of software) are known
|  collectively as "news-servers".

2.2.  Identification of news-servers

|  News-servers need to identify themselves, by inserting their <path-
|  identity>s (F-3.1.6), into Path, Injection-Info and Xref headers. An
|  injecting agent MUST identify itself with the same <path-identity> in
|  both Path and Injection-Info headers.  To ensure that a <path-
|  identity> provides a unique identity for the news-server concerned,
|  it SHOULD be one of:
 
|  1. A fully qualified domain name (FQDN) associated with an A record,
|     which SHOULD identify the actual machine inserting this <path-
|     identity> and, ideally, should also be "mailable" (see below).
 
|  2. A fully qualified domain name (FQDN) associated with an MX record,
|     which MUST be "mailable".
 
|  3. An arbitrary name believed to be unique and registered at least
|     with all other news-servers receiving articles directly from the
|     given one.
 
|  4. An encoding of an IP address - <IPv4address> or <IPv6address> [RFC
|     3986].
 
|  The FQDN of a news-server is "mailable" if its administrators can be
|  reached by email using both of the forms "usenet@<FQDN>" and
|  "news@<FQDN>", in conformity with [RFC 2142].
 
|  For an injecting agent prepending to a Path header (7.2.2), the
|  <path-identity> MUST be option 1 or 2 and the FQDN MUST be mailable,
|  and if the agent offers its services to the general public the form
|  "abuse@<FQDN>" MUST also be available, unless a more specific
|  complaints address has been provided in a <complainto-param> of an
|  Injection-Info header (F-3.2.14).  For other agents, options 1
|  through 3 are to be preferred.

7.2.1.  Proto-articles

        NOTE: An article that is offered for reinjection has, by
        definition, already been injected once, and is not therefore to
        be considered as a proto-article.  Hence a genuine proto-article
|       will not contain any Injection-Date header nor any "POSTED" in
|       its Path header, though that header MAY contain <path-identity>s
|       corresponding to machines traversed between the posting agent
|       and the injecting agent proper.

7.2.  Duties of an Injecting Agent

   10.It MUST then prepend the <path-identity> of the injecting agent,
|     followed by a '!', the <path-keyword> "POSTED" and a further "!"
|     (or "!!" if appropriate) to the content of the Path header; this
|     SHOULD then be followed by CRLF and WSP if it would otherwise
|     result in a line longer than 79 characters.  The prepended
|     <path-identity> MUST be an FQDN mailable address (F-3.1.6).
 
|       NOTE: This could result in more that one "POSTED" <path-keyword>
        in the case of reinjection.

7.3.  Duties of a Relaying Agent

   1. It MUST establish the trusted identity of the source of the
      article and compare it with the leftmost <path-identity> of the
      Path header's content. If it matches it MUST then prepend its own
|     <path-identity> and a '!!' <path-delimiter> to that content. If
|     it does not match then it prepends instead two entries to that
|     content; firstly the true established <path-identity> of the
|     source followed by a '!', the <path-keyword> "MISMATCH" and a
|     further '!', and then, to the left of that, its own <path-
|     identity> followed by a '!!' <path-delimiter> as usual. This
|     prepending of two entries SHOULD NOT be done if the provided and
|     established identities match. All such prepended material SHOULD
|     be followed by CRLF and WSP if it would otherwise result in a line
|     longer than 79 characters.
 
7.3.1.  Path-Header Example

|     Path: foo.isp.example!!foo-server!!bar.isp.example!MISMATCH!
|        2001:DB8:0:0:8:800:200C:417A!!old.site.example!barbaz!!
|        baz.isp.example!POSTED!!dialup123.baz.isp.example!not-for-mail
 
        NOTE: That article was injected into the news stream by
|       baz.isp.example, as indicated ny the <path-keyword> "POSTED"
        (complaints may be addressed to [email-masked]). The
|       injector has chosen to record that it got it from
        dialup123.baz.isp.example. "not-for-mail" is a dummy <tail-
        entry>, though sometimes a real userid is put there.

        The article was relayed, perhaps by UUCP, to the machine known,
        at least to old.site.example, as "barbaz".

        Barbaz relayed it to old.site.example, which does not yet
        conform to this standard (hence the '!' <path-delimiter). So one
        cannot be sure that it really came from barbaz.

|       Old.site.example relayed it to a site claiming to have the IPv6
|       address [2001:DB8:0:0:8:800:200C:417A], and claiming (by using
|       the '!!'  <path-delimiter>) to have verified that it came from
        old.site.example.

|       [2001:DB8:0:0:8:800:200C:417A] relayed it to "foo-server" which,
        not being convinced that it truly came from
|       [2001:DB8:0:0:8:800:200C:417A], inserted the <path-keyword>
|       "MISMATCH" and then did a reverse lookup on the actual source
        and concluded it was known as bar.isp.example (that is not to
|       say that [2001:DB8:0:0:8:800:200C:417A] was not a correct IPv6
        address for bar.isp.example, but simply that that connection
        could not be substantiated by foo-server). Observe that foo-
        server has now added two entries to the Path.

        "foo-server" is a locally significant name within the complex
        site of many machines run by foo.isp.example, so the latter
|       should have no problem recognizing foo-server and using a '!!'
        <path-delimiter>.  Presumably foo.isp.example then delivered the
        article to its direct clients.

|       It appears that foo-server and barbaz decided to fold the line,
        on the grounds that it seemed to be getting a little too long.

And also several minor consequential changes.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 02 Jun 2005 14:10:04 +0200

Charles Lindsey wrote:

> This is, I hope, the final form of the Path header.

>                    *( ALPHA / DIGIT / "-" / "." / ":" / "_" )

See below.

>| 1. A fully qualified domain name (FQDN) associated with an
>|    A record, which SHOULD identify the actual machine

RfC 1034 (3.6) offers s/machine/host/ if it has an A record.
I stumbled over it while checking that we get no "_" here.

>| this SHOULD then be followed by CRLF and WSP if it would
>| otherwise result in a line longer than 79 characters.

See below.

>| All such prepended material SHOULD be followed by CRLF and
>| WSP if it would otherwise result in a line longer than 79
>| characters.

I tried to check the magic number 79 in s-o-1036, found nothing,
but stumbled over this part:

  Note, in particular, that RFC 1036 treats "_" as  a  delim-
  iter, but in fact it is known to appear in relayer names
  occasionally.

So "_" is apparently not exactly compatible with RfC 1036.  If
Henry doesn't publish s-o-1036 before its grandson the latter
could say so.  Most probably irrelevant.

If the 79 is an almost arbitrary magic number I'd prefer one
of the known magic numbers (78 in 2822, or 76 in 2047), too
many magic numbers are confusing.
                                  Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 02 Jun 2005 14:24:17 +0200

Sorry, now that Bruce is back I could stop this, but...
 
>|    A record, which SHOULD identify the actual machine
 
...we need "or AAAA", don't we ?

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 02 Jun 2005 17:59:13 +0200

--On onsdag, juni 01, 2005 19:54:44 +0000 Charles Lindsey 
<[email-masked]> wrote:

> The <keyword> "POSTED"

checking..... is the !! stuff a feature that has seen production use 
somewhere, or is it a new feature that we are adding?

since it could concievably be constructed as having to do with security, 
it's a potential sticky point with the security ADs.

                  Harald

---

From: Russ Allbery <[email-masked]>
Date: Thu, 02 Jun 2005 09:36:42 -0700

> somewhere, or is it a new feature that we are adding?

It's a new feature.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 3 Jun 2005 13:28:13 GMT

>somewhere, or is it a new feature that we are adding?

It is a new feature, but carefully crafted so that it causes no harm on
the existing network.

>it's a potential sticky point with the security ADs.

It is indeed connected with security. The Bad Guys were in the habit of
preloading their Path headers with a long string culled from some other
article, so as to make it appear that its point of origin was far removed
from themselves. The change will make it much easier to spot such
malpractice and to determine where it originated.

But of course the benefit will only appear when the majority of
news-servers routinely do it, hence various "MUST"s carefully placed in
USEPRO to encourage them to do so.

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 3 Jun 2005 13:18:09 GMT

>I stumbled over it while checking that we get no "_" here.

OK, I changed that.

>So "_" is apparently not exactly compatible with RfC 1036.

It is impossible to tell from RFC 1036 exactly what is a delimiter and
what is not.

>many magic numbers are confusing.

OK, I have changed it into "... result in a line of excessive length."
(and the same under Relaying Agents).  Anybody who wants to know what
"excessive" means can go look in USEAGE.

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 3 Jun 2005 13:19:14 GMT

>...we need "or AAAA", don't we ?

Eh? What is "AAAA"?

---

From: Henry Spencer <[email-masked]>
Date: Fri, 3 Jun 2005 11:04:19 -0400 (EDT)

> >...we need "or AAAA", don't we ?

An AAAA record is the IPv6 equivalent of an A record.  IPv6 addresses are
four times as long as IPv4 addresses, so the name seemed obvious... :-)

                                                          Henry Spencer
                                                       [email-masked]

---

From: Russ Allbery <[email-masked]>
Date: Fri, 03 Jun 2005 08:11:48 -0700

Charles Lindsey <[email-masked]> writes:
> Frank Ellermann <[email-masked]> writes:

> Eh? What is "AAAA"?

The IPv6 equivalent of an A record.

---

From: Bruce Lilly <[email-masked]>
Date: Fri, 3 Jun 2005 12:00:28 -0400

> It is impossible to tell from RFC 1036 exactly what is a delimiter and

Been there, discussed that.  1036 describes "version B2.11 of the News
program".  We know precisely which characters were delimiters, and that
information has been presented on this WG's mailing list.  See
http://www.imc.org/ietf-usefor/2004/Jun/0566.html
and
http://www.imc.org/ietf-usefor/2004/Jul/0265.html

Underscore was not a Path field delimiter as documented by "punctuation"
in RFC 1036.  That term means the set of characters:
#define NETCHRS  "!:@^%,"/* Punct. chars used for various networks       */

Note that that set does include the colon character.

---

From: Henry Spencer <[email-masked]>
Date: Fri, 3 Jun 2005 12:34:45 -0400 (EDT)

On Fri, 3 Jun 2005, Bruce Lilly wrote:
> program".

Been *there*, discussed *that*.  No, it doesn't.  Possible intent
notwithstanding, there are many discrepancies between the two, enough that
the B2.11 code is not useful as a model implementation of 1036. 

We used both -- rather cautiously -- as references when we wrote C News. 
They not infrequently disagree. 

Note also that 1036 does not claim to "describe" B2.11.  Rather, it says
it is an update of 850, "reflecting" B2.11 -- that is, that changes have
been made to 850 based on things done in B2.11.  That doesn't imply that
1036 and B2.11 are even intended to match exactly.

>in RFC 1036.

1036 never says exactly what "punctuation" is supposed to mean for its
purposes, although it does say that the characters "considered part of
host names" are "letters, digits, periods and hyphens". 

                                                          Henry Spencer
                                                       [email-masked]

---

From: Bruce Lilly <[email-masked]>
Date: Fri, 3 Jun 2005 15:04:50 -0400

> We used both -- rather cautiously -- as references when we wrote C News. 

And a year ago we discussed what C news (and INN) do and do not
treat as delimiters.

Why are we still flogging this horse?

---

From: Henry Spencer <[email-masked]>
Date: Fri, 3 Jun 2005 15:28:16 -0400 (EDT)

> Why are we still flogging this horse?

Because you again tried to resolve vagueness in 1036 by appealing to the
B2.11 code, as if it was a model implementation of 1036, which it wasn't. 

The vagueness in 1036 is inherent and unfixable.  We need to think about
what the best resolution of it is today, rather than trying to deduce what
1036's authors intended nearly 20 years ago. 

The practices followed by existing implementations are relevant only
insofar as they impose backward-compatibility constraints.  In that
regard, B2.11 can safely be ignored, and even C News is marginal. 

                                                          Henry Spencer
                                                       [email-masked]

---

From: Bruce Lilly <[email-masked]>
Date: Fri, 3 Jun 2005 15:53:51 -0400

> B2.11 code, as if it was a model implementation of 1036, which it wasn't. 

No, the code provides insight into what the 1036 text "punctuation"
means.

We know that it does NOT mean underscore, and that it does mean colon.

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 03 Jun 2005 22:30:18 +0200

> We know that it does NOT mean underscore, and that it does

| Letters, digits, periods and hyphens are considered part of
| host names; other punctuation, including blanks, are
| considered separators.

That's 100% clear, and whether it's wrong is beside the point.

If "we" want the underscore for UUCP host names, then it needs
a "note" like the existing "note" for the colon.  Or it needs
s-o-1036 published as RfC, that should simplify some problems
of inheritance.

I'd just drop underscore in path-identity and be done with it.
Allowing it only in the tail-entry is no syntactical problem.

                        Bye, Frank

---

From: Henry Spencer <[email-masked]>
Date: Fri, 3 Jun 2005 16:40:40 -0400 (EDT)

> No, the code provides insight into what the 1036 text "punctuation"

The only way it can provide such insight is if you first assume that it is
meant to be an exact implementation of 1036 (or equivalently that 1036 was
written as a specification matching the B2.11 code).  That assumption is
unsubstantiated, and must be considered highly questionable given the
numerous differences between what 1036 says and what B2.11 does.  The
B2.11 code cannot provide *reliable* insight into ambiguities in 1036. 

> We know that it does NOT mean underscore, and that it does mean colon.

We might *speculate* to that effect, but we don't *know* it.

The fact is, 1036 doesn't say exactly what "punctuation" means.  Period. 

Moreover, it doesn't really matter.  We're trying to write a new document,
not to help 1036's authors revise theirs; what they *intended* is simply
irrelevant. 

                                                          Henry Spencer
                                                       [email-masked]

---

From: Bruce Lilly <[email-masked]>
Date: Sun, 5 Jun 2005 09:03:12 -0400

On Fri June 3 2005 16:40, Henry Spencer wrote:
> written as a specification matching the B2.11 code).

Incorrect.  The only "assumption" is that when the RFC 1036/B News 2.11
authors -- the same people -- write "punctuation" or an abbreviation thereof
in the same limited context in the same timeframe with an explicit cross-
reference between documents, that it in fact means the same thing in that
limited context.

> That assumption is 
> numerous differences between what 1036 says and what B2.11 does.

Utter nonsense.  The real, minimal "assumption" is quite reasonable, unlike
other claims that have been made in this WG, e.g. that the term is based
on some Unicode or POSIX set (both Unicode and POSIX post-date 1036).

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 6 Jun 2005 16:40:58 GMT

>four times as long as IPv4 addresses, so the name seemed obvious... :-)

Ah! I see.

It now says:

|  1. A fully qualified domain name (FQDN) associated with an "A" or
|     "AAAA" record, which SHOULD identify the actual host inserting
|     this <path-identity> and, ideally, should also be "mailable" (see
|     below).

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 6 Jun 2005 16:45:54 GMT

>Why are we still flogging this horse?

We indeed flogged it very thoroughly, and one thing we concluded was that
aguments related to what some implementation of B News did simply did not
wash.

But the text now proposed is the final outcome of that discussion a year
ago, and what is and is not allowed in both <path-identity>s and
<path-delimiters> has been reduced to a minimum that should cause no
problems whatsoever on the existing network.

So stop flogging.