Subject: Re: [1586] Multiple POSTED in Path: header

From: Russ Allbery <[email-masked]>
Date: Sun, 01 Feb 2009 17:57:29 -0800

"Charles Lindsey" <[email-masked]> writes:

> 3.4.1 of the present draft contains:
>
> A proto-article has the same format as a normal article except that the
> Injection-Info and Xref header fields MUST NOT be present; the Path
> header field SHOULD NOT contain a "POSTED" <diag-keyword>; and any of
> the following mandatory header fields MAY be omitted:  Message-ID, Date,
> and Path. In all other respects, a proto-article MUST be a valid Netnews
> article. In particular, the header fields which may be omitted MUST NOT
> be present with invalid content.
>
> That "SHOULD NOT" was formerly "MUST NOT", so it is an improvement.
>
> But I propose changing it to:
>
> .. the Path header field MAY contain a "POSTED" <diag-keyword>; ...

Since that is equivalent to saying nothing about the POSTED
<diag-keyword>, were we to decide on this change, I think I would omit the
sentence entirely and just add the note (with corresponding minor wording
changes).  Or even better, move the NOTE to a subsection of 3.2 that
explains the meaning of a POSTED keyword and the circumstances under which
a Path header could be constructed containing several of them.

> and adding a NOTE such as the following:
>
>    NOTE: Whereas the presence of two "POSTED" <diag-keyword>s will often
>    indicate a malicious attempt to disguise the true origin of an article,
>    it could also arise following some ususual gatewaying or injecting
>    scenario (taking advantage of the "MAY contain" above), in which case
>    it could be useful for detecting unintended loops or mismanaged
>    gateways. The whole intent of these <path-diagnostic)s is to assist
>    humans in assessing unusual situations, and it would be unwise for
>    subsequent agents automatically to assume one possibility or the other.
>
> Essentially, I am arguing for not throwing away any information which might
> be of subsequent diagnostic use, possibly at the expense of making people
> think a bit before accusing posters of trollery. It is up to the "netkops"
> to deal with these issues, and our draft should not try to prejudge their
> efforts.

The corresponding text in 3.4.2 is:

   In some cases, offering the same proto-article to all injecting
   agents may not be possible (such as when gatewaying, after injection,
   articles found on one Netnews network to another supposedly-
   unconnected one).  In this case, the posting agent MUST remove any
   Xref header field and rename or remove any Injection-Info, Path, and
   other trace header field before passing it to another injecting
   agent.  (This converts the article back into a proto-article.)

Since multiple injection is the only means by which multiple POSTED
<diag-keywords> can arise with their intended meaning and purpose, it
seems pointless to relax the requirement too far in one area and not in
the other.

I think our options here are:

1. Treat the Path header as special and permit (or even encourage)
   retaining it while not permitting retention of the other trace headers,
   on the grounds that it has an intrinsic ordering and therefore isn't as
   prone to confusion as having multiple trace headers that aren't
   ordered.

2. Leave the existing wording, which requires multiple injection to remove
   or rename the Path header but allows readers to add spurious POSTED
   <diag-keyword>s by violating a SHOULD.  I'm not entirely comfortable
   with it, but I can't think of any specific protocol harm it could
   cause.

3. Restore the MUST in the proto-article section to match the MUST in
   3.4.2 and make the document consistent but possibly too strict.

4. Relax the MUST generally in 3.4.2, but then we will run into issues
   with Injection-Info, which the posting agent is not permitted to supply
   and which doesn't have an intrinsic ordering the way that the Path
   header does.

The more I think about it, the more option 1 seems like a workable idea,
but I think that's reversing some decisions that we made some time back.
That makes me nervous at this stage of the process.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 2 Feb 2009 12:25:55 GMT

>a Path header could be constructed containing several of them.

Sure.

>seems pointless to relax the requirement too far in one area and not in

Agreed. I think that MUST is a bit too strong as regards the Path header.
Indeed, it violates 2119 insofar as the protocol does not actually break
if is it not done - same for Xref for that matter, so it is more a matter
of what constitutes Good Practice. But failing to remove/rename
Injection-Info and other tracers would be a more serious matter, because
it would then be telling an untruth, so the MUST is indeed needed there.

>   prone to confusion as having multiple trace headers that aren't

Yes, I think I like that.

>   3.4.2 and make the document consistent but possibly too strict.

That would be to increase violation of 2119, which is why we demoted it
before.

>   and which doesn't have an intrinsic ordering the way that the Path

That seems to agree with my remark concerning Injection-Info above, so I
think we agree that bit remains a MUST.

>That makes me nervous at this stage of the process.

+1

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131            Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Russ Allbery <[email-masked]>
Date: Mon, 02 Feb 2009 18:46:33 -0800

"Charles Lindsey" <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

> Agreed. I think that MUST is a bit too strong as regards the Path
> header.  Indeed, it violates 2119 insofar as the protocol does not
> actually break if is it not done - same for Xref for that matter, so it
> is more a matter of what constitutes Good Practice.

If you include Xref in an NNTP POST to an INN server, the article will be
rejected.  That's why I made it a MUST; the protocol does actually break
in practice if you don't strip Xref.  (Whether it *should* is a separate
question, of course, but there's a huge installed base that behaves that
way.)

I agree that current servers do not reject posts that contain a POSTED
<diag-keyword>.

> Yes, I think I like that.

If we want to move forward with this, I think I could use some help
analyzing what needs to change in the document to make it consistent again
with this new approach.

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 03 Feb 2009 13:27:51 +0100

Charles Lindsey wrote:
> A proto-article has the same format as a normal article except that
> the Injection-Info and Xref header fields MUST NOT be present; the
> Path header field SHOULD NOT contain a "POSTED" <diag-keyword>; and
> any of the following mandatory header fields MAY be omitted:
> Message-ID, Date, and Path. In all other respects, a proto-article
> MUST be a valid Netnews article. In particular, the header fields
> which may be omitted MUST NOT be present with invalid content.
>

I disagree with this idea, and think we should keep the SHOULD NOT.

Not throwing away information can be accomplished by tricks like 
Old-Path:. As the proposed note effectively says, saying that multiple 
POSTs are generally acceptable is an invitation to the script kiddies to 
play with it.

                    Harald

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 4 Feb 2009 10:38:14 GMT

>question, of course, but there's a huge installed base that behaves that

You surprise me. I had always assumed that Xref was only meaningful for a
particular server and the clients that used it, and that if ever it
received an already-present Xref (from whatever source) it would simply
discard it and substitute its own. But if it is as you say, then I suppose
that MUST has to remain (otherwise, SHOULD would have sufficed).

>analyzing what needs to change in the document to make it consistent again

Sure. If we agree on the effects we want to achieve, then I am due for a
careful read through the whole document to see what final inconsistencies
might be lurking there,

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 4 Feb 2009 10:47:39 GMT

>POSTs are generally acceptable is an invitation to the script kiddies to 

If the script kiddies try it, they will get caught out because two POSTEDs
will then be present and any suspicion of trollery will be confirmed.

But in the case of some carefully crafted Gatewaying scheme, or, worse, in
the case of some broken Gatewaying scheme that ought to be spotted,
destroying the evidence will do harm. And I doubt people will want to
bother with Old-Post hacks.

There are plenty of Netkops around who will gleefully denounce the trolls,
and plenty of experienced newsadmins who will notice indications of broken
Gatewaying (especially if consistently repeated) and quietly warn those
causing it.