Subject: Ticket status, USEFOR

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 19 Jul 2005 12:00:19 +0200

There are still 13 open tickets on the USEFOR document (and 2 on USEPRO).

I think we should aim for getting those 13 tickets to "Text accepted" or 
"No change needed" state within the next 3 weeks, so that Ken can generate 
a hopefully-final USEFOR-06 and submit it for publication just after the 
IETF meeting. This should then be able to go to WG Last Call.

The outstanding USEFOR tickets (in numeric order) are:

1003 USEFOR 3.1.3 - Cleanup ABNF for msg-id
 "Document updated"

1021 USEFOR 3.1.5 Newsgroups header ABNF and description needs cleanup
 "Document updated"

1028 USEFOR 3.1.2 Date: What zones should be on the MUST accept list?
 "Document updated"

1029 USEFOR 3.2.2 References: Should comments be a MUST NOT?
 "No consensus"

1030 USEFOR general: Backwards compatibility and handling incompatibilities
 "No consensus" - on June 28, I suggested "No change needed".

1031 USEFOR general: Standard terminology and text
 "No consensus" - on June 28, I suggested "No change needed".

1032 USEFOR general: Document changes from RFC 1036
 "Text needed" - this section is still blank in -05

1042 USEFOR 3.1.5: Newsgroups folding?
 "Document updated"

1046 USEFOR 5. MIME boundary security considerations
 "Text needed"

1047 USEFOR 3.1.6: Path field delimiters and components
 "Text needed"

1048 USEFOR 3.2.1 Injection-date syntax
 "Proposed no change"

1050 USEFOR 3.2.11: Xref field for local use only?
 "No change needed" - I've closed this one, so it'll be gone from my
  next list.

1052 USEFOR general: Document changes from RFC 2822
 "Text proposed" - appendix C of -05.

1053 USEFOR 2.1 Relationship to RFC 2822
 "Proposed no change".

For the ones in "Document updated", Ken has indicated in the tracker that 
the chairs should judge whether the issues are closed or not.

When discussing the specific issues, please include the number in the 
subject header, and STAY ON TOPIC.

Thanks!

                        Harald

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 20 Jul 2005 07:46:47 +0200

Harald Tveit Alvestrand wrote:

> we should aim for getting those 13 tickets to "Text accepted"
> or "No change needed" state within the next 3 weeks, so that
> Ken can generate a hopefully-final USEFOR-06 and submit it 
> for publication just after the IETF meeting.

> 1046 USEFOR 5. MIME boundary security considerations

My last complete proposal was in
<http://mid.gmane.org/[email-masked]>

Bruce proposed another aspect in
<http://article.gmane.org/gmane.ietf.usenet.format/28940>

But that went in the (IMHO) wrong direction of "there's
no real problem unless implementors are too naive":

<http://article.gmane.org/gmane.ietf.usenet.format/28953>

It is a real problem, because 2231 didn't want to screw
the boundaries, so nobody expects this, naive or not.  Bye

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 20 Jul 2005 08:19:34 +0200

Harald Tveit Alvestrand wrote:
 
> 1028 USEFOR 3.1.2 Date: What zones should be on the MUST
> accept list?  "Document updated"

Yes, but it shouldn't say "from" instead of "orig-date" ;-)

And IIRC we wanted to mention 2.2 _and_ 2.1 in 3.1.2, not
only 2.2, because the "forget obs-" info is in 2.1, and
the "but accept GMT" part of 3.1.2 needs this 2.1 detail.

How about adding an example below the <orig-date> syntax ?  

| This is an example for the deprecated GMT syntax, GMT is
| interpreted as +0000, compare [RFC2822] chapter 4.3: 
|
|      Date: Wed, 20 Jul 2005, 22:33:44 GMT

                         Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 20 Jul 2005 08:43:11 +0200

Harald Tveit Alvestrand wrote:

> 1021 USEFOR 3.1.5 Newsgroups header ABNF and description
> needs cleanup  "Document updated"

=> resolved

> 1029 USEFOR 3.2.2 References: Should comments be a MUST NOT?

It's now good enough - I'd still prefer the <msg-id-list> ABNF,
but that's not essential.

> 1042 USEFOR 3.1.5: Newsgroups folding? "Document updated"

=> resolved

> 1048 USEFOR 3.2.1 Injection-date syntax  "Proposed no change"

ACK.  I'm not exactly sure why we don't add this timestamp to
the Injection-Info, maybe because 2822 <date-time> is messy (?)
It would fit nicely between the <path-identity> and the first
semicolon introducing the parameters.

> 1053 USEFOR 2.1 Relationship to RFC 2822

Maybe needed in #1028 (3.1.2), otherwise => resolved.  Bye

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 20 Jul 2005 09:03:46 +0200

>> 1048 USEFOR 3.2.1 Injection-date

Sorry, I used an incorrect number in the subject.  This beast
could be a privacy issue:  Some users - posting when they are
supposed to do something else - intentionally use timestamps
like 00:00 -0000.  The Injection-Date could defeat these
efforts.

Why are there mandatory "security considerations", mandatory
"IANA considerations", optional "I18N considerations", but no
"privacy considerations" - is that something we should invent
for USEFOR ?
                         Bye, Frank

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 20 Jul 2005 09:14:55 +0200

> "privacy considerations" - is that something we should invent

We can certainly add whatever sections we feel that the document needs.
This may, however, be something that fits better in USEAGE.

                       Harald

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 20 Jul 2005 10:19:49 +0200

[Injection-Date and privacy]
> This may, however, be something that fits better in USEAGE.

Yes, or in USEPRO, it's already very complex.  I just saw that
the rules for Injection-Info and Injection-Date are different:

The former is replaced in the case of a "reinjection", but the
latter is kept as is.  So joining them into one header field
won't work if the "reinjection" rules are correct.  Bye, Frank

---

From: Bruce Lilly <[email-masked]>
Date: Wed, 20 Jul 2005 11:02:51 -0400

> like 00:00 -0000.  The Injection-Date could defeat these

What privacy issue? "use timestamps" where and for what purpose?
Injection-Date records the time of injection; how precisely is
that a privacy issue?

> "privacy considerations" - is that something we should invent

Privacy (see definition in BCP 36, a.k.a. RFC 2828) is a
security-related issue.  Privacy issues could be a subsection
of the security considerations section.

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 20 Jul 2005 14:22:40 GMT

>1021 USEFOR 3.1.5 Newsgroups header ABNF and description needs cleanup

I think this is complete technically, but it needs some explanation of
the forbidden components, etc, in a NOTE.

>1029 USEFOR 3.2.2 References: Should comments be a MUST NOT?

I thought we had agreed to leave it as SHOULD NOT. Given that we have
accepted "SHOULD NOT generate" for folded Newsgroups, I don't think we can
justify anything stronger here.

> "No consensus" - on June 28, I suggested "No change needed".

Agreed.

> "No consensus" - on June 28, I suggested "No change needed".

Agreed.

> "Text needed" - this section is still blank in -05

There is suitable text in 3.1 of USEPRO, which should probably be moved to
USEFOR (though a couple of items might remain). The reason I say "probably"
is that the related section on "Transitional Arrangements" is less
obviously a USEFOR matter but, having just reviewed it, many of the items
in that are already covered by "do not do this yet" remarks in USEFOR.

>1046 USEFOR 5. MIME boundary security considerations

There is some proposed wording in the Security Considerations of USEPRO to
cover this. All we need to do is decide whether to include it. I certainly
think it is a USEPRO matter and, indeed, it would probably be better to
put all the Security Considerations into USEPRO, on the grounds that they
all deal with things that can go wrong with the protocols.

>1047 USEFOR 3.1.6: Path field delimiters and components

There is ongoing discussion of section 2.3 of USEPRO which is relevant to
this. The particular issue which interacts with USEFOR is the extent to
which servers should be allowed to identity themselves in the Path using
their IP address. It is agreed that this should be
discouraged/deprecated/whatever, but I think it would be going too far to
forbid it entirely.

I would suggest letting that USEPRO discussion proceed (perhaps it needs a
ticket) and when that is settled the syntax in USEFOR can be adjusted to
match.

>1048 USEFOR 3.2.1 Injection-date syntax

Agreed.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 21 Jul 2005 08:02:43 +0200

Charles Lindsey wrote:
 
 [1046 USEFOR 5. MIME boundary security considerations]
> There is some proposed wording in the Security Considerations
> of USEPRO to cover this.

These considerations need to be in usefor-06, because they are
a consequence of the "MUST MIME and 2231" in usefor-??, it's no
NetNews protocol problem.  Servers aren't supposed to know what
MIME is, only gateways might be mildly interested.

> I certainly think it is a USEPRO matter

IBTD.

> probably be better to put all the Security Considerations
> into USEPRO, on the grounds that they all deal with things
> that can go wrong with the protocols.

Yes, and these MIME problems are no NetNews protocol problems,
so that part shouldn't go into the next usePRO, it's attached
to several MUSTs in useFOR-05 chapter 2.3.

The part about CTL characters in article bodies could also go
to useFOR-06.  No idea why you mention only bodies, it's not
necessarily better in any displayed header field.

                      Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 21 Jul 2005 08:16:46 +0200

Bruce Lilly wrote:

>> could be a privacy issue:  Some users - posting when they
>> are supposed to do something else - intentionally use
>> timestamps like 00:00 -0000.  The Injection-Date could
>> defeat these efforts.
 
> What privacy issue? "use timestamps" where and for what
> purpose? Injection-Date records the time of injection; how
> precisely is that a privacy issue?

If you're doing it in your office hours, and it isn't a part
of your job.  The Injection-Date is a new header field, it's
not obvious for normal users.  The same problems as with the
Injection-Info.

I'd still like to be convinced why we must split the info in
two header fields, "because the Injection-Date is no variant
header" is only a formal argument, I don't get the idea behind
this decision.

All this "reinjection" stuff in usepro-04 is very confusing,
maybe it deserves its own "unusual operations" chapter.  Why
replace Injection-Info but not Injection-Date in this case ?

> of the security considerations section.

Okay, then that's something for chapter 8 in usepro-05.  Bye

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 21 Jul 2005 09:07:40 +0200

(Changing the subject, since this is specific)

> discouraged/deprecated/whatever, but I think it would be going too far to

On reviewing, I have found you arguing that systems should be allowed to so 
identify themselves, citing 4% of messages that have IP addresses in the 
path. But your analysis of July 5 seemed to indicate that in all cases 
where it was possible to guess at the purpose, the IP address was injected 
in the path as "diagnostic information", not in a way that would make you 
think it was intended to manage propagation.

I think there's general agreement that IPv4 addresses MUST be allowed to 
occur in the Path: header (because of current practice and useful 
diagnostic ability), and more-or-less agreement that IPv6 addresses should 
be allowed because of being "address format neutral", but I don't see any 
support in the group for letting people use it as identifiers.

> ticket) and when that is settled the syntax in USEFOR can be adjusted to

Or the other way around.... yes, the USEPRO discussion needs a ticket.

                        Harald

---

From: Bruce Lilly <[email-masked]>
Date: Thu, 21 Jul 2005 09:20:06 -0400

> header" is only a formal argument, I don't get the idea behind

The injection date needs to be examined to avoid propagating "stale"
articles (so the argument goes).  That's reasonable for the
Injection-Date field syntax.  It would be unreasonable to have to
parse the convoluted Injection-Info field (as currently sort-of
defined (the ABNF has major flaws last time I looked)).
 
> replace Injection-Info but not Injection-Date in this case ?

It should be elided.  Only Charles seems to think there is such
a thing.  See John Stanley's relevant comments in yesterday's
messages.

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 22 Jul 2005 15:52:32 GMT

>support in the group for letting people use it as identifiers.

Yes, I think it is agreed that both IPv4 and IPv6 addresses will be there.
The issue is whether they MUST be reserved for "diagnostic information"
only.

What I do not see is what you actually gain by writing in such a "MUST".
It does not make the "dead:beef" problem go away (not that I think that is
a serious problem anyway). There is no change in interoperability, and no
harm that I can see (provided any IP address you use is a 'public' one for
a machine that you own). The only difference it would make, that I can
see, is if you said to your upstream "I shall use my IP address
123.123.123.123 in my Path; please do not send me articles with that
already in the Path", then you upstream should reply "Sorry! The standard
does not allow me to do that". Indeed, I do not think a single line of
code in any implementation would be changed.

There may be good social/political arguments for the retriction, but those
are more usually addressed by "SHOULD" wording than by "MUST".

Of more immediate concern, however, is what is meant by "diagnostic
information". As USEPRO is currently written, the only diagnostic
information that may be written is when you have detected that the (IP
address of) the source of the article was different from what the source
site claimed to be its <path-identity>. And in that case you are supposed
to put the true IP address (or equivalent domain-name) followed by
"MISMATCH".

But it seems that some sites are currently including the source IP address
as a diagnostic even when no mismatch has been detected. I think we should
probably reword it to indicate that is allowed (though maybe with some
discouragement on account of creating unnecessary clutter in the Path).

Also, the present wording allows just one entry per relaying agent (or two
with MISMATCH), and yet some sites (e.g. nntpserver.com and demon) want to
put in extra identification entries for various plausible reasons. We
probably ought to permit that too (perhaps with discouragement).

If people are agreed we need to allow such things, then I can propose
some wording. Probably needs a new ticket.

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 22 Jul 2005 15:16:26 GMT

>not obvious for normal users.  The same problems as with the

Then you had better be careful, or issue an 'at' job to inject it after
you have left for home.

>header" is only a formal argument, I don't get the idea behind

Having it as a "posting-date" parameter of Injection-Info was discussed by
the WG, and rejected (I don't recall the precise reason, but it is all in
the archives).

>replace Injection-Info but not Injection-Date in this case ?

Because Injection-Date indicates that moment of time when the article
entered the network and started to flood through lots of different routes
in parallel. Some of those routes might have passed through a reinjection
point, and some not, yet they must all be treated the same when detecting
'staleness'; otherwise some site might accept it again after it had
already accepted and expired it once.

---

From: Russ Allbery <[email-masked]>
Date: Fri, 22 Jul 2005 09:19:08 -0700

Charles Lindsey <[email-masked]> writes:
> Frank Ellermann <[email-masked]> writes:

> Having it as a "posting-date" parameter of Injection-Info was discussed
> by the WG, and rejected (I don't recall the precise reason, but it is
> all in the archives).

The intention is for every news server to parse this field instead of
Date.  Given that, it should be easy to parse.  MIME-style parameters are
not.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Russ Allbery <[email-masked]>
Date: Fri, 22 Jul 2005 09:28:11 -0700

> What I do not see is what you actually gain by writing in such a "MUST".

Most other IETF protocols at this level strongly discourage use of IP
addresses for a variety of reasons: they carry no immediately
human-readable information, they are generally far more transient than
domain names in terms of allocation and controlling authority, they cannot
be "unwound" easily to determine the originating domain the way that
hostnames can, they often have no publically available record associated
with them that points to the actual organization responsible for the
message (as opposed to some service provider that organization has hired),
and use of IP addresses encourages and facilitates very unuseful behavior
such as using private IP addresses in public messages or using transient
DHCP-assigned IP addresses that will quickly be reassigned to some other
party.

I believe all of that is sufficient to warrant strongly discouraging use
of IP addresses for anything other than diagnostic information.  I'm
ambivalent about SHOULD vs. MUST.

> site claimed to be its <path-identity>. And in that case you are
> supposed to put the true IP address (or equivalent domain-name) followed
> by "MISMATCH".

> But it seems that some sites are currently including the source IP
> address as a diagnostic even when no mismatch has been detected. I think
> we should probably reword it to indicate that is allowed (though maybe
> with some discouragement on account of creating unnecessary clutter in
> the Path).

This reduces to the case where no matching information is stored and all
sites are mismatches, which conveniently is generally exactly what's going
on with the sites that always embed an IP address, so the only affect of
the change that you propose would be to allow one to omit the "MISMATCH"
entry.  I guess I don't really care.

> Also, the present wording allows just one entry per relaying agent (or
> two with MISMATCH), and yet some sites (e.g. nntpserver.com and demon)
> want to put in extra identification entries for various plausible
> reasons. We probably ought to permit that too (perhaps with
> discouragement).

There's no reason to even discourage it, as such a site would become
compliant anyway just by adding another internal machine that the traffic
bounces through.  So long as they follow the other rules for Path entries,
I don't see any reason why they shouldn't be able to add as many internal
entries as they want.

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 22 Jul 2005 22:57:47 +0200

Russ Allbery wrote:

>> Having it as a "posting-date" parameter of Injection-Info
>> was discussed by the WG, and rejected

That's clear.

> The intention is for every news server to parse this field
> instead of Date.  Given that, it should be easy to parse.
> MIME-style parameters are not.

Dito, same reason.  How about this:

~~~ old ~~~
injection-date = "Injection-Date:" SP date-time CRLF

injection-info = "Injection-Info:" SP [CFWS] path-identity
                 *( [CFWS] ";" [CFWS] inj-info-param )
                 [CFWS] CRLF
~~~ new ~~~
injection-info = "Injection-Info:" SP *WSP path-identity
                 1*WSP datetime
                 *( ";" [CFWS] inj-info-param [CFWS] ) CRLF
~~~ end ~~~

The added <date-time> ends with [CFWS], therefore I could join
the two [CFWS] before semicolon or CRLF into one [CFWS] behind
<inj-info-param>.

And I got rid of the first [CFWS] before the <path-identity>.

It won't work with Charles' concept of a "reinjection" keeping
the old <injection-date>, if that's critical forget it, please.

                          Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 25 Jul 2005 10:56:47 GMT

>DHCP-assigned IP addresses that will quickly be reassigned to some other

Yes, I agree with all that. It is a classic case calling for "SHOULD NOT",
as in my currently proposed text.

>> address as a diagnostic even when no mismatch has been detected. ...

>entry.  I guess I don't really care.

I don't quite follow there. For sure we don't want sites putting in
MISMATCH unless they have actually checked and found something fishy. If
they put the sending IP address in without MISMATCH, then they are saying
"This is where we got it from; compare it with the next thing in the Path
yourself if you are bothered". Just so long as they don't follow it with a
"!!" if they have not actually done the check.

Anyway, if others agree, I can write some wording to indicate that such
"diagnostic" usages are in order, and an IP used here does not violate the
SHOULD (or even MUST) NOT discussed above.

>> want to put in extra ........

>I don't see any reason why they shouldn't be able to add as many internal

OK, I can easily write that in if people are happy.

---

From: Russ Allbery <[email-masked]>
Date: Mon, 25 Jul 2005 11:07:53 -0700

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> This reduces to the case where no matching information is stored and
>> all sites are mismatches, which conveniently is generally exactly
>> what's going on with the sites that always embed an IP address, so the
>> only affect of the change that you propose would be to allow one to
>> omit the "MISMATCH" entry.  I guess I don't really care.

> they put the sending IP address in without MISMATCH, then they are
> saying "This is where we got it from; compare it with the next thing in
> the Path yourself if you are bothered". Just so long as they don't
> follow it with a "!!" if they have not actually done the check.

I wonder if we need another keyword that says "I haven't bothered to check
whether the Path is right; here's the IP address just for the hell of it."

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 26 Jul 2005 11:24:03 GMT

>whether the Path is right; here's the IP address just for the hell of it."

Hmmm! One could go on adding features for ever until we landed up with
something as complex as the mail Received header :-( .

OK, we have a site upstream.example.com, whose IP address is (apparently)
123.123.123.123, and a site downstream.example.com. So the Path might
contain:

    ....!downstream.example.com!!123.123.123.123!upstream.example.com!.....

The 123.123.123.123 was put there "just for the hell of it" by downstream,
and was preceded by "!!" (because for sure it was taken straight from the
packets received). The "!" afterwards indicates "we didn't (bother to)
check whether that is the IP used by upstream.

What can a reader of that Path deduce? The following are possible:

a) 123.123.123.123 was a diagnostic put there by downstream.
b) 123.123.123.123 was a genuine independent site (not upstream) which
inserted it as it passed by (violating a SHOULD NOT by using an IP rather
than a domain name).
c) 123.123.123.123 was put there by the sending agent at upstream (again
violating that SHOULD NOT). Maybe it was the same host as
upstream.example.com, or maybe it passed through both hosts at upstream.

(c) is a less likely deduction, for surely the host 123.123.123.123 knows
upstream.example.com well enough to be able to put a "!!" between them.

And (b) is less likely than (a), though you cannot be sure. So the
question is "how important is it to be able to distinguish between these
various cases"? If it is important, then another keyword might be in
order.

Don't forget that downstream is not precluded from using the domain name
(from a Reverse Lookup, perhaps) of 123.123.123.123 in its diagnostic, so
you might see

    ....!downstream.example.com!!upstream.example.com!upstream.example.com!...
or
    ....!downstream.example.com!!relay.upstream.example.com!upstream.example.com!...

So that gives you several more deducible scenarios corresponding to (a),
(b) or (c). All of which makes the idea on a further keyword a little more
attractive.

OK, let us try to digest all that and figure out all the consequences.

---

From: Bruce Lilly <[email-masked]>
Date: Wed, 27 Jul 2005 08:40:55 -0400

> something as complex as the mail Received header :-( .

The syntax for the Received field, which is specified in one of our
normative references, so a Document Editor certainly should be familiar
with it, is:
--------
received        =       "Received:" name-val-list ";" date-time CRLF

name-val-list   =       [CFWS] [name-val-pair *(CFWS name-val-pair)]

name-val-pair   =       item-name CFWS item-value

item-name       =       ALPHA *(["-"] (ALPHA / DIGIT))

item-value      =       1*angle-addr / addr-spec /
                         atom / domain / msg-id
---------
i.e. 5 concise rules, whereas the usefor draft gives the following as
syntax for Injection-Info:

   inj-info-param  =  post-host-param /
                      post-acct-param /
                      sender-param /
                      logging-param /
                      complainto-param /
                      ext-param
                      ;; At most one of "post-host-param",
                      ;; "post-acct-param", "sender-param",
                      ;; "logging-param" or "complainto-param"
                      ;; is allowed.

   ext-param       = parameter
   [[Should also allow for x-attributes?]]

   post-host-param =  "posting-host" "=" host-value

   host-value      =  dot-atom / [ dot-atom ":" ]
                      ( IPv4address / IPv6address ) ;  see [RFC 3986]

   post-acct-param =  "posting-account" "=" value

   sender-param    =  "sender" "=" sender-value

   sender-value    =  mailbox / "verified"

   logging-param   =  "logging-data" "=" value

   complainto-param=  "mail-complaints-to" "=" address-list

i.e. twice as many rules, many of which are multi-line, complex rules.
It's quite clear which is more complex.

> inserted it as it passed by (violating a SHOULD NOT by using an IP rather

http://www.dictionary.com/search?q=violating

1 entry found for violating.
vi·o·late   Audio pronunciation of "violating"  P   Pronunciation Key  (v-lt)
 tr.v. vi·o·lat·ed, vi·o·lat·ing, vi·o·lates 
 To break or disregard (a law or promise, for example).
 To assault (a person) sexually.
 To do harm to (property or qualities considered sacred); desecrate or defile.
 To disturb rudely or improperly; interrupt: violated our privacy.

As SHOULD NOT is a recommendation, not a law or promise, and involves no harm,
one cannot be meaningfully accused of "violating a SHOULD NOT".  One can be
said to have *disregarded* a SHOULD NOT (iff that is known to be the case)
or it may simply be the case that there were "valid reasons in particular
circumstances to ignore a particular item" (BCP 14).

> violating that SHOULD NOT).

Again inappropriate use of "violating".  The remote site cannot determine
that "upstream" put text there, only that a site ostensibly *claiming* to
be "upstream: put it there -- that is an important (and obvious) fact.

> upstream.example.com well enough to be able to put a "!!" between them.

No, in the first place, that's not where "!!" appears.  In the second
place, the text may have been placed there by the site claiming to be
"upstream" (said site not actually *being* "upstream"), possibly to
implicate the site with the specified IP address.  Any claims of
"less likely", given the uncertainty of what actually happened, is
completely unsupported.
 
> And (b) is less likely than (a),

An unsupported absolute assertion...

> though you cannot be sure.

...with conflicting evasiveness.  If you're not sure which is or is not
less likely, don't make an assertion about likelihood.  Such conflicting
stuff (avoiding the n-word) doesn't add to the discussion; it decreases
the signal-to-noise ratio.

> So the  

I read Russ' statement
"I wonder if we need another keyword that says 'I haven't bothered to check
whether the Path is right; here's the IP address just for the hell of it.'"
as a satirical comment on the silliness of inserting diagnostic information
(an IP address) w/o the keyword explicitly provided for indication of
diagnostic information -- that having been your suggestion, Charles.

Which brings us back to reserving use of IP addresses for diagnostic use as
discussed a month ago.

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 28 Jul 2005 13:39:48 GMT

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>........ I read Russ' statement

And I read it as meaning exactly what it said - that another keyword (or
maybe some other syntactic device) might help to indicate exactly what the
various items in the Path were trying to say. And I saw some merit in the
idea.

>Which brings us back to reserving use of IP addresses for diagnostic use as

It is a bit premature to be reserving certain addresses for diagnostic use
until it has been clarified what "diagnostic uses" are envisioned. At the
moment, the only diagnostic provision in the protocol as set out in USEPRO
is for an address followed by MISMATCH. It is now apparent that people are
including other addresses for diagnostic reasons, and I think we are
agreed that some further provision is needed.

One essential difference between a conventional Path entry and a purely
diagnostic entry is that you should never consider the diagnostic entries
when deciding whether or not an article has already passed through a given
site, and therefore need not be relayed back to it. That suggests that
it should be immediately obvious, somehow, which entries are diagnostic
and which not (a slight generalization of what Russ was wondering about).

So here is an alternative syntax which might achieve that:

   path = "Path" ":" SP [FWS] *( path-entry "!" [FWS]) tail-entry [FWS] CRLF
   path-entry = path-identity [ "!" "POSTED" ] [ "!" / path-source ]
   path-source = path-address ( ".SEEN" / ".MATCH" / ".MISMATCH" )
   path-identity = path-address
   tail-entry = path-address

where path-address is any domain, IP address or UUCP-like address, and is
used for all of path-identity, path-source and tail-entry. That might be
changed later, but it will do for now.

In the following examples, "upstream.example.com" is a site which has
just sent an article (or so it is claimed) to "downstream.example.com".
"123.123.123.123" is the IP address from which downstream received it,
which may or may not be the IP address of upstream. The examples show the
full <path-entry> inserted by downstream, followed by (at least the start
of) the <path-entry> inserted by upstream.

1. downstream.example.com!123.123.123.123.SEEN!upstream.example.com...
2. downstream.example.com!123.123.123.123.MISMATCH!upstream.example.com...
3. downstream.example.com!123.123.123.123.MATCH!upstream.example.com...
4. downstream.example.com!!upstream.example.com...
5. downstream.example.com!upstream.example.com...

All the diagnostic <path-source>s are immediately distinguishable from the
<path-identity>s by the presence of a big uppercase keyword.

Even if you thought 123.123.123.123 was a site to which you were not
supposed to send articles, no current implementation would recognize it as
such in any of those examples.

Ex1. Downstream recorded that 123.123.123.123 was the source of the
article "just for the hell of it". It makes no claim to have verified that
it belonged to upstream.

Ex2. Downstream recorded that 123.123.123.123 was the source of the
article and asserted that it no way matched any known address associated
with upstream. Some current practice uses exactly that notation.

Ex3. Downstream recorded that 123.123.123.123 was the source of the
article and asserted that it was indeed a known address used by upstream.
In which case you may ask why it bothered to clutter the Path by quoting
it in full.

Ex4. Downstream noted the IP (whatever) of the source and asserts that it
was indeed a known address used by upstream. But it did not clutter the
Path with unnecessary detail. That is the notation proposed in the current
drafts, and will hopefully be the commonest case.

Ex5. Downstream made no checks and makes no assertions. That is
essentially current practice which we hope will be replaced by Ex4 over
time.

I have shown no example using the optional "!POSTED", but that would only
be inserted by an injecting agent, in which case any <path-source> shown
would presumably be the posting host.

NOTE: The use of an IP address for the <path-source> in the examples does
not imply that a corresponding domain name could not have been used
instead (indeed, it would be more user-friendly to have done so). So Ex3
could as well have been written
3. downstream.example.com!upstream.example.com.MATCH!upstream.example.com...

Observe that folding is not allowed in the middle of a <path-entry>.

OK, all that is still half baked, but hopefully it shows some interesting
possibilities.

---

From: Russ Allbery <[email-masked]>
Date: Thu, 28 Jul 2005 09:30:32 -0700

Charles Lindsey <[email-masked]> writes:

> maybe some other syntactic device) might help to indicate exactly what
> the various items in the Path were trying to say. And I saw some merit
> in the idea.

I did mean what I said, yeah.  MISMATCH implies some degree of checking
has happened and the Path failed, which strikes the wrong tone for a site
that just doesn't want to bother checking anything at all and has no idea
if the Path would have passed or not.  It seems like we have a (fairly
common) bit of semantics that we have no way to express.

> diagnostic entry is that you should never consider the diagnostic
> entries when deciding whether or not an article has already passed
> through a given site, and therefore need not be relayed back to it. That
> suggests that it should be immediately obvious, somehow, which entries
> are diagnostic and which not (a slight generalization of what Russ was
> wondering about).

I don't know that it matters if you do or not.  I have a hard time seeing
a scenario where it would realistically make a difference.

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 28 Jul 2005 18:39:14 +0200

Charles Lindsey wrote:

> path-address is any domain, IP address or UUCP-like address,
> and is used for all of path-identity, path-source and
> tail-entry. That might be changed later, but it will do for
> now.

SHOULD be changed.  I'd like to split obs-name and tail-entry
(allowing hyphen) from a proper FQDN ("mailable", no hyphen).

What Bruce, Harald, Russ, and you discuss asks for IP as 3rd
case (or only case ?) in the case of a <path-source>.

> All the diagnostic <path-source>s are immediately
> distinguishable from the <path-identity>s by the presence
> of a big uppercase keyword.

Interesting.  It certainly answers Russ' question.

> Ex2.
[...]
> Some current practice uses exactly that notation.

Not exactly, but similar.

> Ex3.
[...]
> In which case you may ask why it bothered to clutter the Path
> by quoting it in full.

Yes, example 4 and !! is shorter => *.MATCH isn't needed.

> OK, all that is still half baked, but hopefully it shows some
> interesting possibilities.

Yes.  Does somebody really need and want the "*.SEEN" case ?

If Russ' example was meant to demonstrate that ignorants will
do whatever pleases them, then hoping that they adopt "*.SEEN"
would be futile.
                          Bye, Frank

---

From: Henry Spencer <[email-masked]>
Date: Thu, 28 Jul 2005 12:58:45 -0400 (EDT)

> common) bit of semantics that we have no way to express.

Not exactly:  that's the current semantics, and it makes sense (as in
Charles's latest proposal) to express it with the current syntax, a `!'
between the names.  Anything *else* should be expressed otherwise. 

His suggestion of using `!!' to indicate what we hope will eventually be
the usual situation -- address verified -- seems good to me.  That's
supposed to be a common case, so it should be terse, yet distinct from the
old common case. 

Exactly how we indicate other situations, I'm less concerned about.

> > ...you should never consider the diagnostic
> > through a given site... That suggests that it should be immediately 
> > obvious, somehow, which entries are diagnostic and which not...
> 

I'm inclined to agree, given that any realistic convention *must* be
designed so that sites which don't make the distinction will nevertheless
almost always get the right answer as to whether the article has already
passed through a neighboring site. 

                                                          Henry Spencer
                                                       [email-masked]

---

From: Russ Allbery <[email-masked]>
Date: Thu, 28 Jul 2005 10:07:34 -0700

>> has happened and the Path failed, which strikes the wrong tone for a
>> site that just doesn't want to bother checking anything at all and has
>> no idea if the Path would have passed or not.  It seems like we have a
>> (fairly common) bit of semantics that we have no way to express.

> between the names.  Anything *else* should be expressed otherwise.

Hm, yeah, that's a good point.  That might be sufficient (which means that
we can't strongly deprecate ! since there are configurations where people
will continue using it, unless we're intending to require or strongly
recommend that everyone check paths).

---

From: Russ Allbery <[email-masked]>
Date: Thu, 28 Jul 2005 10:08:27 -0700

Frank Ellermann <[email-masked]> writes:

> If Russ' example was meant to demonstrate that ignorants will do
> whatever pleases them, then hoping that they adopt "*.SEEN" would be
> futile.

No, I'm more coming at this from the angle that large news sites with
hundreds of peers are likely to look at path checking and decide to just
not bother.

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 28 Jul 2005 20:51:56 +0200

Russ Allbery wrote:

> I'm more coming at this from the angle that large news sites
> with hundreds of peers are likely to look at path checking
> and decide to just not bother.

Okay, and why would they want to use *.SEEN ?  Your example:

A!1.2.3.4!C!...

A inserted 1.2.3.4, the IP of C, or rather the IP of a site
claiming to be C, unverified.  Charles transformed it into:

A!1.2.2.4.SEEN!C!...

Is the idea that A knows what it did, and won't send this
article back to 1.2.3.4 ?  I'm a bit lost with news paths.

Is 1.2.3.4.SEEN! some kind of private info used only by A
internally ?  Traditional path id.s separated by "!", new
path id.s separated by "!!", "!*.SEEN!", "!*.MISMATCH!" ?

Adding "POSTED" to these examples would help.  So far I
think we can forget "!*.MATCH!", because "!!" is better.

                    Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 29 Jul 2005 14:09:15 GMT

>> and decide to just not bother.

OTOH, even though they might not bother with the hard work of checking,
there is a reasonable chance that they would add '.SEEN', since there is
no rocket science involved there.

>article back to 1.2.3.4 ?  I'm a bit lost with news paths.

The idea is that, if C intends to use "C" as its <path-identity>, then it
so informs its peers, who then enter "C" into their 'sys' files (along
with any other aliases they are told about - even 1.2.3.4).

But nobody in his right mind is going to enter an identity ending in
".SEEN" into his 'sys' file, and nobody in his right mind is going to tell
his peers that he will use such an identity (but, for sure, we shall see
various trolls trying it on :-( ).

>internally ?

A might find it useful if he has to analyze some Path later on. But I
suppose the intent is that it should be useful to others outside who _can_
be "bothered to do the check". All we know is that _some_ sites are doing
it already (but without any ".SEEN" to clarify who is doing what).

>think we can forget "!*.MATCH!", because "!!" is better.

I only put "MATCH" in the set for the sake of completeness. The final text
would discourage its use without good cause.

OTOH, you need to bear in mind that the "SEEN" IP will not always be an
automatic match for the last identity shown in the Path. Suppose the
relaying takes place over a dial-up link. Then some
authentication-identity is probably the correct thing to put in the
SEEN/Whichever. Or take my case, where the relaying goes through an SSH
tunnel, and will appear to the server to have come from a different
machine than my own (which doesn't have a fixed IP anyway). Or the
transport medium is via Carrier Pigeon, so you write "Budgie.SEEN" :-) .

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 29 Jul 2005 12:33:32 GMT

>recommend that everyone check paths).

Currently (and in our drafts for many years) checking paths is a MUST.

Yes, we know full well that it will be a long time before even 95% of
Usenet sites do it, but shaming the slow implementors with the tag
"non-compliant" should provide _some_ incentive for them to catch up, and
it provides ammunition to throw at them in the news.admin.* groups. And I
think we agree that widespread adoption would make it much harder for
Mallet and his cronies.

But that point is discussable.

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Fri, 29 Jul 2005 13:35:34 -0400

Charles Lindsey wrote, in part...

> it provides ammunition to throw at them in the news.admin.* groups.

I think it is a grave problem when drafts are written by anyone who thinks
that "shame of non-compliance" is a motivator by itself.

A specification gets voluntary compliance not because it comes from
the IETF, or disregarding it is shameful, but because implementors
agree with the consequences and benefits as explained by the specification.

I think this is a fundamental point of understanding that the Chair
gets to set, in accordance with RFC2119.  Once that is set properly,
we can go about removing all the MUST/MUST NOTs which have nothing to
do with interoperability, and replacing them with explanations of
consequences and benefits.

Implementors must be able to make ENGINEERING choices of "benefit vs cost"
instead of SOCIAL choices of "compliance vs shame."

I think John is doing a fine job of explaining this to Charles.  The
chair even ruled that RFC2119 terms must be used appropriately.
So, I am wondering why the chair isn't policing that decision when
it comes up.

Can the chair either tell John why Charles can use MUST/MUST NOT when there
are no interoperability problems (e.g. in the complaints email text), or
tell Charles to start regarding the instructions in RFC2119 and stop
using them to create shame.

Letting participants go back and forth when there is a fundamental
disagreement about whether there is an interoperability issue is not
productive.

---

From: Henry Spencer <[email-masked]>
Date: Fri, 29 Jul 2005 15:12:15 -0400 (EDT)

On Fri, 29 Jul 2005, Forrest J. Cavalier III wrote:
> ...Once that is set properly,
> do with interoperability...

RFC 2119 is quite explicit that achieving minimal interoperability is not
the only reason to use MUST/MUST NOT; they are also appropriate "to limit
behavior which has potential for causing harm".

There are obviously judgement calls in how to apply this, but actual
judgement is required, not just mindless repetition of "nothing to do with
interoperability, so take it out". 

                                                          Henry Spencer
                                                       [email-masked]

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 29 Jul 2005 21:29:20 +0200

> A might find it useful if he has to analyze some Path later
> on. But I suppose the intent is that it should be useful to
> others outside who _can_ be "bothered to do the check". All
> we know is that _some_ sites are doing it already (but
> without any ".SEEN" to clarify who is doing what).

Hmm...  Lots of strange things are done for obscure reasons,
if we specify it "we" should at least know what it's good for.

I better stay out of this and discuss the magic SP with Bruce.
Don't make the syntax too baroque without good reason, please.

> Or the transport medium is via Carrier Pigeon, so you write
> "Budgie.SEEN" :-) .

Now if that helps Budgie or somebody somehow...  very obscure.

                            Bye, Frank

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Fri, 29 Jul 2005 16:05:57 -0400

> interoperability, so take it out". 

How about "avoids no harm, has no practical benefit, and has nothing to do with
interoperability, and is outside out charter?"

Sure, decisions have to be made on some points.  The Chair can set how we
arrive at that decision.

I say it is clear that RFC2119's "with care and sparingly" means that anyone who
is called to explain why MUST/MUST NOT is appropriate, must defend the use by
showing what harm or interoperability issues there are IN THE REAL APPLICATION,
not just theoretical.  And must show that there IS NO PRACTICAL ALTERNATIVE to the
intended effect.

The Chair can set a policy that anyone can question an RFC2119 imperative, and
if no one can defend its use, it gets removed.  Too many people in this WG
argue that documents have inertia, and longstanding text is assumed to be above
dispute.

RFC2119 says this...
   "They must not be used to try to impose a particular method on implementors
   where the method is not required for interoperability."

Concrete example....
Having a way to report bad behavior to injecting site admins is a good thing, but
the abuse@ mandates (that Charles advocates) are "imposing particular methods",
which is an expressly forbidden use by RFC2119.

Additionally requiring SMTP mailboxes is outside our charter/protocol/article format.

It is a stretch to say that harm is caused by having to dig into a header
for a complaints address.  Until we have digitally signed path header fields, everything
in a header field is suspect. (There is all sorts of preloading going on today, and no
way to stop or detect it.)  Encouraging reliance on being able to get at an injection
site admin by looking at the path could cause more harm than not specifying a method.
There is a reason for all of the private opaque injection info index data, and
generally complaining to any abuse@ address about a particular article will be
ignored if you do not include the opaque info from all the headers so that the
origin of the article can be verified.

And who thinks the Bad Boys and Girls on Usenet are going to be nice and
set up an effective abuse@ system?  If they aren't voluntarily going to
follow netiquette, will a "MUST" in an IETF draft give them pause?  I don't
object to text that says "users expect to be able to report problems", I object
when someone thinks that the 4 letters "MUST" are so cheap they can be used
anywhere.

I think the Chair should provide guidance in specific situations when someone
questions that RFC2119 mandates are (in)appropriate.  John questioned.  Charles
has refused to respond.  It is the chair's responsibility to restore order
by telling John the reasons he is wrong to question, or Charles the reasons
he is wrong to refuse to respond with justification.