Subject: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT? (fwd)

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 29 Jun 2005 13:24:24 +0200

If nobody wants to say anything about this, I'm going to close it with a 
"no change needed". I can't accept a change based on a single voice and no 
raised objection.

                      Harald

---------- Forwarded Message ----------
Date: onsdag, juni 08, 2005 13:32:05 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Cc:
Subject: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT?

New ticket:

At the moment, section 3.2.2 of usefor-04 says:

 o Comments in CFWS between message identifiers can cause
 interoperability problems, so comments SHOULD NOT be generated,
 but MUST be accepted.

 --On 3. juni 2005 01:45 -0400 "Forrest J. Cavalier III"
<[email-masked]> wrote:

 > BTW, CFWS in References is a silly unnecessary complication. (CFWS
 in
 > any structured header
> is a bad idea,IMNSHO. - I implement article processing software.)
 >
 > usepro Transitional Arrangements even admits comments in References
 will
 > break existing software.
 >
 > So, why are comments going to be allowed in the References header?
 >
 > Is the idea to make References irrelevant and unreliable, and then
 we
 > don't
> care that it cannot be parsed by legacy software?

 While we cannot change RFC 2822, it may be logical to change SHOULD
NOT to MUST NOT.

---------- End Forwarded Message ----------

---

From: Russ Allbery <[email-masked]>
Date: Wed, 29 Jun 2005 10:36:55 -0700

Harald Tveit Alvestrand <[email-masked]> writes:

> "no change needed". I can't accept a change based on a single voice and
> no raised objection.

I don't think the interoperability problem caused by comments in
References is sufficient to really justify a MUST NOT.  I think comments
in References is very silly, but it's not our silliness, it's RFC 2822.
As a mail to news gateway author, I'd rather push news towards being able
to cope with RFC 2822 silliness rather than being forced to do more and
more fixups at the conversion step.

So I'm happy with the existing text.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Bruce Lilly <[email-masked]>
Date: Thu, 30 Jun 2005 09:09:57 -0400

>  interoperability problems, so comments SHOULD NOT be generated,

If in fact there are "interoperability problems", they should be described,
and if there is a problem meeting BCP 14 criteria, then MUST NOT would be
indicated.  However, the only issue appears to be that some UAs (no other
software has need to parse a References field) are not compliant with the
specifications (RFC 822, as referenced by RFCs 850 and 1036, has always
permitted comments), and that isn't an interoperability problem of the
sort described in BCP 14.

s/cause interoperability problems/be misinterpreted by non-conforming parsers/
would be more accurate, avoiding the apparent conflict with BCP 14 w.r.t.
serious interoperability issues. 

>  --On 3. juni 2005 01:45 -0400 "Forrest J. Cavalier III"
[...]

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 30 Jun 2005 21:15:29 +0200

Bruce Lilly wrote:

> the only issue appears to be that some UAs (no other
> software has need to parse a References field) are not
> compliant with the specifications

That's not exactly the point:

A few of these "UAs" can be Web interfaces trying to display
some sort of "threads" using (among others) the References.

Some of these broken UAs can mangle the References in wild and
wonderful ways causing more wonderful side effects with other
UAs, which can normally handle all syntactically well-formed
References.

> that isn't an interoperability problem of the sort described
> in BCP 14.

If one UA gets it wrong and that causes another UA to crash it
is only a SNAFU.  But if the crshing UA is a Web interface or
another piece of software running in unattended mode it's bad.

> s/cause interoperability problems/be misinterpreted by
> non-conforming parsers/ would be more accurate

IBTD, it's about conforming parsers confronted with the invalid
input caused by another non-conforming parser elsewhere.  If
what you said boils down to a "don't replace the SHOULD NOT
by a MUST NOT" I'd support it.
                               Bye, Frank

---

From: Bruce Lilly <[email-masked]>
Date: Thu, 30 Jun 2005 17:20:56 -0400

> UAs, which can normally handle all syntactically well-formed

Do you mean when creating a f^H(avoiding controversial term) response?
If so, and the UA/posting agent/(controversial word) agent generates
invalid syntax, then the injection agent ought to reject that response
article, avoiding damage.  If not, then how does such mangled content
get from one UA to another?

I believe that rejecting invalid articles at the injection agent is
consistent with WG consensus, though USEPRO section 7.2 doesn't
actually say that an injection agent may reject a non-conforming
article, even though it is tasked with ensuring conformance.
Perhaps forwarding to a moderator (permitted as the sole alternative
to injection)... but I digress.

We are after all talking about parenthesized comments, which
have always been part of RFC 822 syntax and aren't exactly rocket
science.

> another piece of software running in unattended mode it's bad.

I think the answer is to reject non-articles at the earliest
opportunity (injection agent). I.e. prevent the rogue UA/posting
agent/whatever from causing damage; the injection agent enforces the
syntax rules.
 
> input caused by another non-conforming parser elsewhere.

In that hypothetical case, it's not the comment that's causing a
problem, it's something else.  So it would be incorrect to say
that the comment(s) "cause interoperability problems".  The
"something else", whatever that might be, either results in a
syntactically valid field or not.  If not, then it along with
the article containing it is rejected (or forwarded to some
moderator) instead of being injected.  If the field is
syntactically valid, what is the problem?

> If 

What it boils down to is: "interoperability problems" implies "MUST NOT".
The original suggestion was to therefore replace SHOULD NOT with MUST
NOT.  If (and only if) I can be convinced that there really is an
interoperability problem caused by a mere comment, I'll support that
change.  I don't see such an interoperability problem, so my counter-
proposal is to avoid the conflict by replacing "interoperability
problems" with something more accurate, and I have suggested *specific*
text for that replacement, with some sort of relevant argument to
support that suggestion (the argument is NOT "because unripe bananas
are green").  I'm ambivalent about leaving the "SHOULD NOT"; comments
are for human consumption, and few people care about reading
References fields, even if their UA displays them -- of course adding
human-readable comments that are unlikely to be read isn't a great
idea.  For that matter, the entire text could be elided; lots of
non-conforming UAs will misparse lots of things causing no perceptible
problems elsewhere (provided injection agents reject invalid syntax
that such a UA might generate in a response) -- if we add text to cover
every such possibility we'll be back to 115 pages, at least if the WG
stays active for another 8 years so any masochists with the stomach for
it can do so... 

I guess we have at least the following options:
1. no change
2. s/SHOULD NOT/MUST NOT/
3. s/cause interoperability problems/be misinterpreted by non-conforming
   parsers/
4. elide the entire text

Although I suggested the third option, on further consideration I'd like
to switch to the fourth (with the third as a fallback)

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 01 Jul 2005 01:04:27 +0200

Bruce Lilly wrote:

> 3. s/cause interoperability problems/be misinterpreted by
>    non-conforming parsers/
> 4. elide the entire text

> Although I suggested the third option, on further
> consideration I'd like to switch to the fourth (with the
> third as a fallback)

We have no general rule stating that unnecessary comments
in a field not designed to be displayed are a stupid idea.

IIRC 2822 only says "if you must do this use at least the
end of a syntactical unit" (or similar), that doesn't help
for a <msg-id-list>, the only unit in sight is a <msg-id>.

Without a general rule removing the entire text (your 4)
is not among my preferences.  It's also very different
from Forrest's MUST NOT (your 2).  If that would be only
between you and me we'd end up with your 3rd option.

 From Harald's "hat on" POV it's probably "no change" (1).

                        Bye, Frank

---

From: Bruce Lilly <[email-masked]>
Date: Thu, 30 Jun 2005 20:22:49 -0400

> in a field not designed to be displayed are a stupid idea.

To borrow a statement from another list, we also don't have a
general rule against drinking gasoline.  Sometimes the obvious
does NOT need to be stated.

> for a <msg-id-list>, the only unit in sight is a <msg-id>.

2822 says (in accordance with RFC 2234 sect. 3.1):
  msg-id          =       [CFWS] "<" id-left "@" id-right ">" [CFWS]
which explicitly says where CFWS may appear.  BTW, there is no msg-id-list,
-list rules usually imply comma separators, and there are none in
References or In-Reply-To.  822/2822 specify that comments are semantically
equivalent to a space character, and a space character is a typical
separator between msg-ids.
 
> is not among my preferences.

I find it hard to get worked up about a lack of second-hand
interoperability problems related to comments which are rarely
inserted in a References field.  The only UAs I know of that do that
are the ones that used to put (badly broken) phrase syntax (which is
MUST NOT generate in 2822) in References fields.  Those broken
messages are still around, and in the grand scheme of things a
you-don't-have-to-parse-obs-references-because-MUST=MAY contortion is
going to do a great deal more harm than a comment.  Well, that's the
law of unintended consequences for you.

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Thu, 30 Jun 2005 22:07:41 -0400

> to switch to the fourth (with the third as a fallback)

Based on recent discussion, and after seeing how long this went unresolved,
I want to suggest a fifth alternative that takes care of one of my concerns that
people start putting structured comments in references, or start using
them to XSS web interfaces....

5. Agents MAY remove comments from References.

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 01 Jul 2005 04:09:25 +0200

Bruce Lilly wrote:

> To borrow a statement from another list, we also don't have
> a general rule against drinking gasoline.  Sometimes the
> obvious does NOT need to be stated.

In that case it's not so obvious:  We know that some UAs have
difficulties with excessively long References.  We know that
some servers reject excessively long References.  We know that
I might be able to trim References manually, but we're even
less confident about UAs.  We have a simple followup-and-trim
algorithm, but we're far from sure that all UAs get it right
in the presence of comments.  And of course we know this...

> msg-id = [CFWS] "<" id-left "@" id-right ">" [CFWS]

...and so the implementation strategy "grab <msg-id> content
and add it to the old <msg-id-list> separated by CFWS, then
trim it" might introduce [CFWS] found in the <msg-id>.  Stupid
or not, shit happens.  In that case it's not really broken, it
is only stupid, potentially causing harm for the next (old) UA
that didn't expect the legal [CFWS] => the first UA SHOULD NOT.

> BTW, there is no msg-id-list,

In some cases where I disagree with Charles' ABNF I use my own:

   references  = "References:" SP msg-id-list CRLF
   msg-id-list = [CFWS] *( msg-id CFWS ) msg-id [CFWS]

> -list rules usually imply comma separators

A list can be:  item *( delim item )  or:  *( item delim ) item

Among other ideas, let's not start with LISP.  And delim can be
what it takes to separate items, if you're lucky nothing at all,
in fact this _could_ work for a msg-id-list, but we want a CFWS.

Check out <path-list> (no comma), <key-value-list> (semicolon,
in the SPF-1 RFC proposed here to get <injection-info> right),
I won't dig through more RFCs to find lists not using a comma.

> interoperability problems related to comments which are
> rarely inserted in a References field.

Not rarely, it used to be NEVER based on s-o-1036, and we want
to change this adopting the RfC 2822 format. Carefully and not
wholesale, e.g. "we" (TINW) want a CFWS instead of 2822 [CFWS].

> in the grand scheme of things a
> you-don't-have-to-parse-obs-references-because-MUST=MAY
> contortion is going to do a great deal more harm than a
> comment.  Well, that's the law of unintended consequences
> for you.

That's "when Bruce starts to rant there are no more serious
bugs and showstoppers" for me... <gd&r>

                           Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 01 Jul 2005 04:54:51 +0200

> 5. Agents MAY remove comments from References.

BTW, are you sure that we want this in USEFOR ?  The proper
way to generate and trim References is in USEPRO.  And if we
move this issue to USEPRO we could combine Bruce's 4th point
"delete SHOULD NOT" with your 5th point "agents MAY remove".

All we then need is a hint that the separator used to be SP,
now it's CFWS, and that's not the same as 2822 [CFWS].

Here's an idea of 4+5 in USEFOR-05

=== OLD ===
   o  Message identifiers MUST be separated with CFWS.

   references      =  "References:" SP [CFWS] msg-id *(CFWS msg-id)
                      [CFWS] CRLF
=== NEW ===
   o  Message identifiers MUST be separated with CFWS, typically a
      space or FWS.

   references      =  "References:" SP msg-id-list CRLF
   msg-id-list     =  [CFWS] *( msg-id CFWS ) msg-id [CFWS] CRLF
=== END ===

Your point 5 going to USEPRO, is that okay ?  Bye, Frank

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Thu, 30 Jun 2005 23:46:51 -0400

> Your point 5 going to USEPRO, is that okay ?  Bye, Frank

USEFOR can CFWS in ABNF without limit, but in USEPRO have this
idea....
       Comments in CFWS between message identifiers can cause
       interoperability problems, so comments SHOULD NOT be generated,
       but MUST be accepted, and MAY be removed.

I know, "MUST accept+MAY remove" is probably unheard of, but that MAY
puts "innovators" on notice that they can't expect comments to propagate, and
helps implement a messy place that XSS scripting could happen in
web UIs.  Should a web UI be forced to accept (and
display?) a hacked up References field in an attempt to XSS?  There
have been too many clever XSS and "response splitting" style attacks to
think that all HTML that gets a green-light in one browser is not
a XSS in another.

Sure, avoiding XSS is a problem for a web agent to handle CFWS in
any header fields, but References is likely to go through an additional
level of processing that gets A HREFs added (to link back to
precursors.)

No, I don't like it that there might be two non-identical copies of
an article propagating, but I suppose you can limit the
MAY remove to non-transport agents.

Yes, this is probably a weird idea, but please find a real reason to
dismiss it, not just that it is weird to write a "MUST accept+MAY
remove."

Date: Fri, 1 Jul 2005 01:28:41 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Sender: [email-masked]

On Thu, 30 Jun 2005, Charles Lindsey wrote:
> Legacy software does all sorts of things which we don't want them to do
> any more.
> Therefore, we write our standard so that those legacy implementations
> become non-compliant (because that is a good carrot to encourage them to
> be upgraded).
> OTOH, of course, we so arrange things (by making transitional/fallback
> arrangements where necessary) that the legacy software can continue to
> interoperate at least for the medium term...

Exactly.  It is important that legacy software continue to *work*, because
there is a lot of it around, and even with the best intentions, getting
even a large fraction of it upgraded will take a while.  But there is no
reason to declare it compliant, and indeed every reason not to.

Backward compatibility is a technical issue, not a compliance issue.  It
does not require us to permanently endorse every old mistake, only to
provide a transition path, so that software can move toward future full
compliance while remaining interoperable with the old mistakes for now. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Typo (was: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT?)
Date:  Fri, 01 Jul 2005 06:36:48 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 8
Sender: [email-masked]

>>> o Message identifiers MUST be separated with CFWS,
>>>    typically a space or FWS.
>>>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT? (fwd)
Date:  Fri, 01 Jul 2005 06:18:33 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 37
Sender: [email-masked]

>> Your point 5 going to USEPRO, is that okay ?

> USEFOR can CFWS in ABNF without limit, but in USEPRO have
> this idea....
>      Comments in CFWS between message identifiers can cause
>      interoperability problems, so comments SHOULD NOT be
>      generated, but MUST be accepted, and MAY be removed.

> I know, "MUST accept+MAY remove" is probably unheard of,

No, it's almost like RFC 2821 about source-routes "MUST accept,
SHOULD ignore, MAY respect".

> this is probably a weird idea, but please find a real reason
> to dismiss it

I've no reason to dismiss it, we already discussed "MAY remove"
in conjunction with trimming.  Charles started a quest for the
"semantical content" (= no comments, but then we stumbled over
the problem with backslashes within <no-fold-2822-crap>).  And
somebody else also wanted to strip the comments in an attempt
to stay within our obscure unfolded length limit 998.

Strip comments and reinject, that could be evil, I've no idea
how stuff like a signed header works.  But you didn't propose
to strip and reinject, so that point is probably moot.  Bye

Date: Thu, 30 Jun 2005 23:46:51 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT? (fwd)
Sender: [email-masked]

> Your point 5 going to USEPRO, is that okay ?  Bye, Frank

USEFOR can CFWS in ABNF without limit, but in USEPRO have this
idea....
       Comments in CFWS between message identifiers can cause
       interoperability problems, so comments SHOULD NOT be generated,
       but MUST be accepted, and MAY be removed.

I know, "MUST accept+MAY remove" is probably unheard of, but that MAY
puts "innovators" on notice that they can't expect comments to propagate, and
helps implement a messy place that XSS scripting could happen in
web UIs.  Should a web UI be forced to accept (and
display?) a hacked up References field in an attempt to XSS?  There
have been too many clever XSS and "response splitting" style attacks to
think that all HTML that gets a green-light in one browser is not
a XSS in another.

Sure, avoiding XSS is a problem for a web agent to handle CFWS in
any header fields, but References is likely to go through an additional
level of processing that gets A HREFs added (to link back to
precursors.)

No, I don't like it that there might be two non-identical copies of
an article propagating, but I suppose you can limit the
MAY remove to non-transport agents.

Yes, this is probably a weird idea, but please find a real reason to
dismiss it, not just that it is weird to write a "MUST accept+MAY
remove."

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT? (fwd)
Date:  Fri, 01 Jul 2005 04:54:51 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 34
Sender: [email-masked]

=== OLD ===

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT?  (fwd)
Date:  Fri, 01 Jul 2005 04:09:25 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 61
Sender: [email-masked]

                           Bye, Frank

Date: Thu, 30 Jun 2005 22:07:41 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT? (fwd)
Sender: [email-masked]

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1047 Path-identity: A proposal (perhaps)
Date: Thu, 30 Jun 2005 21:49:12 -0400
Cc: Frank Ellermann <[email-masked]>
Sender: [email-masked]

On Thu June 30 2005 20:51, Frank Ellermann wrote:
> 
> Bruce Lilly wrote:
> > doesn't address (no pun intended) where IP address literals
> > are or are not allowed.
> 
> It was based on Charles' revised syntax for a <path-identity>:
> 
> | path-identity = ( ALPHA / DIGIT )
> |                 *( ALPHA / DIGIT / "-" / "." / "_" ) /
> |                 IPv4address / IPv6address    ; see [RFC 3986]

That's inconsistent with Harald's "diagnostic stuff".

> > That text (and the draft ABNF) implies that
> >   Path: MISMATCH!example.net!tail-entry
> > is a valid complete field, which might not be what is
> > intended; the text doesn't say that there needs to be
> > anything to the left of MISMATCH.
> 
> Then it was wrong, "!MISMATCH!" is the same idea as "!!", only
> for a mismatch.  In my hardcore nitpicking mode it should be:

I guess it's up to somebody who cares for the newfangled "feature"
to open a ticket proposing a specific text change...
 
> path-identity "!" path-identity "!MISMATCH!" path-list
> 
> a!b!MISMATHCH!c!...!not-for-mail as in "I got this starting
> with c!... but I think it should be b, anyway I am a, enjoy".
> 
> Or should this be a!!b!MISMATCH!c!...!not-for-mail (?)  Bye.

Good question; sounds reasonable to me.  Anyway, I'm waiting  for
resolution of the issues before ABNF-wrangling.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 Path-identity: A proposal (perhaps)
Date:  Fri, 01 Jul 2005 02:51:13 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 58
Sender: [email-masked]

Bruce Lilly wrote:

> That still permits foo!!tail-entry, foo!MISMATCH!tail-entry

Yes, and I wasn't consequent with the [FWS], next attempt:

 path-list      = [FWS] path-identity [FWS]
                  *( path-delimiter [FWS] path-identity [FWS] )
                  ( "!" [ "POSTED!" ] ) [FWS] tail-entry [FWS]

 path-delimiter = "!" [[ "MISMATCH" / "POSTED" ] "!" ]

That's a bit convoluted, maybe a simple list of delimiters is
better, see above: 
                  ( "!" / "!POSTED!" ) [FWS] tail-entry [FWS]

 path-delimiter = "!" / "!!" / "!MISMATCH!" / "!POSTED!"

> doesn't address (no pun intended) where IP address literals
> are or are not allowed.

It was based on Charles' revised syntax for a <path-identity>:

| path-identity = ( ALPHA / DIGIT )
|                 *( ALPHA / DIGIT / "-" / "." / "_" ) /
|                 IPv4address / IPv6address    ; see [RFC 3986]

> I made some quick notes this morning:
> path-identity "!POSTED" needs to be a group

Okay, path-identity "!POSTED!" tail-entry catches the normal
case, path-identity "!POSTED!" path-list is for reinjection.

> path-identity "!!" path-identity needs to be a group

Also okay, I have path-identity "!!" path-list, and no way to
combine it with a tail-entry.

> "MISMATCH!" path-identity needs to be a group

Dito as path-identity "!MISMATCH!" path-list, not with a tail.

> That text (and the draft ABNF) implies that
>   Path: MISMATCH!example.net!tail-entry
> is a valid complete field, which might not be what is
> intended; the text doesn't say that there needs to be
> anything to the left of MISMATCH.

Then it was wrong, "!MISMATCH!" is the same idea as "!!", only
for a mismatch.  In my hardcore nitpicking mode it should be:

path-identity "!" path-identity "!MISMATCH!" path-list

a!b!MISMATHCH!c!...!not-for-mail as in "I got this starting
with c!... but I think it should be b, anyway I am a, enjoy".

Or should this be a!!b!MISMATCH!c!...!not-for-mail (?)  Bye.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT?  (fwd)
Date: Thu, 30 Jun 2005 20:22:49 -0400
Cc: Frank Ellermann <[email-masked]>
Sender: [email-masked]

 

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date:  Fri, 01 Jul 2005 02:10:30 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 50
Sender: [email-masked]

Bruce Lilly wrote:

> o there exist archived messages

Sure, no reason to consider e.g. the A-News format.

> o combined mail/news UAs (the norm; Communicator,
>   Thunderbird, Outlook Express, pine, etc.) need
>   to deal with standard 2822 semantics for the
>   standard 2- and 3-letter zone abbreviations.

Sorting by date, yes, but in that case "MUST accept"
doesn't necessarily imply "interpret correctly".

> o of course, "GMT" was also declared obsolete four
>   years ago...

Unlike the rest of this zoo, that was declared obsolete
11 (eleven) years ago:

| timezone   = "UT" / "GMT"
|            / ( "+" / "-" ) hh mm [ space "(" zone-name ")" ]
[...]
| This is a restricted subset of the MAIL date format.
[...]
| The timezone name in parentheses, if present, is a comment;
| software MUST ignore it, except that reading agents might
| wish to display it to the reader.  Timezone names other than
| "UT" and "GMT" MUST appear only in the comment.
|
|  NOTE: Attempts to deal with a full set of timezone names
|  have all foundered on the vast number of such names in use
|  and the duplications (for example,  there  are at least FOUR
|  different timezones called "EST" by somebody).  Even the
|  limited set of North American zone names authorized by MAIL
|  is subject to confusion and misinterpretation.  Hence the
|  flat ban on non-UT timezone names except as comments.
|
|  NOTE: RFC 1036 specified that use of GMT (aka UT, UTC) was 
|  preferred.  However, the local time (in the poster's
|  timezone) is arguably information of possible interest to
|  the reader, and this requires some indication of the
|  poster's timezone.  Numeric offsets  are an unambiguous way
|  of doing this, and their use was indeed sanctioned by RFC
|  1036 (that is, this is a change of preference only).

Now is it already time again to go through the WG Charter,
find s-o-1036 as relevant, etc., the whole drill for every
minor point with a minor difference from RFC 2822 ?  Bye

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date: Thu, 30 Jun 2005 19:37:18 -0400
Cc: Graham Drabble <[email-masked]>
Sender: [email-masked]

On Thu June 30 2005 18:01, Graham Drabble wrote:

> FWIW I'm quite happy to allow them to do so. Nothing critical is likely 
> to break because something has the time +-12hrs so making people code 
> exceptions for things that were declared obsolete (and MUST NOT 
> generate) four years a go seems silly.

A few points to bear in mind:
o if we paid due attention to backward compatibility, existing software
  will be largely conforming, and we're not making people code exceptions
  because the 822 2- and 3-letter zone abbreviations have long been
  required
o there exist archived messages (e.g. Frank's favorite non-Netnews but
  news-like mailing list interface gmane).  Many are more than four years
  old
o while an Injection-Date field could not have been generated 4 years ago
  (as it was not then, and is not yet, standardized), that doesn't apply
  to Date, Expires, etc.  There remains the possibility of a special
  exemption for Injection-Date.
o FAQs and other old material may be periodically reposted with a new
  Injection-Date field, but with a Date field corresponding to the last
  change.  That applies to things like a response to "can somebody please
  post a copy of..." requests also.
o combined mail/news UAs (the norm; Communicator, Thunderbird, Outlook
  Express, pine, etc.) need to deal with standard 2822 semantics for the
  standard 2- and 3-letter zone abbreviations.
o of course, "GMT" was also declared obsolete four years ago...

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1047 Path-identity: A proposal (perhaps)
Date: Thu, 30 Jun 2005 19:14:35 -0400
Cc: Frank Ellermann <[email-masked]>
Sender: [email-masked]

On Thu June 30 2005 17:44, Frank Ellermann wrote:

> Some of Bruce's points are not yet covered, MISMATCH!MISMATCH
> etc., maybe we have to declare these beasts as special cases
> of a path-delimiter, brain-storming:
> 
> 
>  path-list = [FWS] path-identity 
>              *( path-delimiter [FWS] path-identity [FWS] )
>              path-delimiter tail-entry [FWS]
> 
>  path-delimiter = "!" / "!!" / "!MISMATCH!" / "!POSTED!"
> 
> No more path-keyword.  Does that solve Bruce's problem ?  Bye

That still permits foo!!tail-entry, foo!MISMATCH!tail-entry, and
doesn't address (no pun intended) where IP address literals are or
are not allowed.

I made some quick notes this morning:
path-identity "!POSTED" needs to be a group
path-identity "!!" path-identity needs to be a group
"MISMATCH!" path-identity needs to be a group 
based on the text description following the ABNF in the usefor draft.
That text (and the draft ABNF) implies that
  Path: MISMATCH!example.net!tail-entry
is a valid complete field, which might not be what is intended; the
text doesn't say that there needs to be anything to the left of
MISMATCH.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT?  (fwd)
Date:  Fri, 01 Jul 2005 01:04:27 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 29
Sender: [email-masked]

                        Bye, Frank

To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
From: Graham Drabble <[email-masked]>
Date: Thu, 30 Jun 2005 23:01:08 +0100
Organization: Home
Sender: [email-masked]

On 30 Jun 2005 Bruce Lilly <[email-masked]> wrote in
news:[email-masked]: 

> My interpretation is that some WG members might favor that for
> Injection-Date specifically (although no text has been proposed
> even though there is an open ticket for that purpose), but I see
> no discussion that explicitly supports interpreting "EST" as
> unknown in the legacy fields.

FWIW I'm quite happy to allow them to do so. Nothing critical is likely 
to break because something has the time +-12hrs so making people code 
exceptions for things that were declared obsolete (and MUST NOT 
generate) four years a go seems silly. My view would be to go with 
something like:

Agents MAY accept the obs-zone syntax specified in RFC2822 section 4.3 
but MUST NOT generate it. Any zone that the agent is unable to parse 
SHOULD be treated as -0000.

This way even if they don't follow the MAY accept they SHOULD still get 
GMT right.

-- 
Graham Drabble
http://www.drabble.me.uk/

Date: Thu, 30 Jun 2005 15:09:51 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>To describe a substantial part of your audience as "irrelevant" 

He didn't describe a substantial part of our audience as irrelevant. He 
("Forrest J. Cavalier III") said:

]...BUT WE HAVE NO CONCERN for people who refuse to read and
]understand NORMATIVE references.  We can totally ignore such people when
]writing, because they are irrelevant.

People who refuse to read normative references have no business
implementing news software according to our standard, and are thus not our
audience. Specifically, when we explicitely point people to RFC2822 as a 
basis for our standard, those who refuse to read RFC2822 are irrelevant. 

We cannot force them to read anything, but neither are we bound to 
consider them when we write the standard.

If your attitude is that we must accept every person who refuses to read 
RFC2822 and hold their tiny little hands through the process of 
implementing news software, then you have no business editing a news 
standard. You will be too likely to stuff in extraneous material and 
rewrite perfectly acceptable material in your goal to protect the ignorant 
from themselves.

Date: Thu, 30 Jun 2005 17:58:48 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: Harald Tveit Alvestrand <[email-masked]>
CC: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

Charles Lindsey wrote:

> In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:
> 
> 
>>Charles Lindsey wrote:
>>
>>>I pointed out that in the real world implementors and other readers DO
>>>take shortcute when reading documents. That is a fact of life. You have to
>>>live with it.
> 
> 
>>No I do not have to live with it.  As has been said so many times: we
>>need a clear, complete, unambiguous specification that is as easy to read
>>as possible, BUT WE HAVE NO CONCERN for people who refuse to read and
>>understand NORMATIVE references.  We can totally ignore such people when
>>writing, because they are irrelevant.
> 
> 
> To describe a substantial part of your audience as "irrelevant" is no way
> to write any document, whether technical or otherwise.
> 
> If that is your attitude, then we will just have to agree to differ.
> 
No, we don't have to agree to differ on this either.

I request the chairs issue a clear statement that we will use RFC2119
imperatives and when used we expect ALL implementors to understand them as
defined in RFC2119.  We will also expect them to understand English.

I request the chairs issue a clear statement that we expect all implementors
claiming conformance are expected to read and understand NORMATIVE references.
(I thought we already had that declaration of the obvious, but guess not...)

I request the chairs issue a clear statement that we will repeat and highlight
text from NORMATIVE references only when we have reason to believe that our
draft text is so difficult to write clearly that it is likely to mislead
an implementor into violating a normative reference.

To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 30 Jun 2005 14:53:51 -0700
Sender: [email-masked]

Forrest J Cavalier <[email-masked]> writes:

> Sure, Anyone who writes or maintains date parsers will tell you there is
> all sorts of cruft that you see in Date fields.  That's why INN's date
> parser is written in yacc, and at 500+ lines it's a nightmare to
> modify/understand.  Who wants to touch that?

Thankfully, it's not that bad any more.  It's now pure C, and including
the code to parse NNTP dates ("20050630 174410" sorts of things that are
completely different than what we're talking about) and code to generate
the Date header, it's now:

   869   288   |date.c
----------------
   869   288    total lines/statements

   258  lines had comments        29.7 %
    38  comments are inline       -4.4 %
    93  lines were blank          10.7 %
    18  lines for preprocessor     2.1 %
   538  lines containing code     61.9 %
   869  total lines              100.0 %

The yacc stuff couldn't even handle the full RFC 2822 obs syntax, and
allowed all sorts of nonsense that really no one should ever use in a
Usenet Date (and no one does).

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 Path-identity: A proposal (perhaps)
Date:  Thu, 30 Jun 2005 23:44:56 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 57
Sender: [email-masked]

Charles Lindsey wrote:

> So please can we be as brief as possioble in USEFOR, and
> point to USEPRO for the gory details

Hear, hear!  SCNR

> the term <path-identity> needs to include everything that
> may occur in the list, except the two keywords

...or the <tail-entry>.

> So it would be better as:
[...]

ACK.

> Here, now, is the text currently proposed for USEPRO.
[...]

>| To ensure that a <path-identity> provides a unique identity
>| for the news-server concerned, it SHOULD be one of:

How about "it MUST be" ?  Or even simply "it is" ?!?
[...]

>| The FQDN of a news-server is "mailable" if its
>| administrators can be reached by email using both of the
>| forms "usenet@<FQDN>" and "news@<FQDN>", in conformity
>| with [RFC 2142].

Maybe say 'forms "usenet@" and "news@" at the FQDN represented
by the <path-identity>, in conformity with [RFC 2142].'  Issue:
I'm not sure about using <FQDN> if that's no term in the ABNF.

>| "abuse@<FQDN>" MUST also be available

Dito, '"abuse@" address MUST' etc., if your <FQDN> doesn't fly.

>> Does this make sense?
> Not yet :-(

Getting much better, limiting ":" to IPv6 is a step forward.

Some of Bruce's points are not yet covered, MISMATCH!MISMATCH
etc., maybe we have to declare these beasts as special cases
of a path-delimiter, brain-storming:

 path-list = [FWS] path-identity 
             *( path-delimiter [FWS] path-identity [FWS] )
             path-delimiter tail-entry [FWS]

 path-delimiter = "!" / "!!" / "!MISMATCH!" / "!POSTED!"

No more path-keyword.  Does that solve Bruce's problem ?  Bye

Date: Thu, 30 Jun 2005 17:44:10 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
> If anyone but Bruce wants to argue in favour of recognizing *all* timezones 
> mentioned in RFC 2822, which is the logical consequence of Bruce's 
> argument, please speak up now.

Harald, you misinterpreted silence as lack of thought on this issue.  You
are way to quick to declare "no action needed" in this case.

I'm mostly with Bruce.  I object to specific text mentioning allowing GMT to be
treated as -0000, because in fact RFC2822 says to treat it as +0000 already.
Why clutter the draft by repeating what RFC2822 says?

I object to any phrasing "unknown timezone."  It's ambiguous.

Then again, I don't really care if all other timezones are treated -0000
either.   RFC2822 already says the military zones, as if they ever show up
on Usenet!, should be treated as -0000.  (Which is something Russ was
concerned about I think he wrote.)

The list of obs-timezones in RFC2822 is short and unambiguous, just 8
zones, deferring to that text is not going to burden implementors any
more than putting up with all the other RFC2822 baggage.  And I'd rather
8 unambiguous things to implement than interpreting any of the text
that overrides RFC2822 I've seen proposed for this.

Sure, Anyone who writes or maintains date parsers will tell you there is
all sorts of cruft that you see in Date fields.  That's why INN's date parser
is written in yacc, and at 500+ lines it's a nightmare to modify/understand.
Who wants to touch that?  And for what?  So that you can accept some cruft
like H:M:S expressed as 5:6:00 and no zone or swahili time?  Who cares?  It's
a stupid date field, parse invalid dates as 1/1/1970 as far as I care about
fixing problems as stupid as that.

But there is something I do care about.....

I say that implementing an RFC2822-compliant date parser is "good enough"
to reuse in netnews.  I disagree with forcing implementors should have to read
this draft, and then wonder "gee, now that I read this tortured
usefor-usefor-04 prose with references and modifications to RFC2822, do
I have to do anything different, or can I just reuse my RFC2822 parsedate?"

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT?  (fwd)
Date: Thu, 30 Jun 2005 17:20:56 -0400
Cc: Frank Ellermann <[email-masked]>
Sender: [email-masked]

 

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1022 USEFOR 2.1 obs-phrase: Resolution proposal
Date:  Thu, 30 Jun 2005 23:08:49 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 11
Sender: [email-masked]

Charles Lindsey wrote:
 
> Do people think that would be a good idea, if it turns out
> to be feasible?

I'd like it.  For some time (back in 2002) I had some serious
difficulties to get the differences between posting-agent (UA),
followup-agent (UA), and injection-agent (news server) right.

                         Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:21788
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047 Path-identity: A proposal (perhaps)
Date: Thu, 30 Jun 2005 19:47:58 GMT
Lines: 130
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>If we explained it, would this be a reasonable text to insert in 3.1.6?
>(I've modified the BNF too. Only modified stuff included here)
>--------------------------------------------------------------------
>  A path-identity identifies an agent. For the news system to work
>  properly, the path-identity must be globally unique.
>  Traditional usage has had two means of generating unique identifiers:
>  DNS names and asserted identities. The first gets its uniqueness from
>  the DNS; the second gets its uniqueness from tradition. New servers
>  should use DNS names.

>  In some cases, one wishes to embed something into a path-entity that
>  is not a name. One common thing to embed is an IP address, serving
>  as an identity when the name of the agent is not known.

Hmmm! That looks more of a USEPRO matter. Indeed there is that and much
more in USEPRO, and I wanted it put in USEFOR, but Alexey told me it did
not belong there. So please can we be as brief as possioble in USEFOR, and
point to USEPRO for the gory details (see below)

>   path-list       =  [FWS]
>                      *( ( path-identity / path-keyword / path-address )
>                      [FWS]
>                      path-delimiter [FWS] )
>                      tail-entry [FWS]

>   path-identity   =  ( ALPHA / DIGIT )
>                      *( ALPHA / DIGIT / "-" / "." / "_" )

>   path-keyword    = "POSTED" / "MISMATCH"

>   path-address    =  IPv4address / no-fold-literal ; see [RFC2373]

A good idea, but the term <path-identity> needs to include everything that
may occur in the list, except the two keywords, because it is used with
that meaning in many places. So it would be better as:

   path-list       =  [FWS]
                      *( ( path-identity / path-keyword )
                      [FWS]
                      path-delimiter [FWS] )
                      tail-entry [FWS]

   path-identity   =  ( ALPHA / DIGIT )
                         *( ALPHA / DIGIT / "-" / "." / "_" ) /
                      IPv4address / IPv6address ; see [RFC 3986]

   path-keyword    = "POSTED" / "MISMATCH"

>  NOTE: The path-address contains characters that some systems consider
                                                       ^
                                                    legacy
>     path-delimiters. This can cause problems; path-address should
>     therefore only be used when absolutely necessary.
>     See [USEPRO] for details on when to use them.

However, that is better said in USEPRO. Moreover, it is only the ":" in an
<IPv6address> that is problematical, so that is the only case to mention
explicitly.

>--------------------------------------------------------------------

>Note that in recycling no-fold-literal, I've added [] around addresses... 
>and that a naked IPv4 address is allowed separately, because people have 
>said this is already present (too much for us to remove).

Yes, but you have also brought in a whole lot of other things, including a
further bunch of former <path-delimiter>s (even "!").

Moreover, current practice has been not to include [...] around IP
addresses when used in <path-identity>s, and "[" and "]" were also
<path-delimiter>s in the old days (though not so harmful in this
particular context). Hence I have reverted to <IPv6address> in my revised
syntax.

Here, now, is the text currently proposed for USEPRO. Maybe we should
review it now; there are several things that have come up in recent days
which maybe ought to go into it.

   News-servers need to identify themselves, by inserting their <path-
   identity>s (F-3.1.6), into Path, Injection-Info and Xref headers. An
   injecting agent MUST identify itself with the same <path-identity> in
   both Path and Injection-Info headers.  To ensure that a <path-
   identity> provides a unique identity for the news-server concerned,
   it SHOULD be one of:

   1. A fully qualified domain name (FQDN) associated with an "A" or
      "AAAA" record, which SHOULD identify the actual host inserting
      this <path-identity> and, ideally, should also be "mailable" (see
      below).

   2. A fully qualified domain name (FQDN) associated with an "MX"
      record, which MUST be "mailable".

   3. An arbitrary name believed to be unique and registered at least
      with all other news-servers receiving articles directly from the
      given one.

   4. An encoding of an IP address - <IPv4address> or <IPv6address> [RFC
      3986].

   The FQDN of a news-server is "mailable" if its administrators can be
   reached by email using both of the forms "usenet@<FQDN>" and
   "news@<FQDN>", in conformity with [RFC 2142].

   For an injecting agent prepending to a Path header (7.2.2), the
   <path-identity> MUST be option 1 or 2 and the FQDN MUST be mailable,
   and if the agent offers its services to the general public the form
   "abuse@<FQDN>" MUST also be available, unless a more specific
   complaints address has been provided in a <complainto-param> of an
   Injection-Info header (F-3.2.14).  For other agents, options 1
   through 3 are to be preferred.

>Does this make sense?

Not yet :-(

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21779
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1021 USEFOR Newsgroups header - resolution proposed
Date: Thu, 30 Jun 2005 18:26:15 GMT
Lines: 132
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>OK, here's my "proposed text" again, incorporating feedback.

>---- restricting syntax; there are no changes to the ABNF -----

> A newsgroup component SHOULD NOT consist of digits only, and
> SHOULD NOT contain uppercase letters. Such components MAY be
> used only to refer to existing groups that do not conform to this
                                       ^
                          created prior to this standard
> naming scheme.

>  NOTE: All-digit components conflict with one widely used storage
>    scheme for articles. Mixed case groups cause confusion between
>    systems with case sensitive matching and systems with case insensiteve
>    matching of newsgroup names.

>----- restricting use of specific characters ------------

> Components beginning with underline ("_") are reserved for use by
> future versions of this standard and MUST NOT be generated by
> posting agents (whether in Newsgroups headers or in newgroup control
> messages [USEPRO]). However, such names MUST be accepted by
> relaying and serving agents.

s/by posting agents//

(I.e. MUST NOT be generated; period. And also removes any argument about
what injecting agents do if they see one).

s/by relaying and serving agents/all agents/

(Not so sure there, since it apparently restricts the right of injecting
agents to reject them. OTOH, all mentions, esp. normative ones, of the
various different kinds of agent are probaly best removed from USEFOR).

Also s/(Whether ...)//, since it does not actually add anything.

> Components beginning with "+" and "-" are reserved for private
> use and MUST NOT be generated by posting agents (whether in
> Newsgroups headers or in newgroup control
> messages [USEPRO]) without a private prior agreement to do so.
> However, such names MUST be accepted by
> relaying and serving agents.

Same suggestions as above, for the same reasons. So I would suggest:

   MUST NOT be generated without a private prior agreement with the
   intended recipients (such as between a news-server and its clients).

>---- restricting component names ----

> The following newsgroup names are reserved, and MUST NOT be used as
>the name of a newsgroup:

> Groups starting with the component "example"
> The group "poster"

> The following newsgroup names have been used for specific purposes in
>various implementations and protocols, and MUST therefore not be used
>for the names of normal newsgroups. They MAY be used for their specific
>purpose, or by local agreement.

> Groups starting with the component "junk"

No, I think you can omit that one ("junk.foo is bizarre, but not harmful)

> Groups starting with the component "to"
> Groups containing the component "all"
> Groups containing the component "ctl"

  Groups starting with the component "control"

> The group "control"

But you can omit that, because it is already excluded because its first
(& only) component is excluded.

> The group "junk"
> The group "to"

And you can omit that, because it is already excluded as above.

OTOH, I am not sure your classification into two categories is the right
one. Surely the distinction should be between the things which will cause
things to break (poster, to, all, ctl) and the things which flout accepted
conventions (example, junk, control, and even single-component names,
though it now seems agreed not to mention these here).

Anyway, it is still necessary to give _reasons_ for all these exclusions,
probably in a NOTE, because that is how you handled the reasons for
all-digit groups, etc. Here are some texts that might be useful:

ctl:
because it formerly caused such articles to be interpreted as control
messages

all:
because it is used as a wildcard in some implementations

to:
because of its special meaning to route certain control messages on a
point-to-point basis [USEPRO]

poster:
because of its special meaning in the Followup-To header

control:
used by many serving agents for pseudo-newgroups to hold control
messages

junk:
used by many serving agents to store invalid articles

example:
reserved for examples in this and other standards

> NOTE: Some newsgroups violating these restrictions exist. It is not a
>protocol violation to use these names to access those newsgroups.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21776
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Date: Thu, 30 Jun 2005 16:21:33 GMT
Lines: 111
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>NEW:

>   The Message-ID header contains a single unique message identifier.
>   News is more dependent on msg-id uniqueness and fast comparision than
>   email is, and some news software would have trouble with the full range
>   of possible msg-ids permitted by RFC 2822; this section therefore
>   restricts the syntax of <msg-id> compared to Section 3.6.4 of
>   [RFC2822].
>   The global uniqueness
>   requirement for <msg-id> in [RFC2822] is to be understood as applying
>   across all protocols using such message identifiers, and across both
>   Email and Netnews in particular.

There are still people (even, apparently, on this list) who seem to think
that it is only "legacy software" that will "have trouble", and look
forward to the day when these irksome restrictions can be lifted (and hence
seek to express all these limitations by verbiage). Therefore I would like
to see something a little stronger, such as:

"Netnews is critically dependent on the uniqueness of <msg-id>s and on the
ability to compare them rapidly. Retaining the full range of possible
<msg-id>s permitted by RFC 2822 would introduce severe performance
penalities, and would also be in conflict with [NNTP]; this section
therefore ..."

>NEW:

>   no-fold-quote   =  DQUOTE
>                         ( "." *mqtext /
>                           *mqtext "." /
>                           *mqtext mqspecial *mqtext )
>                         DQUOTE
Correct.

>NEW:

>   Some software will try to match the <id-right> of a msg-id in
>   a case-insensitive fashion; some will match it in a case sensitive
>   fashion. Implementations MUST NOT generate two message-IDs where the
>   only difference is the case of characters in the <id-right> part.

That seems a protocol issue to me, and therefore should go in USEPRO.
Indeed, USEPRO already contains (7.3):

        NOTE: Some old software may treat message identifiers that
        differ only in case within their <id-right> part as equivalent,
        and implementors of agents that generate message identifiers
        should be aware of this.

That could be altered if people want it stronger.

>   If domain literals are used, the syntax found in [RFC2821] section
>   4.1.3 is RECOMMENDED.

Except that may change in 2821bis, and RFC 2822 took great care to allow
for all sorts of domain literals that might be invented in the future. So
I think we should stick to the RFC 2822 line. Perhaps:

   If <no-fold-literal>s are used, they SHOULD be restricted to the
   formats allowed for Internet addresses by the relevant standards.

(Which is slightly stronger that what is said in 3.4.1 of RFC 2822, but
uses some of the same terminology.)

>     NOTE: [RFC2822] section 3.6.4 recommends that the <id-right> should
>     be a domain name or a domain literal. Domain literals are troublesome
                       ^^^^^^^^^^^^^^^^^^^
		       (or a domain literal IP address)
>     since many IP addresses are not globally unique; domain names are
            ^^^^
	    some
>     more likely to generate unique message-IDs
                                     ^^^^^^^^^^^
				     message identifiers.
>     In no circumstance is it reasonable for software receiving a <msg-id>
>     to change behaviour based on a theory about how the <msg-id> was
>     generated.

Just using correct terminology from RFC 2822. "message-ID" is not a
defined technical term. "Message identifier" is the semantic concept
(ignoring '<' and '>'), and <msg-id> is its syntactic embodiment.

The only genuine not-globally-unique IP addresses are the ones set aside
for local use (127.0.0.1 is the best-known). Even IP addresses obtained on
a DHCP lease are uniquely yours so long as you hold them. Hence
s/many/some/.

And just to cover some conventions in current and private use,
s/In no circumstance is it reasonable/It would be unwise{unreasonable}/.
"In no circumstances" is, in any case, a bit too strong to go in a NOTE.

>Makes sense?

Yes indeed.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21777
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Date: Thu, 30 Jun 2005 16:39:12 GMT
Lines: 45
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Wed June 29 2005 05:36, Harald Tveit Alvestrand wrote:
> 
>>    The global uniqueness
>>    requirement for <msg-id> in [RFC2822] is to be understood as applying
>>    across all protocols using such message identifiers, and across both
>>    Email and Netnews in particular.

>OK.  I wouldn't mind the latter part expanded to "all applications which
>use the Internet Message Format [RFC2822]".

Be careful there. The present wording already covers "all applications
which use the Internet Message Format", but it also includes anything
else (such as X.400) which might use something compatible with our
"message identifiers".

>  NOTE: Although the length of a msg-id is constrained to 997 octets, and
>        a legal domain name may be up to 255 octets in length, some news
>        software implementations silently truncate a msg-id at 255 octets.
>        Such implementations SHOULD be upgraded as soon as practicable to
>        support a msg-id length of 997 octets.  Implementations that
>        exchange messages which may be sent to a legacy implementation
>        should limit msg-id length to no more than 255 octets if possible.

>Clearly the problem here is some legacy implementations, since a domain name
>has always been allowed to be up to 255 octets (RFC 882).

No, the limit (250, not 255) is hard coded into many implementations and,
more to the point, it is hard coded into the new NNTP standard-to-be,
which has just passed its IESG Last Call and is now in the RFC Editor's
queue.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21783
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date: Thu, 30 Jun 2005 18:56:08 GMT
Lines: 42
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Seth Breidbart wrote:
> 
>> I agree it's odd.  I still think we should do it.
>[...]
>> "MUST NOT generate" because there's no reason to allow
>> generation

>Yesterday Charles wrote something in the direction of
>SHOULD NOT => permission, and now you apparently propose
>SHOULD NOT => allowed.  Do all here think that a SHOULD
>(NOT) is something like "please (not)", or worse that
>SHOULD NOT is just a shorthand for "of course you can" ?

>There is a reason to allow generation, legacy software.

No. You misunderstand the intention of our standards.

Legacy software does all sorts of things which we don't want them to do
any more.

Therefore, we write our standard so that those legacy implementations
become non-compliant (because that is a good carrot to encourage them to
be upgraded).

OTOH, of course, we so arrange things (by making transitional/fallback
arrangements where necessary) that the legacy software can continue to
interoperate at least for the medium term (long term, in some future
standard, we may outlaw some such things entirely, but that will be
somebody else's decision to take).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21784
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1042 USEFOR 3.1.5 Newsgroups folding - resolution proposal
Date: Thu, 30 Jun 2005 19:06:08 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>Suggested addition to section 3.1.5:

>Folding the Newsgroups: header field over several header lines has been 
>shown to harm propagation significantly. Folded Newsgroups header fields 
>SHOULD NOT be generated, but MUST be accepted.

>The only valid reason identified at this time for generating folded 
>Newsgroups header fields is to test propagation of articles using that 
>feature.

I agree with the intent, but not with the wording. But at least we seem to
be settling for SHOULD NOT generate both for this case and for comments in
References, which is certainly consistent.

The problem is not that "folding over several header lines has been shown
to harm propagation", but that it has hitherto been totally excluded from
the syntax. Therefore, we need to say that it is a "newly introduced
feature with little support in current implementations". And then you say
it "MUST be accepted but SHOULD NOT be generated" (adding "at the present
time", or other qualifying wording if you wish).

And I would omit the paragraph about "to test propagation of articles".
Implementor know how to do that without being told.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21790
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1021 Single component newsgroups (Re: Definition of "private" (Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts)
Date: Thu, 30 Jun 2005 20:29:24 GMT
Lines: 24
Sender: [email-masked]

In <AEF08FC6BEDD4E5AABF1F47A@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>I think that's the consensus of the group at the moment:

>It's not sensible for the article *format* standard to say anything about 
>single component newsgroup names. They should be permitted in syntax.

>We'll return to the topic with USEAGE (if we get that far).

OK, I could take the single-component newsgroup-names to USEAGE, but it
seems that the intent of Harald's latest proposal is to leave the rest of
these "funny" newsgroup-names (control.*, junk.*, to.*,
all-digit-components, etc.) in USEFOR.  Is that understanding correct?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21787
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Thu, 30 Jun 2005 19:20:01 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Russ, I think I need some clarification:

>Are you saying that the "server-name" part remains unchanged?  If not, how
>is the end result conceptually different from remove/generate?  Also, if
>not, (i.e. the "server-name" is the same), how is this effectively
>different (from the POV of any connecting client) from "one big serving
>agent"?

I think others have answered your main point. However, even if B is a pure
slave of A, it _might_ still be reasonable for articles passed to B to
change the <server-name> to B, even if they retain the identical
<article-locator>s.

Actually, the wording does not preclude that, since B can always claim to
have removed the old Xref header and replaced it with one that was almost
identical. Indeed, the whole Xref slaving matter could have been covered
by that getout in the old wording, but the new does make it more clear.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21789
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1022 USEFOR 2.1 obs-phrase: Resolution proposal
Date: Thu, 30 Jun 2005 20:24:58 GMT
Lines: 35
Sender: [email-masked]

In <[email-masked]> [email-masked] (Ralph Babel) writes:

>Harald Tveit Alvestrand wrote:

>> News articles MUST conform to the syntax specified
>> in Section 3 of [RFC2822]. Netnews agents

>"Netnews agent" isn't defined anywhere. Didn't
>we at some point agree that Usefor was about
>on-the-wire article _format_, not "agent" behavior?

I think Ralph has a point here. USEPRO currently defines two terms:

user agent	encompassing posting, reading and followup agents
news-server	encompassing injecting, relaying and serving agents

All the individual *-agent terms are, of course heavily used in USEPRO.

My preference would be to define and use those terms only in USEPRO, and
try to get by with only "user agent" and "news-server" within USEFOR. I
have not trawled throught USEFOR to see if we could actually get away with
that, but on the face of it it _ought_ to be possible.

Do people think that would be a good idea, if it turns out to be feasible?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21785
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1043 "header": Suggested resolution
Date: Thu, 30 Jun 2005 19:10:00 GMT
Lines: 29
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>Charles Lindsey wrote:
>> 
>> I pointed out that in the real world implementors and other readers DO
>> take shortcute when reading documents. That is a fact of life. You have to
>> live with it.

>No I do not have to live with it.  As has been said so many times: we
>need a clear, complete, unambiguous specification that is as easy to read
>as possible, BUT WE HAVE NO CONCERN for people who refuse to read and
>understand NORMATIVE references.  We can totally ignore such people when
>writing, because they are irrelevant.

To describe a substantial part of your audience as "irrelevant" is no way
to write any document, whether technical or otherwise.

If that is your attitude, then we will just have to agree to differ.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21781
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1022 USEFOR 2.1 obs-phrase: Resolution proposal
Date: Thu, 30 Jun 2005 18:40:40 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>I suggest the following text change to section 2.1 of USEFOR:

>NEW:

>   News articles MUST conform to the syntax specified in Section 3 of
>   [RFC2822].  Netnews agents MAY also accept the obsolete syntax
>   specified in Section 4 of [RFC2822], and MUST accept the "UTC" timezone
>   (see section 3.1.2) and the obs-phrase construct (see [RFC2822] section
>   4.1) but they MUST NOT generate
>   productions of such syntax.

s/UTC/GMT/ of course.

And then, please, a little explanation of what obs-phrase is supposed to
achieve, perhaps with a 
  John Q. Public <[email-masked]>
example. And maybe also that the "obs" in <obs-phrase> is a bit of a
misnomer. Otherwise, it is not clear to the reader why we have singled out
this particular "obs" for special attention.

>Makes sense?

Yes.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21782
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date: Thu, 30 Jun 2005 18:48:44 GMT
Lines: 32
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>Based on comments to Ken Murchison's text, I currently have the following 
>as resolution:

> The Date header is the same as that specified in Sections 3.3 and
> 3.6.1 of [RFC2822] with the added restrictions detailed in
> Section 2.2. However, the use of "GMT" as a time zone (part of <obs-
> zone>), although deprecated, is widespread in news articles today.
> Therefore, agents MUST accept <date-time> constructs which use the
> "GMT" zone.

But you are now giving the same normative rule (MUST accept "GMT") in two
places, which is not usually a good idea. Better to say

  ... hence the reason why section 2.1 REQUIRES its continued
  acceptance (but not generation).

>Makes sense?

Yes.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21778
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Date: Thu, 30 Jun 2005 16:47:50 GMT
Lines: 26
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>But here I strongly disagree, I care _very much_ if somebody
>generates a <[email-masked]> without my permission
>because it's my name space.  And therefore <id-right> is a bad
>name.

But that is not a violation of USEFOR, which is to be a format standard.

It might be a proper matter for attention in USEPRO, but IMO it is really
an argument between you and the abuse dept. of ISP of the perpetrator.

Moreover, if it is someone issuing a cancel for one of your spammed
articles, it might even be a right and proper thing to do :-) .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21780
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1021 USEFOR Newsgroups header - resolution proposed
Date: Thu, 30 Jun 2005 18:31:38 GMT
Lines: 20
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>Can I suggest the following arrangement of text...

No, I think I prefer the original arrangement. Start with the ones which
are simple restrictions on the syntax (all-digit and uppercase-names),
which should come immediately after the syntax, Deal next with the ones
that have systematic treatment ("_", "+", "-"). And finally the apparently
bizarre specific component-names.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21786
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Thu, 30 Jun 2005 19:11:20 GMT
Lines: 18
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>I am not sure we have basis for a MUST/SHOULD, but how is it a kludge
>to be able to mark cross-posts as read?

Because, with hindsight, there might have been better ways of achieving
that effect.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date:  Thu, 30 Jun 2005 22:28:46 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 40
Sender: [email-masked]

Russ Allbery wrote:
 
> What I'd really like to do is not deviate from RFC 2822 any
> more than is absolutely necessary

Sounds like you want the complete obs-stuff in and then maybe
exclude some of the worse nightmares explicitly.  In theory
it's possible that we'd arrive at the same subset with this
strategy, but in practice I doubt it

I would want to get rid of the "North-American" zones because
they are known to cause harm (= folks outside of North-America inventing
their own zone names, or new semantics for existing
names, maybe I'd insist on I18N considerations for this stuff,
it's a can of worms combined with ugly ratholes).

> I think that treating the North American time zones as
> unknown when RFC 1036 said they were fine is an iffy
> decision.

Combined with the general "MAY accept obs-stuff" everybody can
pick what he likes best.  With "MAY do this" on one side, and
"SHOULD do that" on the other side, is there still a realistic
chance to get it completely wrong, e.g. reject such articles ?

> I can understand why other software authors don't want to be
> bothered with folding and comments

[CFWS] is only at the end of 2822 <date-time>, ignoring it is
no problem.  The other 2822 [FWS] and FWS should be also okay.

Otherwise your preference of "2822 as verbatim as possible"
has a problem, "obs-zones but no FWS" is IMNSHO no option.

It's in theory possible to copy the 2822 <date-time> ABNF with
its numerous obs-rules, and then delete those that we don't
like.  OTOH we tried that already.  Harald's proposal is the
cleanest solution for me.
                          Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date:  Thu, 30 Jun 2005 21:49:44 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 44
Sender: [email-masked]

Seth Breidbart wrote:
 
> "MUST NOT" = don't do it.  Period.
> "SHOULD NOT" = don't do it unless you have a good reason.

ACK, and don't use these 2119 keywords where it's unnecessary.
Necessity includes "potential to cause harm".  Or here also
all "differences from 2822".

Based on that I got "don't quote MUST NOT generate GMT", but
it's of course also a matter of taste to a certain degree.

> the question is which is worse: being _more permissive_
> than RFC 2822 (which we aren't anywhere else, are we?) or
> decreeing that legacy software is non-conformant with a
> new spec?

Charles apparently intends to be more permissive about some
length limits (78 and 998), I'm not sure at the moment. 

BTW, it's not that I wanted any explicit "SHOULD NOT generate
GMT", I only didn't like the explicit "MUST NOT generate GMT",
because there's neither a potential harm nor any difference
from 2822.  Maybe that's a bit too shrewd... ;-)

>> in my parallel universe it's not allowed to implement any
>> new software generating GMT if the specification says
>> "SHOULD NOT" -
 
> Unfortunately, some implementors won't agree with you.
> They'll find other reasons that are good enough for them.
> (At least, that's my guess.)

Yes, that's possible.  And Bruce's IMAP "sort-by-date" example
was interesting.  I know nothing about IMAP and trust that Ned
or Bruce would tell us if we're about to screw up.  So far I
don't see a problem, except from "Russ wants some problematic
time zone names as a MUST accept", which would put us back to
square one.  

I'm happy with only GMT, eliminating the complete zone name zoo
is also very tempting.
                         Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT?  (fwd)
Date:  Thu, 30 Jun 2005 21:15:29 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 32
Sender: [email-masked]

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: NEW: blanket statements vs. the law of unintended consequences, also Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date: Thu, 30 Jun 2005 15:02:25 -0400
Cc: Harald Tveit Alvestrand <[email-masked]>
Sender: [email-masked]

On Thu June 30 2005 09:57, Harald Tveit Alvestrand wrote:

> Bruce, I have not heard anyone but you arguing that *all* OBS-syntax should 
> be accepted.

Harald, I don't believe that I have have said so, especially not with
that emphasis.  I have said that I believe that we should adhere to
2822 as closely as possible, making specific exceptions where there is
some justification.

[oh, my!]
> raise an issue that  
> specifically suggests alternate text for 2.1, where we say that OBS-syntax 
> is a "MAY" accept, not a "MUST" accept?

So MUST means MAY except where (e.g. obs-phrase) MAY means MUST. Unless
of course MUST means MAY again.

While the 2.1 text says "MAY" it doesn't specifically say that it
overrides 2822, so it's not 100% clear.  It's unclear what the order
of precedence is in interpreting 2822 vs. 2.1 vs. the presumably
agreed-upon obs-phrase text, etc.

Blanket statements flout the law of unintended consequences and lead to
convoluted rules such as the above.  I propose eliminating the blanket
statement in section 2.1 (i.e. the second sentence of the first paragraph),
with the understanding that that does not preclude adding limited-scope
relaxation of specific parts of 2822 obs-syntax parsing requirements
where appropriate and where justifiable.

> Acting as if we had changed 2.1 when we have not is ... not useful.

OK.  Among other not-useful things:
o presenting text in isolation which refers to section 2.2 but not 2.1
  where 2.1 may be applicable

Plenty of blame to go around.

To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 30 Jun 2005 11:57:03 -0700
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> It was in a reply to Russ, and Russ wants the "North-American"
> cherries without the military zones.

Not exactly.  What I'd really like to do is not deviate from RFC 2822 any
more than is absolutely necessary, and I think that treating the North
American time zones as unknown when RFC 1036 said they were fine is an
iffy decision.  What I was trying to say about the military zones is that
I don't *care* about them, and I don't think that the RFC 1036 concern is
applicable there since they were already deprecated by other standards.
However, I would much prefer to adopt RFC 2822 as verbatim as possible
without cherry-picking.

The main thing that's going to get in the way of that is not supporting
time zones, but rather accepting additional junk in the middle of the Date
header.  INN does already handle the full obs syntax of RFC 2822 in the
current development version, but I can understand why other software
authors don't want to be bothered with folding and comments, particularly
since there's no good reason for having either other than consistency with
mail.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date:  Thu, 30 Jun 2005 20:34:32 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 37
Sender: [email-masked]

Bruce Lilly wrote:
 
> any purported "design principle" were replaced by the
> specific text in Harald's proposal

He didn't write "MUST accept GMT" as redundant rule, as you
claimed, but because it's an exception from the "MAY accept",
and because MUST and MAY are different.

And the "MAY accept" is certainly not only "purported".

> If the intent is to contradict 2822, which is a normative
> reference, then it's necessary to state so unambiguously

| News articles MUST conform to the syntax specified in
| Section 3 of [RFC2822].  Netnews agents MAY also accept the
| obsolete syntax specified in Section 4 of [RFC2822], but
| they MUST NOT generate
[...]

That's unambiguous, and one of the tickets added text for the
two exceptions (MUST accept GMT and obs-phrase) explicitly.

>> That's in Bruce's obs-package, we're cherry picking
> Speak for yourself.  I want no cherry-picking.

It was in a reply to Russ, and Russ wants the "North-American"
cherries without the military zones.  You may now please stop
telling me what I can write or not, unless I'm plain wrong.

But while we're at it, of course you'd be also cherry-picking
if you'd want some obs-2822-rules but not others.  

And whatever IMAP might do, if it simply ignores "GMT" all is
fine, "GMT" is the same as +0000.  None of our business, we're
not inventing new time zones different from 2822.  Bye, Frank

Date: Thu, 30 Jun 2005 19:36:38 +0200
From: [email-masked] (Ralph Babel)
To: [email-masked]
Subject: Non-Admin: Please watch your language
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> Bruce, I have not heard anyone but you arguing that *all*
> OBS-syntax should be accepted. Can you now shut up about
> this issue, [...]

Just couldn't resist.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date: Thu, 30 Jun 2005 13:22:52 -0400
Cc: Frank Ellermann <[email-masked]>
Sender: [email-masked]

On Thu June 30 2005 12:02, Frank Ellermann wrote:

> Sigh.  That's the point of this thread.  It's a general design
> principle of USEFOR to replace 2822 "MUST accept obs-ceneties"
> by a "MAY accept" keeping the 2822 "MUST NOT generate".
> 
> So when Bruce wrote...
> 
> | that text is redundant and unnecessarily specific to "GMT"
> 
> ...it's only his POV because he disregards the general design
> principle.  Of course it's necessary to say "MUST accept GMT"
> if the general rule is only a "MAY accept".

No, such text and any purported "design principle" were replaced
by the specific text in Harald's proposal, which replaced the specific
text in Ken's proposal, proposed to replace the specific text in
the current draft.

> If Bruce doesn't like "unknown" s/unknown/other/ is perfectly
> clear,

I disagree. If the intent is to contradict 2822, which is a
normative reference, then it's necessary to state so unambiguously,
especially where is affects a 2822 field such as Date.  I don't
believe such a contradiction can fly (see below).

> That's in Bruce's obs-package, we're cherry picking

Speak for yourself.  I want no cherry-picking.

IMAP is the Internet Message Access Protocol, which is used to
access mail and news and other messages.  There is a protocol
extension draft, mentioned in this WG several times, which provides
a mechanism for a client (UA) to request the server to collate
message order by factors including the Date field.  The client
and server deal with messages; there is no artificial distinction
between "news" and "mail" and "other".  IMAP is a Standards Track
protocol compliant with RFCs 822, 850, 1036, and 2822, among
others.  Given such a request, how do you propose that such a
server should treat a message in a folder containing the field
  Date: 30 Mar 2001 12:34:56 EST
Three-part question: do you believe that should be treated as
having 822/2822 semantics (equivalent to -0500) or as -0000,
why, and do you believe that you can convince IMAP implementers?

2003-06-30:c

From: Bruce Lilly <[email-masked]>
Reply-To: Bruce Lilly <[email-masked]>
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: Decision procedures
Date: Thu, 30 Jun 2005 12:51:14 -0400
Cc: Harald Tveit Alvestrand <[email-masked]>
Sender: [email-masked]

On Thu June 30 2005 04:36, Harald Tveit Alvestrand wrote:
> Bruce,
[...]
> > Yes, we know that you and Alexey decide *if* consensus has been reached,
> > because that's explicitly what 2418 says.  But 2418 explicitly says that
> > the WG gets to agree on the means.  I have posted what a past co-chair
> > has indicated as the means, and I believe that that method had WG
> > consensus behind it.
> 
> I believe you are referring to your posting:
> 
> 
> > For reference, a past
> > WG Chair had the following to say, and if I recall correctly, Alexey
> > has agreed:
> >   On the topic of voting: Especially on this topic, I think voting
> >   would be stupid. The decision is about rough consensus. One screaming
> >   person does not indicate that there is no rough consensus, but one or
> >   two well-reasoned arguments against a screaming huge crowd does. And
> >   a huge number of "I'd prefer X, but I couldn't care less" votes
> >   versus 2 or 3 well-argued "X will spell doom for the Internet, and Y
> >   will save it" votes *is* rough consensus for Y over X. So voting
> >   generally doesn't help me decide one way or the other that there is
> >   rough consensus.

Correct.

> This is a quote from Pete Resnick's message
> <http://www.imc.org/ietf-usefor/2003/Jun/0099.html> - no need to be coy 
> about who the previous chair is.

Coyness (http://www.dictionary.com/search?q=coy) has nothing to do with
it.  Yes, Pete wrote that; is that relevant (as distinct from the fact
that he was WG Chair at the time)?

For the record: I think Pete's a splendid fellow and did an outstanding
job as WG (co-)Chair.

I also think he had an excellent series of points regarding consensus,
which it why I quote it; I would probably not do so if I thought
otherwise (sometimes, one has to state the obvious).
 
> >From the same message:
> 
> >  (It's
> >  also clear, BTW, that people are not listening to each other and
> >  aren't trying to think of where we can come to consensus, which is
> >  why people think they need a vote.)
> 
> Familiar?

Yes, and I would point out that in neither that case nor recently have *I*
called for "a vote" or "counting heads" or anything remotely like that.
 
> > The question, still unanswered, is whether or not that method, which
> > specifically calls for taking well-reasoned arguments into account, is
> > being followed by the current co-chairs in determining *if* consensus
> > is reached and where that consensus lies.  If so, it would help if the
> > co-chairs were more vociferous about what fails to qualify as a
> > well-reasoned argument; we have certainly had plenty of unsupported
> > assertions and "false statement[s] deliberately presented as being true"
> > (http://dictionary.reference.com/search?q=lie) -- it would also help if,
> > when announcing that consensus has been reached, there were something
> > more substantive than a list of names.  If not, there seems to be a
> > problem, as I don't recall a WG consensus decision (based on that
> > established means of determining consensus) to change the means of
> > determining consensus.
> 
> One thing that probably should have been included: The fact that you think 
> that you have a well-reasoned argument and your opponent's argument is 
> totally beside the point has nothing to do with whether or not we have 
> consensus.

Fine.  But there's no question that an unsupported assertion is no well-
reasoned argument or that an unsupported assertion which contradicts the
facts is "a falsehood".

> In fact, you having what you think is a well-reasoned argument and totally 
> failing to convince anyone else that the argument is reasonable is just 
> another way of becoming the "rough" in "rough consensus".

Maybe.  Perhaps it means that the reasoning is not understood (which is
why I encourage asking questions and why I do so myself when there is
some doubt about some claim or position).

> I will NEVER rule  
> "rough consensus" on anything where I know that the majority of the group 
> thinks it's a stupid idea.

I believe it's important to distinguish whether somebody thinks something
is a "stupid" idea (not my epithet, just quoting) because of prejudice,
ignorance, or logical reasoning, as well as (in the last case) whether
it's a mild preference or a strong conviction (really irrelevant in the
other two cases).  See also Pete's comments about "a huge screaming crowd"
and "couldn't care less".

> When stating lists of names (which has happened rarely), I think of it 
> mainly to make sure that the record shows that the decision was made in 
> full knowledge of who does NOT agree; that these people have raised 
> arguments against what has been ruled the consensus, their arguments have 
> been heard, and the chairs have ruled that the group has consensus to 
> proceed in spite of the disagreement.

I still believe it would be helpful to provide some indication of rationale,
so for example, if the list of names says
"so-and-so is against the proposal (no reason)"
one might contribute by saying "I agree with so-and-so and here are some
reasons...".  Unless of course the WG and/or its chairs don't care about
reasons, in which case "me too" would suffice.

It's also one thing if "their arguments" is the empty set and another if
they correspond to Pete's "x will spell doom for the Internet" (provided
of course that there is in fact an argument and not merely an unsubstantiated
claim).

> I will absolutely refuse to poison the atmosphere of this list further by 
> stating that specific people are lying. I'll also refuse to say that 
> specific people are idiots, trolls, stiff-necked fools or any other 
> epithets.

The usual way of not saying something is, well, to not say it...

> What should matter in determining consensus is the content of the  
> argument.

Agreed.  How can that be done when there is no argument, just some
claim (or, for that matter, an irrelevant argument like "I'm opposed to
the X option because unripe bananas are green")?
 
> "False" is a statement about the argument. "Lie" is a statement about the 
> person making it.

No "lie" as a noun is a statement about a statement.  One that can
be substantiated, or not.

But back to the issue: where do we stand on the matter of the means
of determining consensus?  Do unsupported assertions count?  Do
false statements count?  Is "I'm in favor of x, but I couldn't care
less" weighed the same as "x will spell doom for the Internet"?  It
matters (for example, if unsupported assertions don't count, any WG
participant who wishes to make a point had better provide some
substance; conversely, if unsupported assertions count, we can simply
revert to an "is too"/"is not" shouting match (or disband, saving
the bandwidth, time, effort, and angst)).
  
2003-06-30:b (responding to specific issues)

Date: Thu, 30 Jun 2005 18:20:07 +0200
From: [email-masked] (Ralph Babel)
To: [email-masked]
Subject: Re: #1022 USEFOR 2.1 obs-phrase: Resolution proposal
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> if we're talking about accepting at all, we must be
> willing to talk about the ones who accept. Otherwise,
> there's eternal wandering around in passages of the
> passive voice to be found, with no actor in sight.

So since we're defining an article format, maybe we
shouldn't be talking about "accepting" in the first
place, but instead simply introduce two levels of
conformance - much like 2822 with its "obs-" tokens.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Date:  Thu, 30 Jun 2005 18:40:29 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 36
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
 
> now I get the concern..... the requirement for uniqueness is
> not only that you generate names in a namespace that you
> control, but that I do *not* generate names in that
> namespace.

Yes.  Russ' point about the $alz convention not withstanding:

A conventional bid "4 SA" doesn't mean that you want to play
sans atout, or at least not only 4 SA.  And it's a convention
how to break some 1036-rules intentionally and efficiently,
it's the same situation with s-o-1036 or its future grandson.

> Should we add a "MUST NOT generate msg-ids in id-rights you
> don't control"?

It's probably clear that I prefer a simple <id-domain> instead
of tons of MUSTard with the same effect for the bad <id-right>.

Yes, that MUST could fix one of these id-left / right problems.
 
> That's why I'm concerned with the syntax of RHS
> domain-literals

And <no-fold-quote> and <no-fold-literal> are also bad names,
we don't want to inherit the 2822-rule about their "semantical
content" (= invisible backslash in a quoted-pair).

A clean minimal interface is a new <msg-id> and from there use
new productions, don't try to overload more old RfC 2822 names
if the syntax or semantics differs from 2822.  Especially if
those differences are very subtle like a "visible" backslash.

                           Bye, Frank

To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 30 Jun 2005 09:19:32 -0700
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:
> Russ Allbery wrote:

>> I'll argue in favor of that, since otherwise we would be going from
>> blessing use if reluctantly (RFC 1036) to allowing parsers to disregard
>> them without any intervening deprecation period.

> Sigh.  That's the point of this thread.  It's a general design principle
> of USEFOR to replace 2822 "MUST accept obs-ceneties" by a "MAY accept"
> keeping the 2822 "MUST NOT generate".

I've clearly missed the point of the thread, and I'm sorry about that.  I
was previously responding to the language about GMT and was also looking
primarily at the generate side.  I had not realized that people were
talking about not accepting the North American time zones.

The military time zones were deprecated by RFC 1123, and more tellingly,
they also don't appear in practice.  I have no problem getting rid of them
except insofar as it adds wording complexity.  But I *do* have a problem
going straight from RFC 1036 to allowing servers to not even parse the
North American time zones, which are still in use (albeit not heavily).

I think we need one more step between here and there.

> That's exactly what we want.  And otherwise "you MAY parse it, else you
> SHOULD treat it as unknown -0000" is fine.

No, I don't agree that this is fine.

> That's in Bruce's obs-package, we're cherry picking, and IMHO reducing
> this to the one cherry GMT is an excellent choice for new parsers.

I also don't agree with this.

> Really, can we close this #1028 now ?

Apparently not.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date:  Thu, 30 Jun 2005 18:02:34 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 50
Sender: [email-masked]

Russ Allbery wrote:

>> If anyone but Bruce wants to argue in favour of recognizing
>> *all* timezones mentioned in RFC 2822, which is the logical
>> consequence of Bruce's argument, please speak up now.

> I'll argue in favor of that, since otherwise we would be
> going from blessing use if reluctantly (RFC 1036) to allowing
> parsers to disregard them without any intervening deprecation
> period.

Sigh.  That's the point of this thread.  It's a general design
principle of USEFOR to replace 2822 "MUST accept obs-ceneties"
by a "MAY accept" keeping the 2822 "MUST NOT generate".

So when Bruce wrote...

| that text is redundant and unnecessarily specific to "GMT"

...it's only his POV because he disregards the general design
principle.  Of course it's necessary to say "MUST accept GMT"
if the general rule is only a "MAY accept".  And of course
this essentially switches GMT back to the obs-2822-MUST-rule.

That's exactly what we want.  And otherwise "you MAY parse it,
else you SHOULD treat it as unknown -0000" is fine.  And as a
bonus we get a GMT => +0000 effect if Harald uses s/any/other/

If Bruce doesn't like "unknown" s/unknown/other/ is perfectly
clear, "anything not covered by numerical 2822-timezones or
the deprecated GMT or whatever you still MAY parse".

> It would surprise me if there are any existing Usenet date
> parsers that don't handle all of the RFC 2822 zones with the
> exception of the badly specified and almost never seen
> military zones.

That's in Bruce's obs-package, we're cherry picking, and IMHO
reducing this to the one cherry GMT is an excellent choice for
new parsers.

If an old / other parser wants more cherries it has a MAY and
a place (2822 obs-) to find them.  And if that has some weird
side effects for an erroneous EDT or EST it's not our fault.

Really, can we close this #1028 now ?  I promise that I won't
mention the utter dubious "MUST NOT generate GMT" anymore if
we stick to the general design priciple "no obs-ceneties excl.
GMT".  And the obs-phrase in Keywords: etc. <shudder />  Bye

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date: Thu, 30 Jun 2005 11:59:18 -0400
Cc: Harald Tveit Alvestrand <[email-masked]>
Sender: [email-masked]

On Thu June 30 2005 10:19, Harald Tveit Alvestrand wrote:

> My reading of the text and the discussion is that the group wishes to allow 
> a parser to NOT recognize "EST" as anything more valid than "ABC".
> (Otherwise, the whole fuss about requiring handling of "GMT" would be 
> completely meaningless.)

The WG discussion has been focused on comparisons of date-time in
Injection-Date, whereas blanket statements about zone abbreviations
affect legacy Date and Expires fields as well as any RFC 2822 and
extension fields which may arrive via gateways.

> This requires permitting the interpretation of "EST" as "unknown", which is 
> not the intent of RFC 2822, but is (in my interpretation) the intent of 
> this working group.

My interpretation is that some WG members might favor that for
Injection-Date specifically (although no text has been proposed
even though there is an open ticket for that purpose), but I see
no discussion that explicitly supports interpreting "EST" as unknown
in the legacy fields.

I have made a constructive comment on the specific text proposed,
which was somewhat vague (as previously discussed on-list by others).
If you believe that some clear wording specifically addressing EST
appearing in *any* date-time in *any* field is the WG consensus,
then by all means counter-counter-propose some text that says so
clearly rather than by providing a vague back door, and we can
debate that along with your original proposal and my two counter-
proposals.  I believe it was you who said that specific text change
proposals followed by debate on the open list was the simplest way to
decide consensus.

You proposed specific text; I carefully reviewed it and commented,
making two alternative counter-proposals addressing a specific
(vagueness) issue:
1. Simply refer to 2822 section 4.3
2. Repeat verbatim the relevant 2822 text (which I quoted so that
   it can be seen as a *specific* text change)

I'll also paraphrase what I wrote before: some people here are
reluctant to suggest specific text because of tendencies to:
o twist around what is suggested
o reject such suggestions by fiat
o nitpick
none of which are conducive to open debate (yes, I have changed
the order of "debate" and "open", and not accidentally).  I'll
leave it to individual readers to decide whether or not that is
an accurate characterization of past WG activity and its
consequences, and whether or not it continues to be a problem.

> If anyone but Bruce wants to argue in favour of recognizing *all* timezones 
> mentioned in RFC 2822, which is the logical consequence of Bruce's 
> argument[...]

No, I have not argued in favor of recognizing the 1-letter zone
abbreviations -- in fact I have gone to some effort to make that
clear by specifically mentioning them.  Nor have I argued in favor
of recognizing the "Other multi-character (usually between 3 and
5) alphabetic time zones" mentioned in RFC 2822, and I have also
made that clear by specific mention.

If you have a question about what I mean, by all means ask for a
clarification (on-list or off) and I'll do my best to provide one,
but please don't put words in my mouth.

2003-06-30:a (responding to specific issues)

Date: Thu, 30 Jun 2005 08:26:25 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1047 Path-identity: A proposal (perhaps)
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>There are only 3 ways I see to get rid of the : character:

>- Stop putting domain-literals in Path: at all (seems unacceptable)
>- Encode IPv6 addresses differently (seems unacceptable)
>- Treat IPv6 and IPv4 differently (seems unacceptable)

>If you've eliminated the impossible, what you're left with.....

How did we leap from "seems unacceptable" to "impossible"?

Let's see. We're stuck with legacy apps that are unlikely to change their 
method of parsing a Path header overnight, all based on a pesky colon. But
the "standard" means of representing an IPv6 address uses colons. What do 
we do?

Hmmm. How about simply eliminating the colons (and the effects) from the 
IPv6 address literal? Bingo -- a 16 character "path-identity" without
any punctuation. No need for the :: shorthand, people aren't typing it in 
over and over again. 

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Thu, 30 Jun 2005 11:12:53 -0400
Cc: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

On Wed June 29 2005 08:51, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >On Mon June 27 2005 15:01, Charles Lindsey wrote:
> 
> >> Simply changing the name of some syntax rule cannot possibly change the
> >> meaning of the document.
> 
> >Ergo, by your own logic, "id-right" does not change the meaning of
> >the RHS of a msg-id from that of a domain name or literal.  Nor
> >does it change the fact that domain names are case-insensitive.
> 
> If you replace <id-right> by <domain> (which is already defined in RFC
> 2822),

You have that backwards; RFC 2822 changed domain (as used in 822) to
id-right for the reason explicitly noted in 2822 ("identical except
that comments and folding white space are not allowed").
 
Still no substantiation of the claim that the semantics were explicitly
changed, I note.

2003-06-30:4

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1047 Path-identity: A proposal (perhaps)
Date: Thu, 30 Jun 2005 10:58:12 -0400
Cc: Harald Tveit Alvestrand <[email-masked]>
Sender: [email-masked]

On Wed June 29 2005 08:33, Harald Tveit Alvestrand wrote:
>   A path-identity identifies an agent. For the news system to work
>   properly, the path-identity must be globally unique.

If we mean "public name" we should probably say so: s/identifies/is a
public name identifying/

Global uniqueness is an interoperability requirement ("to work properly");
s/must/MUST/ would make that clear.

>   Traditional usage has had two means of generating unique identifiers:
>   DNS names and asserted identities. The first gets its uniqueness from
>   the DNS; the second gets its uniqueness from tradition. New servers
>   should use DNS names.

Because of the global uniqueness requirement, s/should/MUST/.  Add
something to address Frank's concern about unauthorized use.

>   In some cases, one wishes to embed something into a path-entity that
>   is not a name. One common thing to embed is an IP address, serving
>   as an identity when the name of the agent is not known.
> 
>    path-list       =  [FWS]
>                       *( ( path-identity / path-keyword / path-address )
>                       [FWS]
>                       path-delimiter [FWS] )
>                       tail-entry [FWS]
> 
>    path-identity   =  ( ALPHA / DIGIT )
>                       *( ALPHA / DIGIT / "-" / "." / "_" )
> 
>    path-keyword    = "POSTED" / "MISMATCH"

The ABNF implies that
  Path: MISMATCH!tail-entry
and
  Path: MISMATCH!MISMATCH!tail-entry
are legal, which I believe is not the intended use of the keywords. I
believe that the intent would be conveyed by changing the naked
"path-keyword" in the path-list rule by
  (path-keyword [FWS] path-delimiter [FWS] (path-identity/path-address))
If I understand the "some cases" argument correctly, the only place an
IP address literal should appear would be after a MISMATCH keyword and
delimiter.  If so, the ABNF could be further revised (simplified) to
so indicate.  There are other issues; the ABNF indicates that
  Path: POSTED!tail-entry
and
  Path: POSTED!!tail-entry
and
  Path: POSTED!!MISMATCH!tail-entry
are legal, and that also conflicts with the text.  I believe that the
ABNF can be fixed (and I'm willing to try), but first we need to reach
some consensus on where an IP address literal is and is not permitted.
Likewise for "POSTED" and "MISMATCH" (USEFOR currently defers to USEPRO,
which mentions neither).

>    path-address    =  IPv4address / no-fold-literal ; see [RFC2373]

RFC 2373 has been obsoleted by RFC 3513.  Obviously ("sometimes...")
RFC 3513 would be a *normative* reference.  However...

Neither RFC 2373 nor 3513 defines or uses "no-fold-literal".

While RFC 2373 had ABNF for "IPv4address" in an Appendix, RFC 3513
defines it nowhere and contains no ABNF.

RFC 2821 has ABNF for "IPv4-address-literal" and "IPv6-addr", which
might be useful.  If I understand yesterday's discussion correctly,
there will already be a normative (obviously) reference to RFC 2821.
Neither production permits CFWS.  I suggest using the aforementioned
2821 ABNF productions:
  path-address = IPv4-address-literal / ( "[" IPv6-addr "]" ) ; [RFC2821]
(but see below regarding brackets).

>   NOTE: The path-address contains characters that some systems consider
>      path-delimiters. This can cause problems; path-address should
>      therefore only be used when absolutely necessary.
>      See [USEPRO] for details on when to use them.

s/some systems/systems conforming to the previous specification/
(modulo bracket issues, which go away if the brackets go away; see below)
s/contains/may contain colon and bracket/
("may" because an IPv4address certainly contains no characters which
are interpreted as delimiters; the remainder to clarify the issue -- but
again, see below regarding brackets)

Because of the backward compatibility/interoperability problem
s/should/MUST/.

> Note that in recycling no-fold-literal, I've added [] around addresses... 
> and that a naked IPv4 address is allowed separately, because people have 
> said this is already present (too much for us to remove).
-------------------------------------
> Message-ID: <[email-masked]>
> In-Reply-To: <[email-masked]>
> References:  <[email-masked]>
>  <[email-masked]>
[...]
> --On onsdag, juni 29, 2005 15:52:55 +0200 Frank Ellermann 
> <[email-masked]> wrote:
[...]
> > Doesn't change the dead:beef:cafe:: problems, [ and ] are also
> > punctuation in 1036.
> 
> no, the only improvement from my proposed text with dead:beef:cafe is that 
> we isolate new-weird-characters like [:] to "diagnostic stuff" - we don't 
> write anything that lets sites name themselves dead:beef.

I think such naming is a red herring.  Good riddance to the possibility,
but it's not the major concern (it's covered by "use DNS names" in any
event).  The issue of interpretation of IPv6 address literal
colon-separated components as path identities remains an
interoperability problem.  We can defer detailed discussion to USEPRO,
but let's be clear that it is an interoperability problem which can
cause articles to fail to propagate properly and use BCP 14 imperatives
accordingly.

> And I think  
> discussion has shown that [] doesn't make the problem worse, so I see 
> little argument for not using them if we use them elsewhere.

I've seen no discussion of brackets on list in response to the proposed
change, other than Frank's comments...

Brackets are not explicitly named in RFC 1036 and are not in the set of
"punctuation" characters used by the RFC 1036 authors in B news 2.11.
Obviously, B news does not treat brackets as delimiters.  C news does.
INN does, according to Russ' 1999-06-19 message.

Bracket characters bracketing an IPv6-addr don't appreciably change
the situation one way or the other; eliminating the brackets or changing
"[" to "@!:^%" wouldn't affect C news or INN.  The characters add to
the length, that's about it.  The only thing I can see in favor of
retaining them it that they would prevent any B news implementations
or others using B news compatible code from misinterpreting the IPv4 part
of a hybrid literal (because B news doesn't treat brackets as delimiters
and "123.45.67.89" differs from "123.45.67.89]"), but then depending
on the (possibly compressed) hex part of the hybrid, treating the IPv4
part as different might be the wrong thing to do.  I'd prefer to lose
the brackets, since they're an unnecessary difference between treatment
of IPv4 and IPv6, seem to simply add length, and are not strictly part
of the RFC 1036 specification (which would complicate wording regarding
the backward compatibility issue).

2005-06-30:3

To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 30 Jun 2005 07:32:20 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> My reading of the text and the discussion is that the group wishes to
> allow a parser to NOT recognize "EST" as anything more valid than "ABC".
> (Otherwise, the whole fuss about requiring handling of "GMT" would be
> completely meaningless.)

I think this is a very iffy decision to make.  I can see not caring about
any time zone other than the RFC 2822 ones, but really, I don't see a lot
of excuse for not parsing all of the traditional textual time zones.

> If anyone but Bruce wants to argue in favour of recognizing *all*
> timezones mentioned in RFC 2822, which is the logical consequence of
> Bruce's argument, please speak up now.

I'll argue in favor of that, since otherwise we would be going from
blessing use if reluctantly (RFC 1036) to allowing parsers to disregard
them without any intervening deprecation period.

It would surprise me if there are any existing Usenet date parsers that
don't handle all of the RFC 2822 zones with the exception of the badly
specified and almost never seen military zones.

Date: Thu, 30 Jun 2005 16:19:23 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Sender: [email-masked]

--On torsdag, juni 30, 2005 08:59:34 -0400 Bruce Lilly <[email-masked]> 
wrote:

> As implementers will need to read the normative RFC 2822 text, a simple
> reference to section 4.3 of RFC 2822 should suffice.  Otherwise, the
> text should be as close as possible (i.e. identical) to the relevant
> 2822 text so as to avoid misinterpretation (Charles has said that an
> implementation might interpret "EST" as "unknown", which is certainly
> not the intent of RFC 2822).

My reading of the text and the discussion is that the group wishes to allow 
a parser to NOT recognize "EST" as anything more valid than "ABC".
(Otherwise, the whole fuss about requiring handling of "GMT" would be 
completely meaningless.)

This requires permitting the interpretation of "EST" as "unknown", which is 
not the intent of RFC 2822, but is (in my interpretation) the intent of 
this working group.

If anyone but Bruce wants to argue in favour of recognizing *all* timezones 
mentioned in RFC 2822, which is the logical consequence of Bruce's 
argument, please speak up now.
If there is none, I'll declare this issue closed.

                            Harald

Date: Thu, 30 Jun 2005 15:57:50 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Sender: [email-masked]

--On torsdag, juni 30, 2005 08:59:34 -0400 Bruce Lilly <[email-masked]> 
wrote:

>>  [...] However, the use of "GMT" as a time zone (part of <obs-
>>  zone>), although deprecated, is widespread in news articles today.
>>  Therefore, agents MUST accept <date-time> constructs which use the
>>  "GMT" zone.
>
> 2822 already requires that parsers accept the obs-zone abbreviations
> (with additional remarks about the 1-letter abbreviations), so that
> text is redundant and unnecessarily specific to "GMT".

Bruce, I have not heard anyone but you arguing that *all* OBS-syntax should 
be accepted. Can you now shut up about this issue, or raise an issue that 
specifically suggests alternate text for 2.1, where we say that OBS-syntax 
is a "MAY" accept, not a "MUST" accept?

Acting as if we had changed 2.1 when we have not is ... not useful.

                          Harald

To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 30 Jun 2005 06:48:42 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> ah - now I get the concern..... the requirement for uniqueness is not
> only that you generate names in a namespace that you control, but that I
> do *not* generate names in that namespace.

> Should we add a "MUST NOT generate msg-ids in id-rights you don't
> control"?

The $alz convention is still used for spam cancels, so I think that's at
most a SHOULD NOT.  (I know I was just arguing against caring that much,
but that was more an argument against documenting the convention in at
least the standards-track portions of the drafts.  I'm not sure that we
want to be encouraging it, particularly, but it does solve some real
problems.)

> (That's why I'm concerned with the syntax of RHS domain-literals; unless
> I know what other people are doing, I don't know what namespace I can
> claim without anyone else stomping on me...)

Yup.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT?  (fwd)
Date: Thu, 30 Jun 2005 09:09:57 -0400
Cc: Harald Tveit Alvestrand <[email-masked]>
Sender: [email-masked]

[...]
>  While we cannot change RFC 2822, it may be logical to change SHOULD

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date: Thu, 30 Jun 2005 08:59:34 -0400
Cc: Harald Tveit Alvestrand <[email-masked]>
Sender: [email-masked]

On Wed June 29 2005 07:21, Harald Tveit Alvestrand wrote:
> 
> Based on comments to Ken Murchison's text, I currently have the following 
> as resolution:
> 
>  [...] However, the use of "GMT" as a time zone (part of <obs-
>  zone>), although deprecated, is widespread in news articles today.
>  Therefore, agents MUST accept <date-time> constructs which use the
>  "GMT" zone.

2822 already requires that parsers accept the obs-zone abbreviations
(with additional remarks about the 1-letter abbreviations), so that
text is redundant and unnecessarily specific to "GMT".
 
>  NOTE: this specification does not change [RFC2822], which says
>   that agents MUST NOT generate <date-time> constructs which
>   include any zone names defined by <obs-zone>.
> 
>   Software that accepts dates with unknown timezones SHOULD treat such
>   timezones as equivalent to "-0000" when comparing dates, as specified
>   in [RFC2822] section 4.3.

There has been some discussion on list about the lack of clarity of
"unknown"; moreover, the suggested text implies that that is what RFC
2822 specifies, whereas the word "unknown" is not present in RFC 2822
at all.  The relevant 2822 text refers specifically to abbreviations
not included in the standardized ones which are in obs-zone for
compatibility with legacy software and archived messages; that text
is:

   Other multi-character (usually between 3 and 5) alphabetic time zones
   have been used in Internet messages.  Any such time zone whose
   meaning is not known SHOULD be considered equivalent to "-0000"
   unless there is out-of-band information confirming their meaning.

That text also refers to out-of-band information which is not mentioned
in the suggested text.

As implementers will need to read the normative RFC 2822 text, a simple
reference to section 4.3 of RFC 2822 should suffice.  Otherwise, the
text should be as close as possible (i.e. identical) to the relevant
2822 text so as to avoid misinterpretation (Charles has said that an
implementation might interpret "EST" as "unknown", which is certainly
not the intent of RFC 2822).

> (Since we're quoting requirements from 2822, I think it makes sense to put 
> the uppercase MUST and SHOULD in a NOTE - usually, it doesn't).

Quoting is OK; paraphrasing can lead to misinterpretations.

2005-06-30:1

Date: Thu, 30 Jun 2005 10:36:14 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked], Bruce Lilly <[email-masked]>
Subject: Re: Decision procedures
Sender: [email-masked]

Bruce,

I believe you are referring

--On onsdag, juni 29, 2005 12:19:37 -0400 Bruce Lilly <[email-masked]> 
wrote:

> On Tue June 28 2005 16:33, Harald Tveit Alvestrand wrote:
>
>> RFC 2418 section 3.3:
>>
>>    The other case is where the discussion has been held entirely over
>>    the mailing list.  The determination of the level of consensus may be
>>    harder to do in this case since most people subscribed to mailing
>>    lists do not actively participate in discussions on the list. It is
>>    left to the discretion of the working group chair how to evaluate the
>>    level of consensus.
>>
>> Alexey and I decide.
>
> The section also states:
>    Consensus
>    can be determined by a show of hands, humming, or any other means on
>    which the WG agrees (by rough consensus, of course).  Note that 51%
>    of the working group does not qualify as "rough consensus" and 99% is
>    better than rough.  It is up to the Chair to determine if rough
>    consensus has been reached.
>
> Yes, we know that you and Alexey decide *if* consensus has been reached,
> because that's explicitly what 2418 says.  But 2418 explicitly says that
> the WG gets to agree on the means.  I have posted what a past co-chair
> has indicated as the means, and I believe that that method had WG
> consensus behind it.

I believe you are referring to your posting:

> For reference, a past
> WG Chair had the following to say, and if I recall correctly, Alexey
> has agreed:
>   On the topic of voting: Especially on this topic, I think voting
>   would be stupid. The decision is about rough consensus. One screaming
>   person does not indicate that there is no rough consensus, but one or
>   two well-reasoned arguments against a screaming huge crowd does. And
>   a huge number of "I'd prefer X, but I couldn't care less" votes
>   versus 2 or 3 well-argued "X will spell doom for the Internet, and Y
>   will save it" votes *is* rough consensus for Y over X. So voting
>   generally doesn't help me decide one way or the other that there is
>   rough consensus.

This is a quote from Pete Resnick's message
<http://www.imc.org/ietf-usefor/2003/Jun/0099.html> - no need to be coy 
about who the previous chair is.

>From the same message:

>  (It's
>  also clear, BTW, that people are not listening to each other and
>  aren't trying to think of where we can come to consensus, which is
>  why people think they need a vote.)

Familiar?

> The question, still unanswered, is whether or not that method, which
> specifically calls for taking well-reasoned arguments into account, is
> being followed by the current co-chairs in determining *if* consensus
> is reached and where that consensus lies.  If so, it would help if the
> co-chairs were more vociferous about what fails to qualify as a
> well-reasoned argument; we have certainly had plenty of unsupported
> assertions and "false statement[s] deliberately presented as being true"
> (http://dictionary.reference.com/search?q=lie) -- it would also help if,
> when announcing that consensus has been reached, there were something
> more substantive than a list of names.  If not, there seems to be a
> problem, as I don't recall a WG consensus decision (based on that
> established means of determining consensus) to change the means of
> determining consensus.

One thing that probably should have been included: The fact that you think 
that you have a well-reasoned argument and your opponent's argument is 
totally beside the point has nothing to do with whether or not we have 
consensus.
In fact, you having what you think is a well-reasoned argument and totally 
failing to convince anyone else that the argument is reasonable is just 
another way of becoming the "rough" in "rough consensus". I will NEVER rule 
"rough consensus" on anything where I know that the majority of the group 
thinks it's a stupid idea.

When stating lists of names (which has happened rarely), I think of it 
mainly to make sure that the record shows that the decision was made in 
full knowledge of who does NOT agree; that these people have raised 
arguments against what has been ruled the consensus, their arguments have 
been heard, and the chairs have ruled that the group has consensus to 
proceed in spite of the disagreement.

I will absolutely refuse to poison the atmosphere of this list further by 
stating that specific people are lying. I'll also refuse to say that 
specific people are idiots, trolls, stiff-necked fools or any other 
epithets. What should matter in determining consensus is the content of the 
argument.

"False" is a statement about the argument. "Lie" is a statement about the 
person making it.

                          Harald

Date: Thu, 30 Jun 2005 09:21:43 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: #1047 Path-identity: A proposal (perhaps)
Sender: [email-masked]

--On onsdag, juni 29, 2005 15:52:55 +0200 Frank Ellermann 
<[email-masked]> wrote:

> Harald Tveit Alvestrand wrote:
>
>> path-address = IPv4address / no-fold-literal ; see [RFC2373]
>
> s/2373/3986/
>
>> recycling no-fold-literal, I've added [] around addresses...
>
> Doesn't change the dead:beef:cafe:: problems, [ and ] are also
> punctuation in 1036.

no, the only improvement from my proposed text with dead:beef:cafe is that 
we isolate new-weird-characters like [:] to "diagnostic stuff" - we don't 
write anything that lets sites name themselves dead:beef. And I think 
discussion has shown that [] doesn't make the problem worse, so I see 
little argument for not using them if we use them elsewhere.

There are only 3 ways I see to get rid of the : character:

- Stop putting domain-literals in Path: at all (seems unacceptable)
- Encode IPv6 addresses differently (seems unacceptable)
- Treat IPv6 and IPv4 differently (seems unacceptable)

If you've eliminated the impossible, what you're left with.....

Date: Thu, 30 Jun 2005 09:07:05 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Ralph Babel <[email-masked]>, [email-masked]
Subject: Re: #1022 USEFOR 2.1 obs-phrase: Resolution proposal
Sender: [email-masked]

--On onsdag, juni 29, 2005 18:03:25 +0200 Ralph Babel 
<[email-masked]> wrote:

> Harald Tveit Alvestrand wrote:
>
>> News articles MUST conform to the syntax specified
>> in Section 3 of [RFC2822]. Netnews agents
>
> "Netnews agent" isn't defined anywhere. Didn't
> we at some point agree that Usefor was about
> on-the-wire article _format_, not "agent" behavior?

A number of "agents" are defined in the definitions sections. I was using 
"Netnews agents" as a way to refer to all of them. Just "agents" are 
probably better. But if we're talking about accepting at all, we must be 
willing to talk about the ones who accept. Otherwise, there's eternal 
wandering around in passages of the passive voice to be found, with no 
actor in sight.

>> MAY also accept the obsolete syntax specified in
>> Section 4 of [RFC2822], and MUST accept the "UTC"
>> timezone (see section 3.1.2)
>
> What "UTC" timezone? "UT" maybe? And didn't we
> just get rid of that one (for whatever reason)?

My spelling error. I misspelled "GMT" as "UTC".

Date: Thu, 30 Jun 2005 09:04:20 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Sender: [email-masked]

--On onsdag, juni 29, 2005 22:35:23 +0200 Frank Ellermann 
<[email-masked]> wrote:

>>> I think this makes it clear that in message-id, the question
>>> of "what" the RHS is matters only to the generator, not to
>>> the parsing entity; all we care about after generation is
>>> that the message-ids are unique.
>
> But here I strongly disagree, I care _very much_ if somebody
> generates a <[email-masked]> without my permission
> because it's my name space.  And therefore <id-right> is a bad
> name.

ah - now I get the concern..... the requirement for uniqueness is not only 
that you generate names in a namespace that you control, but that I do 
*not* generate names in that namespace.

Should we add a "MUST NOT generate msg-ids in id-rights you don't control"?

(That's why I'm concerned with the syntax of RHS domain-literals; unless I 
know what other people are doing, I don't know what namespace I can claim 
without anyone else stomping on me...)

                    Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Date:  Wed, 29 Jun 2005 22:35:23 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 74
Sender: [email-masked]

Bruce Lilly wrote:

> As with orig-date and zone, a redefinition of 2822-defined
> grammar productions is a problem.  Changing the production
> name won't help in this case, as a msg-id is used in
> precisely three places: the Message-ID, References, and
> Supersedes fields,

Also in Control: cancel, the historical s-o-1036 Also-Control,
in article bodies, in news URLs (1738 + several drafts), and
in paths and parameters of http-URLs for various archives,
some of them based on NNTP.

That's why the old idea <msg-id-core> was changed to <msg-id>,
as you say _this_ production name should match RfC 2822.

> all defined in external specifications (2822 and 2156).

As well as 1036, 1738, s-o-1036, and 977bis, all with their
own POV, often some "maximal" concept incompatible with the
other "maximal" ideas.  We need some maximal "least common
denominator" for both syntax and semantics without conflicts.

>> If domain literals are used, the syntax found in [RFC2821]
>> section 4.1.3 is RECOMMENDED.

> I'm not sure about that; the 2821 tagging scheme doesn't seem
> to have generated much traction.  I hear that a 2821bis draft
> might appear soon

ACK, I hope it adopts the STD 66 syntax for domain literals,
maybe modulo the 2821-legacy "IPv6:" tag.

> Informative references to RFCs 1054, 1918, 2715, 3330, 3849
> would be helpful to implementers.

Yes, that's the Bruce as I know him, five RfCs I've never heard
of, I only check it, you say if any important errata exist.

1054: obsoleted by 1112 (STD 5), updated etc., multicasting IPs
1918: private IPs, I didn't recognize the number, shame on me
2715: informational (multicasting again)
3330: informational (special use IPs), added to my local disk,
      apparently the complete IPv4 zoo, nice
3849: IPv6 examples, I didn't recognize the number

Harald's wording for "stay away from domain literals" might be
good enough.  I still like STD 11, Dave's STRONGLY DISCOURAGED,
and those who aren't discouraged simply must know what they do.

>> I think this makes it clear that in message-id, the question
>> of "what" the RHS is matters only to the generator, not to
>> the parsing entity; all we care about after generation is
>> that the message-ids are unique.

But here I strongly disagree, I care _very much_ if somebody
generates a <[email-masked]> without my permission
because it's my name space.  And therefore <id-right> is a bad
name.

> It would seem that the same reasoning would apply to the LHS

No, the name space abuse case is a matter of the <id-domain>,
not of the <id-local>.  As the names say.  As they say it in
various relevant standards in different ways.

Excl. RfC 2822, that dropped the ball with its bad <id-right>
obscuring the real meaning and purpose.  It's not only about
the good idea "unique", it's also about the evil idea "abuse".

 [Further notes incompatible with 997bis and many UAs skipped]

                             Bye, Frank

To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Jun 2005 13:25:05 -0700
Sender: [email-masked]

Seth Breidbart <[email-masked]> writes:
> Russ Allbery <[email-masked]> wrote:

>> You have to parse the whole article before you can reject based on the
>> Path header.  Parsing the message ID lets you decline cancels you're
>> uninterested in before the article is sent to your server.

> Telling your neighbor "cyberspam is a synonym for me" means you don't
> even have to look at the message ID because your neighbor won't attempt
> to send you those cancels in the first place.

Yes, that's even better, but it assumes cooperative upstream peers that
can do this sort of configuration.  Per the feature requests we got for
INN, this was often not the case.  (Not to mention that many news sites
have hundreds of peers, and contacting them all is rather difficult.)

Date: Wed, 29 Jun 2005 16:05:23 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Sender: [email-masked]

Russ Allbery <[email-masked]> wrote:
> Richard Clayton <[email-masked]> writes:

>> Servers that wish to ignore cancels (or wish to distinguish amongst the
>> various parts) are clearly _intended_ to process the last few elements
>> of the Path: header field data, looking for the "pseudosite" strings
>> that they are prepared to accept (or not) [since they're not in the UUCP
>> message maps they're some of the most used non-DNS based strings
>> ... cyberspam, mmfcancel, bincancel, spewcancel and doubtless others
>> I've forgotten]
>
> You have to parse the whole article before you can reject based on the
> Path header.  Parsing the message ID lets you decline cancels you're
> uninterested in before the article is sent to your server.

Telling your neighbor "cyberspam is a synonym for me" means you don't
even have to look at the message ID  because your neighbor won't
attempt to send you those cancels in the first place.

Seth

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Date: Wed, 29 Jun 2005 15:25:34 -0400
Cc: Harald Tveit Alvestrand <[email-masked]>
Sender: [email-masked]

On Wed June 29 2005 05:36, Harald Tveit Alvestrand wrote:
> 
> OK, here's the proposed text for the problems I outlined yesterday:

> NEW:
> 
>    The Message-ID header contains a single unique message identifier.

Seems to need "field" per the decision on "header".

>    News is more dependent on msg-id uniqueness and fast comparision than
>    email is, and

I'm not convinced about "more"; email and news use several of the same
(i.e. combined functionality) UAs and protocols (e.g. IMAP) and the same
use of Message-ID, References, and Supersedes fields, the same "threading"
algorithms in those UAs, etc.  Uniqueness is certainly an issue for email
and for other applications layered on the Message Format, which itself
mentions global uniqueness.  I agree with Henry about "fast" also.

>    some news software would have trouble with the full range 
>    of possible msg-ids permitted by RFC 2822; this section therefore
>    restricts the syntax of <msg-id> compared to Section 3.6.4 of
>    [RFC2822].

Some news software may indeed have trouble, but I believe it is important
to distinguish between broken implementations with few global implications
e.g. UAs and isolated sites that have never complied with RFC 822, 850,
1036 syntax through more serious problems (NNTP constraints) to global
problems (none?).  More detail below.
 
>    The global uniqueness
>    requirement for <msg-id> in [RFC2822] is to be understood as applying
>    across all protocols using such message identifiers, and across both
>    Email and Netnews in particular.

OK.  I wouldn't mind the latter part expanded to "all applications which
use the Internet Message Format [RFC2822]".
 
> OLD (ABNF): [...] NEW:

As with orig-date and zone, a redefinition of 2822-defined grammar
productions is a problem.  Changing the production name won't help in
this case, as a msg-id is used in precisely three places: the Message-ID,
References, and Supersedes fields, and those are all defined in external
specifications (2822 and 2156).

That leaves implementation notes to deal with any concrete issues.

> NEW:
> 
>    Some software will try to match the <id-right> of a msg-id in
>    a case-insensitive fashion; some will match it in a case sensitive
>    fashion. Implementations MUST NOT generate two message-IDs where the
>    only difference is the case of characters in the <id-right> part.

That's OK as one implementation note.
 
>    If domain literals are used, the syntax found in [RFC2821] section
>    4.1.3 is RECOMMENDED.

I'm not sure about that; the 2821 tagging scheme doesn't seem to have
generated much traction.  I hear that a 2821bis draft might appear soon,
perhaps some off-list communication with JCK and IPv6 experts might be
in order.
 
>      NOTE: [RFC2822] section 3.6.4 recommends that the <id-right> should
>      be a domain name or a domain literal. Domain literals are troublesome
>      since many IP addresses are not globally unique; domain names are
>      more likely to generate unique message-IDs.

OK.  Informative references to RFCs 1054, 1918, 2715, 3330, 3849 would be
helpful to implementers.

>      In no circumstance is it reasonable for software receiving a <msg-id>
>      to change behaviour based on a theory about how the <msg-id> was
>      generated.

Pick British or American spelling; I don't care as long as it's consistent.
 
> I think this makes it clear that in message-id, the question of "what" the 
> RHS is matters only to the generator, not to the parsing entity; all we 
> care about after generation is that the message-ids are unique.

It would seem that the same reasoning would apply to the LHS, i.e. there's
no problem with <"foo"@example.edu> so long as example.edu doesn't generate
ids like <[email-masked]>, <f\[email-masked]>, etc.

Lacking any concrete statements from other WG participants about specific
issues, and noting that whitespace is already prohibited in a msg-id by
2822 syntax, here's a stab at some more implementation notes:

  NOTE: The NNTP protocol [RFC977] has no provision for quoting and uses
        the space character as a delimiter.  While msg-id syntax prohibits
        CFWS in a msg-id, a space character is permitted within a
        quoted-pair within a quoted-string in id-left, e.g.
        <"foo\ bar"@example.org>.   Implementations SHOULD therefore avoid
        such an id-left construct.

"SHOULD" because a "private" installation of "cooperating" "systems" known
not to use NNTP need not be concerned about NNTP foibles.

My understanding is that the NNTP issues affects only the Message-ID field
directly, so such a note could go in that field's section.  For that
matter, it's an external protocol issue, so it could go in USEPRO or USEAGE.

  NOTE: The NNTP protocol [RFC977] transmits msg-ids in a fixed-length
        command-line.  While the length of a msg-id is constrained (to 997
        octets including angle brackets by the line length limit of
        [RFC2822]), NNTP cannot handle a msg-id longer than 500 octets
        due to the command-line limit.  Implementations SHOULD therefore
        not generate a msg-id longer than 500 octets including angle
        brackets.

Same rationale as above for "SHOULD".  Same regarding placement.

  NOTE: Although the length of a msg-id is constrained to 997 octets, and
        a legal domain name may be up to 255 octets in length, some news
        software implementations silently truncate a msg-id at 255 octets.
        Such implementations SHOULD be upgraded as soon as practicable to
        support a msg-id length of 997 octets.  Implementations that
        exchange messages which may be sent to a legacy implementation
        should limit msg-id length to no more than 255 octets if possible.

Clearly the problem here is some legacy implementations, since a domain name
has always been allowed to be up to 255 octets (RFC 882).  The onus is
placed on ancient implementations to modernize to 1980s technology, while
interoperating implementations are encouraged to play nice where possible
(clearly a site with a 252-octet domain name cannot do so).  Placement in
USEFOR or USEAGE would probably be fine.

2005-06-29:5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 msg-id issues - attempt to focus
Date:  Wed, 29 Jun 2005 21:10:38 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 44
Sender: [email-masked]

Bruce Lilly wrote:

>> that's what I posted (truncated header):

>> Message-ID: <"\=1B[7;0m"@xyzzy.claranet.de>
[...]

> gmane isn't "Netnews", it's an interface to mailing lists.

It's a news server presenting mailing lists as moderated NGs
(+ one real group =B0junk") plus some magic to get message-IDs
in "X-Posts" (mails to several lists) right.

But okay, let's say that one test on GMaNe isn't good enough,
I repeated it with the same Message-ID in alt.test and:

X-Mozilla-Status: 0001
Message-ID: <"\=1B[7;m"@xyzzy.claranet.de>
Date: Wed, 29 Jun 2005 20:24:33 +0200
From: Message-ID Test <[email-masked]>
Newsgroups: alt.test
Subject: ignore noreply Message-ID Test

It didn't appear.  It didn't show up in junk.  I tested a
normal article in claranet.test, no problem.  If the test
article is for some obscure reason delayed we'll see it as
<http://purl.net/msgid/[email-masked]>

> the ad hominem is unwelcome.

NetNews two *1) nil  Better ?

*1: the person claiming that NO-WS-CTL worked for decades:
    <http://article.gmane.org/gmane.ietf.usenet.format/27849/match=3Ddeca=
des>

*1: nice search engine, <http://rain.gmane.org>  You can test
    author: [email-masked], group: gmane.ietf.usenet.format,
    search word: lie (12 hits),

*1: as long as I don't say "Mr. Lilly" there's no real problem
    from my POV.
                           Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 Path field delimiters - feed behaviour? (fwd)
Date:  Wed, 29 Jun 2005 19:59:18 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

Seth Breidbart wrote:
 
> One April 1, admins from The Hierarchy That Shall Not Be
> Named all decided to add one bogus entry to their Path

Curious, what hierarchy might that be, something starting
with "f" or ending with "2" ?

> It typically showed up several times in every article,
> with no observable effects.

For a _bogus_ path entry I'd expect no effect, but for a
_real_ path entry it could be interesting.  Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date:  Wed, 29 Jun 2005 19:39:21 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 19
Sender: [email-masked]

Charles Lindsey wrote:

> a <path-identity> is a "public name", identifying a
> particular server.

Fine, extend this good idea to s/id-right/is-domain/ ;-)

> Random sites in random places are already using
> "clerew.man.ac.uk", judging by the bogus bounce messages
> I receive from time to time :-( .

Publish a v=spf1 sender policy, it works after some weeks.
State "approved", substate "point raised - writeup needed".

> SHOULD be an A record for the agent concerned, or an MX
> record that is mailable

Don't forget the AAAA case, bye, Frank

To: [email-masked]
Subject: Re: #1047 Path-identity: A proposal (perhaps)
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Jun 2005 10:41:06 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> At the moment, it seems that we're at an impasse partially caused by our
> documents' failure to explain what a "path-identity" is. I don't know if
> I understand it either....

> At the moment, the specific format and use of path-identity is punted to
> USEPRO, where it seems to be still undefined.

> If we explained it, would this be a reasonable text to insert in 3.1.6?
> (I've modified the BNF too. Only modified stuff included here)

This looks okay to me.

To: [email-masked]
Subject: Re: #1042 USEFOR 3.1.5 Newsgroups folding - resolution proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Jun 2005 10:37:22 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> Suggested addition to section 3.1.5:

> Folding the Newsgroups: header field over several header lines has been
> shown to harm propagation significantly. Folded Newsgroups header fields
> SHOULD NOT be generated, but MUST be accepted.

> The only valid reason identified at this time for generating folded
> Newsgroups header fields is to test propagation of articles using that
> feature.

Looks good to me.

To: [email-masked]
Subject: Re: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT?  (fwd)
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Jun 2005 10:36:55 -0700
Sender: [email-masked]

To: [email-masked]
Subject: Re: #1022 USEFOR 2.1 obs-phrase: Resolution proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Jun 2005 10:33:46 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> I suggest the following text change to section 2.1 of USEFOR:

> OLD:

>    News articles MUST conform to the syntax specified in Section 3 of
>    [RFC2822].  Netnews agents MAY also accept the obsolete syntax
>    specified in Section 4 of [RFC2822], but they MUST NOT generate
>    productions of such syntax.

> NEW:

>    News articles MUST conform to the syntax specified in Section 3 of
>    [RFC2822].  Netnews agents MAY also accept the obsolete syntax
>    specified in Section 4 of [RFC2822], and MUST accept the "UTC"
>    timezone (see section 3.1.2) and the obs-phrase construct (see
>    [RFC2822] section 4.1) but they MUST NOT generate productions of such
>    syntax.

You meant GMT here, I assume?

Other than that, this looks fine to me.

To: [email-masked]
Subject: Re: #1021 USEFOR Newsgroups header - resolution proposed
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Jun 2005 10:32:12 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> The following newsgroup names have been used for specific purposes in
> various implementations and protocols, and MUST therefore not be used
> for the names of normal newsgroups. They MAY be used for their specific
> purpose, or by local agreement.

>  Groups starting with the component "junk"

This rule is not strictly necessary.  What do other people think about
allowing things like junk.foo?

You need to add:

   Groups starting with the component "control"

as well; it's not just the group "control".

Other than that, I'm okay with this.  I'd still rather that the protocol
stuff be in USEPRO, but after thinking about it, I don't feel strongly
enough about it to argue it at any additional length, and I don't think it
really hurts much to put it into USEFOR.

I wouldn't object to the changes Forrest suggested either, though.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047: Path field delimiters and components
Date:  Wed, 29 Jun 2005 19:29:38 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 10
Sender: [email-masked]

Charles Lindsey wrote:

> At that point of time, the probability of their existing any
> then current <path-identity> that will cause problems when
> ":" starts to appear (in IPv6 addresses, for example) is
> exceedingly small.

BTW, that's an advantage of Harald's idea, he restricts this
to _only_ IPv6, after s/no-fold-literal/IPv6address/  Bye.

Date: Wed, 29 Jun 2005 13:29:48 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Sender: [email-masked]

Frank Ellermann wrote:

> Yesterday Charles wrote something in the direction of
> SHOULD NOT => permission, and now you apparently propose
> SHOULD NOT => allowed.  Do all here think that a SHOULD
> (NOT) is something like "please (not)", or worse that
> SHOULD NOT is just a shorthand for "of course you can" ?

"All" here?  Please don't be ridiculous!

We need not care about people who cannot read and
correctly understand RFC2119 language.   We can ignore
that they exist when writing.

That leaves us with the ability to use RFC2119 terms
as defined.  So "SHOULD NOT" means "do it this way
only if you have no good alternative in full awareness
and consideration of the pain you cause yourself or
others."

> There is a reason to allow generation, legacy software.
>
> Decreeing that legacy software is non-conformant without
> good reason is also a dubious idea.  And in my parallel
> universe it's not allowed to implement any new software
> generating GMT if the specification says "SHOULD NOT" -
> legacy software is the only possible excuse in this case.

There are N different ways that legacy software will not
conform to these drafts.  To me, that means that making
legacy software non-conformant is not a serious factor
in making a choice over one wording or another, with timezones or
anything else.

BTW, does RFC2822 use the "MUST NOT" language for this?  Instead
I think it says "zone MUST be within the range -9959 through +9959"
That is subtly different.

A "MUST NOT" is taken to mean that something breaks, so a MUST NOT
GENERATE/MUST ACCEPT may be correct, but I agree is kind of weird.
I like the RFC2822 "MUST generate/MUST accept" instead.  (...which we
need not repeat.)

To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Jun 2005 10:28:26 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> OK, here's the proposed text for the problems I outlined yesterday:

This looks good to me.

To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Jun 2005 10:27:29 -0700
Sender: [email-masked]

Richard Clayton <[email-masked]> writes:

> Although I can imagine that there exists software that does practically
> anything -- I believe that the intent of the conventions is that the
> $alz scheme (which means that one is able to predict the cancel message
> ID) is to ensure that only one cancel floods out, no matter how many
> robots generate it (in the extended scheme with a secret, the cabal of
> cancellers share the secret amongst themselves -- same difference!)

> Servers that wish to ignore cancels (or wish to distinguish amongst the
> various parts) are clearly _intended_ to process the last few elements
> of the Path: header field data, looking for the "pseudosite" strings
> that they are prepared to accept (or not) [since they're not in the UUCP
> message maps they're some of the most used non-DNS based strings
> ... cyberspam, mmfcancel, bincancel, spewcancel and doubtless others
> I've forgotten]

You have to parse the whole article before you can reject based on the
Path header.  Parsing the message ID lets you decline cancels you're
uninterested in before the article is sent to your server.

INN does indeed support this (off by default) due to significant popular
demand.

The $alz convention is a kludge, though, and I'm not sure that the
environment in which this sort of thing was important, back in the days of
hundreds of thousands of cancels a day, is still as much of a concern.

Date: Wed, 29 Jun 2005 10:24:32 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1047 Path field delimiters - feed behaviour? (fwd)
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>I haven't seen an answer to this question - would anyone else like to 
>answer?

Russ, as I recall, explained the problem quite sufficiently.  Charles'
question does not demonstrate the problem, and I suspect was chosen
specifically because it does not.

The issue is not systems that have no special relationship to any of the
problem components of the path header. They will do nothing differently.  
That's pretty obvious. If I see "dead" or "dead:beef" in the Path header
field, and I neither feed nor accept articles from either, then of
course there is no difference.

However, if I feed site "dead" and I am RFC1036 conformant, then no 
article that has passed through the site "dead:beef" will I feed to site 
"dead". Nor will I feed the site "dead" any article that has passed 
through the site using the IPv6 domain-literal 
"ff::dead:128.99.32.1".

Furthermore, if I feed site "dead:beef" and am RFC1036 conformant, then I 
will happily feed any article I get that has already passed through 
"dead:beef" back to "dead:beef", since I will not see the path identity 
"dead:beef" in the Path header field, only the two sites "dead" and 
"beef". 

This is not a complete meltdown of the news system, but then, nobody has 
ever said it was. It is a serious disruption of the news feed to the site 
"dead", however. 

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Date:  Wed, 29 Jun 2005 19:16:28 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 23
Sender: [email-masked]

Ralph Babel wrote:

> Actually, you don't recall correctly.

<http://www.xs4all.nl/~rosalind/faq-care.html#cancel-message>
<http://www.killfile.org/faqs/cancel.html#II.B.> $alz-convention
<http://www.killfile.org/faqs/cancel.html#II.C.> cancel example

> Gee, folks, it's not rocket science.

Not guilty, convince Rosalind and Skirvin.  I already had my
fun with trying to cancel two unjustified cancels.  Your idea
also answers Harald's question, sometimes a <msg-id> is parsed.

And you implicitly said that the RHS is important, how about
joining the s/id-left@id-right/id-local@id-domain/ club ?

The news-nntp-uri drafts want at least the "@" (unlike NNTP):
http://purl.net/xyzzy/home/test/draft-lindsey-news-nntp-uri-00
http://purl.net/xyzzy/home/test/draft-ellermann-news-nntp-uri-00

                           Bye, Frank

Date: Wed, 29 Jun 2005 13:09:35 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Sender: [email-masked]

Frank Ellermann <[email-masked]> wrote:
> Seth Breidbart wrote:
 
>> I agree it's odd.  I still think we should do it.
> [...]
>> "MUST NOT generate" because there's no reason to allow
>> generation
>
> Yesterday Charles wrote something in the direction of
> SHOULD NOT => permission, and now you apparently propose
> SHOULD NOT => allowed.  Do all here think that a SHOULD
> (NOT) is something like "please (not)", or worse that
> SHOULD NOT is just a shorthand for "of course you can" ?

I think that SHOULD NOT is more permissive than MUST NOT.

"MUST NOT" = don't do it.  Period.

"SHOULD NOT" = don't do it unless you have a good reason.

> There is a reason to allow generation, legacy software.

That's a reason to require acceptance (which we all agree should be
done).

>> allowing more than FRC 2822 (it says "MUST NOT
>> generate") is something we ought to do only with very
>> good reason; oddness of wording isn't such a reason.
>
> Decreeing that legacy software is non-conformant without
> good reason is also a dubious idea.

So the question is which is worse: being _more permissive_ than RFC
2822 (which we aren't anywhere else, are we?) or decreeing that legacy
software is non-conformant with a new spec?

>  And in my parallel universe it's not allowed to implement any new
> software generating GMT if the specification says "SHOULD NOT" -

Unfortunately, some implementors won't agree with you.  They'll find
other reasons that are good enough for them.  (At least, that's my
guess.)

> legacy software is the only possible excuse in this case.

I suspect people are more ingenious at coming up with excuses than you
believe.

Seth

Date: Wed, 29 Jun 2005 13:05:55 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Sender: [email-masked]

Bruce Lilly <[email-masked]> wrote:

> It's also unclear exactly how this works in practice: suppose servers A
> and B participate in "Xref-slaving".  At some point in time, server A
> receives a crossposted article (newsgroups X and Y, with article numbers
> 12345 and 98765 respectively) from somewhere other than B, includes a
> field
>  Xref: A X:12345 Y:98765
> and forwards it to B, which presumably files the article in the same
> newsgroups with the same numbers.

That means that B is the slave of A.

>  Subsequently, server B receives (not from A; perhaps from one of
> those thousands of clients from all over the globe which are
> connected as clients to server B) an article posted only to group Y,
> and files it as article number 98766.

That would make B a peer of A, not a slave.  Slaves don't get to
assign article numbers.

Seth

To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Jun 2005 10:00:56 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Right! I see your problem now. How about:

>    7.   It MUST (exept when specially configured to preserve the
>         <article-locator>s set by the sending site) remove any Xref
>         header (F-3.2.11) from each article.  It then MAY (and usually
>         will) generate a fresh Xref header.

That sounds better to me, yes.

Date: Wed, 29 Jun 2005 12:56:14 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 msg-id issues - attempt to focus
Sender: [email-masked]

"Charles Lindsey" <[email-masked]> wrote:

> My opinion is that I see no need to be more restrictive than RFC 2822
> here, and that it is better to preserve the names <id-left> and <id-right>
> for consistency with RFC 2822, even though we give them new syntax.

I think that's less consistent (and more confusing) than using new
terms for the new syntax.

Seth

To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Jun 2005 09:55:19 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:
> On Tue June 28 2005 22:46, Russ Allbery wrote:

>> Except that the text you included says:

>>     7. It MUST remove any Xref header (F-3.2.10) from each article.  It
>>        then MAY (and usually will) generate a fresh Xref header.

>> This is wrong for Xref slaving.  The serving agent does not remove the
>> Xref header when Xref slaving is being used.

> Russ, I think I need some clarification:

> Are you saying that the "server-name" part remains unchanged?

Yup.

> Also, if not, (i.e. the "server-name" is the same), how is this
> effectively different (from the POV of any connecting client) from "one
> big serving agent"?

The whole point of Xref slaving is to try to make things look the same to
the connecting client.  However, the connecting client isn't the only
point of view or the only connection that we have to think about.  It's
completely different from the perspective of the serving agents
themselves.

> It's also unclear exactly how this works in practice: suppose servers A
> and B participate in "Xref-slaving".  At some point in time, server A
> receives a crossposted article (newsgroups X and Y, with article numbers
> 12345 and 98765 respectively) from somewhere other than B, includes a
> field

>   Xref: A X:12345 Y:98765

> and forwards it to B, which presumably files the article in the same
> newsgroups with the same numbers.

Right.

> Subsequently, server B receives (not from A; perhaps from one of those
> thousands of clients from all over the globe which are connected as
> clients to server B) an article posted only to group Y, and files it as
> article number 98766.

You can't Xref-slave against more than one server at the same time, nor
can you assign numbers locally.  In an Xref slaving configuration, B will
perform its normal injection agent duties and then rather than storing the
article locally will immediately relay it to A *without* assigning article
numbers.  It will then wait for A to send it back with article numbers
assigned before storing it locally.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date:  Wed, 29 Jun 2005 18:47:07 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 30
Sender: [email-masked]

Seth Breidbart wrote:
 
> I agree it's odd.  I still think we should do it.
[...]
> "MUST NOT generate" because there's no reason to allow
> generation

Yesterday Charles wrote something in the direction of
SHOULD NOT => permission, and now you apparently propose
SHOULD NOT => allowed.  Do all here think that a SHOULD
(NOT) is something like "please (not)", or worse that
SHOULD NOT is just a shorthand for "of course you can" ?

There is a reason to allow generation, legacy software.

Russ found mumble (more than 10% IIRC) GMT timestamps,
and unlike the also popular EDT / EST a GMT is harmless.

> allowing more than FRC 2822 (it says "MUST NOT
> generate") is something we ought to do only with very
> good reason; oddness of wording isn't such a reason.

Decreeing that legacy software is non-conformant without
good reason is also a dubious idea.  And in my parallel
universe it's not allowed to implement any new software
generating GMT if the specification says "SHOULD NOT" -
legacy software is the only possible excuse in this case.

                       Bye, Frank

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 msg-id issues - attempt to focus
Date: Wed, 29 Jun 2005 12:47:46 -0400
Cc: Frank Ellermann <[email-masked]>
Sender: [email-masked]

On Tue June 28 2005 16:10, Frank Ellermann wrote:
> 
> Frank Ellermann wrote:
> 
> > P.S.:  Just for fun I test it with this article.
> 
> Here's what I got back from GMaNe:
> 
> http://article.gmane.org/gmane.ietf.usenet.format:29206:raw
> Message-ID: <[email-masked]>
> 
> And that's what I posted (truncated header):
> 
> Message-ID: <"\[7;0m"@xyzzy.claranet.de>
> Date: Tue, 28 Jun 2005 20:57:02 +0200
> Content-Type: text/plain; charset=us-ascii
> Content-Transfer-Encoding: 7bit
> 
> NetNews one Bruce nil.  Bye, Frank

First, gmane isn't "Netnews", it's an interface to mailing lists.
Second, gmane is know to make specific transformations to some
message identifiers, so you have suceeded in showing that a non-
"Netnews" system in fact makes transformations that it was a priori
known to make; no surprise.
Third, the ad hominem is unwelcome.

2005-06-29:4

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Wed, 29 Jun 2005 12:40:36 -0400
Cc: Russ Allbery <[email-masked]>
Sender: [email-masked]

On Tue June 28 2005 22:46, Russ Allbery wrote:
> 
> Charles Lindsey <[email-masked]> writes:

> > I think it is OK. If you have a farm of serving agents that keep their
> > Xrefs in step, do you regard passing articles between those agents as
> > "relaying"
> 
> Yes.
> 
> > IOW, is that whole farm just "one big serving agent" from the POV of our
> > draft?
> 
> No.  There are reasons why you might want to Xref slave to a news feed
> that isn't at your site or even run by you.  They're rare, but there's no
> particular reason to disallow it.  (Consider, for example, the case where
> you run a personal local news server as basically a cache for your ISP's
> remote news server and want to be able to switch between servers whenever
> you feel like it without renumbering.)
> 
> > If the answer is "relaying", then the present "relayers MAY delete"
> > suffices.
> 
> Except that the text you included says:
> 
>     7. It MUST remove any Xref header (F-3.2.10) from each article.  It
>        then MAY (and usually will) generate a fresh Xref header.
> 
> This is wrong for Xref slaving.  The serving agent does not remove the
> Xref header when Xref slaving is being used.

Russ, I think I need some clarification:

Are you saying that the "server-name" part remains unchanged?  If not, how
is the end result conceptually different from remove/generate?  Also, if
not, (i.e. the "server-name" is the same), how is this effectively
different (from the POV of any connecting client) from "one big serving
agent"?
 
It's also unclear exactly how this works in practice: suppose servers A
and B participate in "Xref-slaving".  At some point in time, server A
receives a crossposted article (newsgroups X and Y, with article numbers
12345 and 98765 respectively) from somewhere other than B, includes a
field
  Xref: A X:12345 Y:98765
and forwards it to B, which presumably files the article in the same
newsgroups with the same numbers.  Subsequently, server B receives (not
from A; perhaps from one of those thousands of clients from all over the
globe which are connected as clients to server B) an article posted only
to group Y, and files it as article number 98766.  At the same time,
server A receives (from one of its global clients) an article crossposted
to newsgroups X and Y again, which it duly files as articles 12346 and
98766 on A, with an Xref field
  Xref: A X:12346 Y:98766
and forwards to B.  What happens to the very different article in newsgroup
Y on B which is already numbered 98766?

2005-06-29:3

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: Decision procedures (Re: #1043 "header": Suggested resolution)
Date: Wed, 29 Jun 2005 12:19:37 -0400
Cc: Harald Tveit Alvestrand <[email-masked]>
Sender: [email-masked]

On Tue June 28 2005 16:33, Harald Tveit Alvestrand wrote:

> RFC 2418 section 3.3:
> 
>    The other case is where the discussion has been held entirely over
>    the mailing list.  The determination of the level of consensus may be
>    harder to do in this case since most people subscribed to mailing
>    lists do not actively participate in discussions on the list. It is
>    left to the discretion of the working group chair how to evaluate the
>    level of consensus.
> 
> Alexey and I decide.

The section also states:
   Consensus
   can be determined by a show of hands, humming, or any other means on
   which the WG agrees (by rough consensus, of course).  Note that 51%
   of the working group does not qualify as "rough consensus" and 99% is
   better than rough.  It is up to the Chair to determine if rough
   consensus has been reached.

Yes, we know that you and Alexey decide *if* consensus has been reached,
because that's explicitly what 2418 says.  But 2418 explicitly says that
the WG gets to agree on the means.  I have posted what a past co-chair
has indicated as the means, and I believe that that method had WG
consensus behind it.

The question, still unanswered, is whether or not that method, which
specifically calls for taking well-reasoned arguments into account, is
being followed by the current co-chairs in determining *if* consensus
is reached and where that consensus lies.  If so, it would help if the
co-chairs were more vociferous about what fails to qualify as a
well-reasoned argument; we have certainly had plenty of unsupported
assertions and "false statement[s] deliberately presented as being true"
(http://dictionary.reference.com/search?q=lie) -- it would also help if,
when announcing that consensus has been reached, there were something
more substantive than a list of names.  If not, there seems to be a
problem, as I don't recall a WG consensus decision (based on that
established means of determining consensus) to change the means of
determining consensus.

2005-06-29:2

Date: Wed, 29 Jun 2005 18:16:11 +0200
From: [email-masked] (Ralph Babel)
To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Sender: [email-masked]

Frank Ellermann wrote:

> In that case an intentional collision, one <cancel.old@id>
> is enough to cancel the <old@id> spam. In a more
> elaborated form it's <cancel.secret.old@id>, because the
> bad guys started to send a dummy <cancel.old@id> before
> the <old@id> spam, this dummy blocked the spam cancel.
> IIRC,

Actually, you don't recall correctly. The bad guys are the
ones that pollute other people's message-ID space instead
of agreeing on a canonical way of encoding the original
message ID in the cancel message ID _without_ resorting
to the original RHS, e.g. by moving it to the LHS of the
cancel's message ID. Gee, folks, it's not rocket science.

To: [email-masked]
Xref: clerew local.usefor:21703
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 msg-id issues - attempt to focus
Date: Wed, 29 Jun 2005 14:06:55 GMT
Lines: 62
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>1 - Is News message-ID different from 2822 message-id?
>  As far as I can see, consensus YES - there exist valid 2822 message-iDs 
>that are not valid News message-ids, but not vice versa (News msgid is a 
>strict subset of 2822 msg-id). And current text (with bug fixed) is a 
>reasonable description of the subset.

>2 - Is the RHS of the message-ID a domain, a domain|ipv4literal or a 
>domain|ipv4literal|ipv6literal, or an "opaquestring"?

>  As far as I can see, there is no clear-cut consensus.

>3 - Is the RHS of the message-ID case sensitive or case insensitive?

>  Here, I think consensus is clear, based on existing implementations' 
>behaviour: Once generated, you cannot change them, and making 2 message-IDs 
>with the same LHS and only different in case on the RHS is NOT a good idea.

>[deep breath]
>Is this a fair statement of the issues being discussed?

Pretty well.

#1 just needs the bug fix.

#2 and #3 are related, insofar as once you have devised the <msg-id>, then
it is opaque from then on.

RFC 2822 clearly allows, and even RECOMMENDs, using a domain-name or an
IP-literal as the <id-right>. Frank wants two things:

#2a) To deprecate IP-literals in favour of domain-names. That would go
beyond what RFC 2822 is saying.

#2b) To use more "suggestive" names in place of <id-left> and <id-right>
(that would not make any technical difference).

My opinion is that I see no need to be more restrictive than RFC 2822
here, and that it is better to preserve the names <id-left> and <id-right>
for consistency with RFC 2822, even though we give them new syntax.

Note that I have proposed the following text as being slightly clearer
that the wording currently in USEFOR:

     NOTE: It in RECOMMENDED in [RFC 2822] that, for ensuring global
     uniqueness, the <id-right> be some domain identifier within whose
     scope the uniqueness of the <id-left> can be guaranteed. Hence, when
     following this recommendation, any <no-fold-literal> used for the
     <id-right> should be of the form of an Internet address, in
     accordance with Section 3.4.1 of [RFC2822].

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21698
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Wed, 29 Jun 2005 13:12:19 GMT
Lines: 60
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon June 27 2005 12:33, Charles Lindsey wrote:
>> 
>> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
>> 
>> >We need to specify the semantics of a "path-identity" as one of:
>> >o an opaque label
>> >o a public name
>> >o domain names, IP-addresses, etc.
>> 
>> No we don't.
>> 
>> 1. I am not clear what is the distinction between "opaque label" and
>> "public name", or whether it matters for our purpose.

>See RFC 1958.

Yes. Having read that, I think a <path-identity> is a "public name",
identifying a particular server.

>So you see no problem in some random site in some random place with
>some random affiliation using the opaque label "clerew.man.ac.uk".

Random sites in random places are already using "clerew.man.ac.uk",
judging by the bogus bounce messages I receive from time to time :-( .

However, the text that is to go in Usefor states quite clearly that any
FQDN used as a <path-identity> SHOULD be an A record for the agent
concerned, or an MX record that is mailable. I would have thought it
pretty obvious that any FQDN or IP address used was intended to be one
relevant to that agent, but if you want me to spell that out in
excruciating detail, then I could do so.
> 
>> >> Also, in order to detect MISMATCHes, you will likely need to know whether
>> >> the IP address the NNTP packet
>> 
>> >NNTP is not the only transport mechanism.
>> 
>> So?

>So a mismatch in source of UUCP data requires no IP address information,
>and your argument equating a specific case with the general case fails.

You missed the word "likely". There are ways other than the IP address
used in packets for identifying the site that is relaying to you - for
example an authorization identity obtained from a SASL exchange.
Naturally, as you observe, UUCP data will not be accompanied by an IP
address, but UUCP incorporates its own methods of identitying the sender.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21705
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Admin: Ticket status
Date: Wed, 29 Jun 2005 14:41:59 GMT
Lines: 67
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>I'm listing the tickets in numeric order.

>1002 USEFOR 6. References - Updates needed

> Resolution proposed on June 14, no objection to content.
> Status: "text accepted".

I still hanker after a bit more semantic text in USEFOR, and even invited
Forrest to raise a ticket to create some. But I am not pushing it. What is
there, plus what is now in USEPRO will suffice, - just.

>1004 USEFOR 3.2.14 - Can we deprecate X- headers?
> Status: Text proposed. Some debate on X-Mailer.

I prefer no action.

>1021 USEFOR 3.1.5 Newsgroups header ABNF and description needs cleanup
>  Status: "Text proposed" - debate. Need to review.

>1022 USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
>  Status: "No consensus" - still only 2 voices.

I thought we were agreed to allow it. It was certainly so in our earlier
drafts.

>1032 USEFOR general: Document changes from RFC 1036
>  Status: "Text needed"

There is text currently in USEPRO, which should be reviewed (also the
"transitional arrangements" text). Maybe it finally gets moved to USEFOR.

>1042 USEFOR 3.1.5: Newsgroups folding?
>  Status: "Text needed". I think there's consensus that at this time,
>  we cannot recommend folding newsgroups.

I think it is agreed that folding should be in the syntax and hence MUST be
accepted (though it will take a while).

The issue is whether we say MUST NOT generate, SHOULD NOT generate (cf
related matter of comments in References), or SHOULD NOT generate until
... (or various other weaker formulations). Could we settle on SHOULD
NOT?

>1046 USEFOR 5. MIME boundary security considerations
>  Status: "Text needed". Principle seems uncontroversial.....

Does not the problem already exist in mail? Have mail standards raised
this as a security consideration?

>1047 USEFOR 3.1.6: Path field delimiters and components
>  Status: "Text needed"

There is a NOTE I proposed in Reply to John. It probably needs more work.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21704
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW USEPRO 7.9 gateway (was: LTRU questions)
Date: Wed, 29 Jun 2005 14:18:32 GMT
Lines: 34
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>In <[email-masked]>
>Harald wrote...

>| So far, no tickets have been opened on USEPRO.

>...therefore I test this procedure now, what do we expect
>from x2news gateways trying to inject articles with an 
>"unfoldable" subject or similar header field body line ?

The current USEPRO text requires the gateway to produice a valid Netnews
article, somehow or other.

I would leave it to the gateway implementor to decide for himself how to
do this. We do provide some pointers in cases which could lead to severe
interoperability, looping, etc, but I do not think empty Subjects come
into that category (indeed, they will probably propagate perfectly well,
modulo a few sites which insist on removing trailing whitespace, followed
by sites which actively test for the required SP after each colon).

Does anyone know of a current mail2news gateway which actually goes to the
trouble of testing for this particular case?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21702
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Wed, 29 Jun 2005 13:49:34 GMT
Lines: 39
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Except that the text you included says:

>    7. It MUST remove any Xref header (F-3.2.10) from each article.  It
>       then MAY (and usually will) generate a fresh Xref header.

>This is wrong for Xref slaving.  The serving agent does not remove the
>Xref header when Xref slaving is being used.

Right! I see your problem now. How about:

   7.   It MUST (exept when specially configured to preserve the
        <article-locator>s set by the sending site) remove any Xref
        header (F-3.2.11) from each article.  It then MAY (and usually
        will) generate a fresh Xref header.

>>> I think the lack of an Xref header would be surprising to clients, so I
>>> would lean towards SHOULD personally, but I haven't really thought
>>> about it at any great length.

>> Do you want us to say that is is a required part of the protocol that
>> serving agents MUST/SHOULD construct an Xref header?

>That would be my inclination, yeah.

OK, then I think you should ask Alexey to raise a ticket if you want to
pursue it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21699
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Wed, 29 Jun 2005 13:19:22 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon June 27 2005 12:59, Charles Lindsey wrote:
>> It is clearly the reponsibility of anyone adopting an identity not based
>> on DNS of IP addresses to choose something that is unique. Our drafts do
>> not, cannot and should not specify how that is ensured.

>Au contraire, requiring uniqueness but failing to indicate a suitable
>method by which uniqueness can be assured is a "known technical omission"
>(see BCP 9).

The presently proposed text leaves it to the site owner to choose a unique
name as best he can. If he screws up, then he screws up.

I will take advice from Harald as to whether that needs to be documented
as a "technical omission".

All I can say is that the problem does not seem to be a serious one on the
present Usenet. Moreover, I am still awaiting an answer to my question as
to whether there is more than one site called "wolf" on Usenet, or whether
that was just a duplicate in the old UUCP maps that merely caused problems
with misdirection of email.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21700
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1043 "header": Suggested resolution
Date: Wed, 29 Jun 2005 13:25:11 GMT
Lines: 31
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon June 27 2005 14:53, Charles Lindsey wrote:

>> since this WG has consistenly used the "header"
>> terminology since the earliest days

>Charles, please stop promulgating unsupported lies; text from
                                               ^^^^
					       ????
>our past drafts has been cited which uses standard terminology.
>Moreover, it has been stated (with supporting evidence) that
>such use was self-inconsistent.

One example has been cited, from USEFOR-00, which was the initial attempt
at the new format-only document, and had not been previously been reviewed
by the WG. It was corrected in USEFOR-01.

All our previous drafts had consistently used the "header", rather than
the "header field", notation, as had Son-of-1036 before that.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21701
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047: Path field delimiters and components
Date: Wed, 29 Jun 2005 13:35:16 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon June 27 2005 15:39, Charles Lindsey wrote:
>> 
>> On the contrary, I stand by every word that I said. It is highly
>> improbable that any currenly existing <path-identity> will run into any
>> problem when ":" ceases to be a path-delimiter.

>That will happen when every instance of currently conforming news software
>handling the Path field is replaced.

No, it will officially "cease" to be a path-delimiter as soon as our
standard is published. At that point of time, the probability of their
existing any then current <path-identity> that will cause problems when
":" starts to appear (in IPv6 addresses, for example) is exceedingly
small.

So small, in fact, that Russ has said that it will not affect the
soundness of his sleep.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21697
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Wed, 29 Jun 2005 12:56:55 GMT
Lines: 26
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon June 27 2005 15:26, Charles Lindsey wrote:

>> They clearly say nothing about the semantics of <id-right>.

>2822 clearly says "the domain name (or a domain literal IP address"; it
>does not say "some random string" or anything like it.  Nor does it say
>that the semantics have changed from RFC 822.

RFC 2822 invites you to "put" a "domain name (or a domain literal IP
address)" when generating an <id-right>. That does not cause it not to be
an <id-right> any more, nor does it confer any semantics upon that
<id-right> that it did not already possess by virtue of its being an
<id-right>.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21696
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Wed, 29 Jun 2005 12:51:23 GMT
Lines: 26
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon June 27 2005 15:01, Charles Lindsey wrote:

>> Simply changing the name of some syntax rule cannot possibly change the
>> meaning of the document.

>Ergo, by your own logic, "id-right" does not change the meaning of
>the RHS of a msg-id from that of a domain name or literal.  Nor
>does it change the fact that domain names are case-insensitive.

If you replace <id-right> by <domain> (which is already defined in RFC
2822), then obviously it makes a considerable difference. But replacing
it, systematically, by <mumble-foo>, or anything else, will have no effect
whatsoever.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 29 Jun 2005 18:03:25 +0200
From: [email-masked] (Ralph Babel)
To: [email-masked]
Subject: Re: #1022 USEFOR 2.1 obs-phrase: Resolution proposal
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> News articles MUST conform to the syntax specified
> in Section 3 of [RFC2822]. Netnews agents

"Netnews agent" isn't defined anywhere. Didn't
we at some point agree that Usefor was about
on-the-wire article _format_, not "agent" behavior?

> MAY also accept the obsolete syntax specified in
> Section 4 of [RFC2822], and MUST accept the "UTC"
> timezone (see section 3.1.2)

What "UTC" timezone? "UT" maybe? And didn't we
just get rid of that one (for whatever reason)?

Date: Wed, 29 Jun 2005 17:58:53 +0200
From: [email-masked] (Ralph Babel)
To: [email-masked]
Subject: Re: #1050: USEFOR 3.2.11 Xref vs. transport
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> Ticket #1050 created. Based on the following discussion,
> I have put it into state "no change needed".

NACK, Bruce has a point here: while "Xref" is optional,
so is "Organization", but the former may well be removed
completely upon transfer of an article to the next hop,
while the latter may not. Therefore, something to
the effect of "Xref MAY be removed from in-transit
articles" needs to be added to section 3.2.11.

Date: Wed, 29 Jun 2005 16:52:28 +0100
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Sender: [email-masked]

In message <[email-masked]>, Forrest J. Cavalier III
<[email-masked]> writes

>> I think Forest is talking about the $alz convention
>> (http://www.killfile.org/faqs/cancel.html section IIB). I think
>> you're note is still OK with this as you don't parse the message-id
>> as much as add cancel. to what was already there. 
>
>Sorry, I thought that there was more widespread understanding
>among the WG that some packages have a "refuse cybercancels" setting that
>causes a relay agent to not accept messages with IDs which start
>"<cancel"

Although I can imagine that there exists software that does practically
anything -- I believe that the intent of the conventions is that the
$alz scheme (which means that one is able to predict the cancel message
ID) is to ensure that only one cancel floods out, no matter how many
robots generate it (in the extended scheme with a secret, the cabal of
cancellers share the secret amongst themselves -- same difference!)

Servers that wish to ignore cancels (or wish to distinguish amongst the
various parts) are clearly _intended_ to process the last few elements
of the Path: header field data, looking for the "pseudosite" strings
that they are prepared to accept (or not)  [since they're not in the
UUCP message maps they're some of the most used non-DNS based strings
... cyberspam, mmfcancel, bincancel, spewcancel and doubtless others
I've forgotten]

viz: since we're not producing a taxonomy of strange behaviour, I don't
see a need to document the exciting things that people have dreamt up on
their own to (not) process cancels ...

>I thought, but could be mistaken, that there were some multi-part
>naming schemes which used formatted message-IDs, but I am less
>sure that anything acts on that.

Since there are better ways of tying multiparts together, I don't think
that leaving a hole for such a scheme is desirable or necessary

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

Date: Wed, 29 Jun 2005 11:52:45 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: #1047 Path field delimiters - feed behaviour? (fwd)
Sender: [email-masked]

Henry Spencer <[email-masked]> wrote:
> On Wed, 29 Jun 2005, Harald Tveit Alvestrand wrote:

>> Conversely, if a message arrives with a Path: header that says
>> Path: there!::FFFF:129.144.52.38!::FFFF:129.145.23.23!not-for-mail
>> will the two occurences of "FFFF" in the Path: cause anything bad to happen?
>
> Not in the implementations I'm familiar with, and probably not in any
> implementation.

Almost certainly not.  One April 1, admins from The Hierarchy That
Shall Not Be Named all decided to add one bogus entry to their Path
entries to force it to first in the propagation statistics.  It
typically showed up several times in every article, with no observable
effects.

Seth

Date: Wed, 29 Jun 2005 11:43:47 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Sender: [email-masked]

Frank Ellermann <[email-masked]> wrote:

>> Trying to revoke RFC 2822's prohibition against generating
>> them is, to my mind, not something we should be doing in
>> this WG.
>
> Just don't talk about it, combining MUST accept with MUST NOT
> generate is odd, I prefer "SHOULD NOT generate" in such cases.

I agree it's odd.  I still think we should do it.  "MUST accept"
because there's a lot of software that generates it and will continue
to, so failure on that point affects interoperability.  "MUST NOT
generate" because there's no reason to allow generation (there are
better equivalents) and allowing more than FRC 2822 (it says "MUST NOT
generate") is something we ought to do only with very good reason;
oddness of wording isn't such a reason.

Seth

Date: Wed, 29 Jun 2005 11:15:50 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: Definition of "private" (Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts)
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> wrote:

> I think the issue of all-numeric, mixed-case and "all" and "ctl" groups is 
> that many people *can't* choose to carry them - their implementations will 
> behave in unexpected ways if they try.
>
> So it makes sense to not require everyone to be able to handle them in 
> order to conform...

It has _always_ been the case that the choice of which groups to carry
is a local option, made on whatever basis (technical, financial,
political, religious, . . .) the local admin decides.  Even if the
syntax _permits_ such an article, all that means is that the server
shouldn't fall over when it sees it; it's allowed to drop it on the
floor for any or no reason.

So I don't see how any wording we put here _requires_ anybody to be
able to handle anything (except in the sense of "don't fall over if
you see it"--and I assure you, they WILL see it no matter what we
write).

Seth

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1047 Path field delimiters - feed behaviour? (fwd)
Date: Wed, 29 Jun 2005 11:13:33 -0400
Cc: Harald Tveit Alvestrand <[email-masked]>
Sender: [email-masked]

On Wed June 29 2005 07:56, Harald Tveit Alvestrand wrote:
> 
> I haven't seen an answer to this question - would anyone else like to 
> answer?
[...]
> --On 23. juni 2005 17:05 -0700 John Stanley <[email-masked]> wrote:
> 
> > It would seem to be unwise to use it in IPV6 addresses as well, since
> > ::FFFF:129.144.52.38 and ::FFFF:129.145.23.23 are clearly two different
> > addresses but the same failure that would make the dead:beef admin an
> > idiot would make both of these people idiots. (I.e., non-conforming
> > servers would match the "path-identity" of FFFF for both systems and
> > feeds  would be disrupted.)
> 
> If, in a message arriving at the old system "here", there is a Path: header
> that says
> 
> Path: there!::FFFF:129.144.52.38!not-for-mail
> 
> and "here" does not know about any system named "FFFF", will any change in
> handling happen?

Change relative to what?  Does "here" "know" about a system "named"
"129.144.52.38?   The root of the issue is that colon always has been
and remains today a delimiter.

> Conversely, if a message arrives with a Path: header that says
> 
> Path: there!::FFFF:129.144.52.38!::FFFF:129.145.23.23!not-for-mail
> 
> will the two occurences of "FFFF" in the Path: cause anything bad to happen?

Not that I know of.  Some statistics gathering scripts might interpret
that as a loop of some sort.  No problem for propagation. (assuming of
course that the conditions for the previous question still apply)
 
> Or is there some third circumstance that I haven't guessed at yet that will
> cause something bad to happen?

Yes,

Path: there!FEDC:BA98:7654:3210:FEDC:BA98:7654:3210!foo

(example IPv6 address literal from RFC 3513 section 2)  Will cause the article
not to be propagated to any sites "known" to "here" "named" "FEDC", "BA98",
"7654", and/or "3210" (a less repetitive IPv6 example would of course provide
more opportunity for damage).

The RFC 3513 example and description of the problem has been previously
described in http://www.imc.org/ietf-usefor/2004/Jul/0192.html
which was part of a discussion also referenced in recent discussion.

An example of a highly problematic IPv6 literal incorporating observations
made by Frank and Russ would be DEAD:BEEF:FEE:BEE:CAFE:ABBE:ED:BE.

Date: Wed, 29 Jun 2005 11:08:45 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Sender: [email-masked]

Graham Drabble wrote:

> On 29 Jun 2005 Harald Tveit Alvestrand <[email-masked]> wrote
> in news:[email-masked]: 
> 
> 
>>--On onsdag, juni 29, 2005 08:33:34 -0400 "Forrest J. Cavalier
>>III" <[email-masked]> wrote:
>>
>>
>>>Harald Tveit Alvestrand wrote:
>>>
>>>>     In no circumstance is it reasonable for software receiving
>>>>     a <msg-id> to change behaviour based on a theory about how
>>>>     the <msg-id> was generated.
>>>
>>>
>>>What about "<cancel.", and other schemes (maybe multiparts.)?
>>>
>>
>>Sorry, I don't understand. Is there software that tries to *parse*
>>a  message-ID?
> 
> 
> I think Forest is talking about the $alz convention
> (http://www.killfile.org/faqs/cancel.html section IIB). I think
> you're note is still OK with this as you don't parse the message-id
> as much as add cancel. to what was already there. 
> 

Sorry, I thought that there was more widespread understanding
among the WG that some packages have a "refuse cybercancels" setting that
causes a relay agent to not accept messages with IDs which start
"<cancel"

I thought, but could be mistaken, that there were some multi-part
naming schemes which used formatted message-IDs, but I am less
sure that anything acts on that.

To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
From: Graham Drabble <[email-masked]>
Date: Wed, 29 Jun 2005 15:23:43 +0100
Organization: Home
Sender: [email-masked]

On 29 Jun 2005 Harald Tveit Alvestrand <[email-masked]> wrote
in news:[email-masked]: 

> --On onsdag, juni 29, 2005 08:33:34 -0400 "Forrest J. Cavalier
> III" <[email-masked]> wrote:
> 
>>
>> Harald Tveit Alvestrand wrote:
>>>      In no circumstance is it reasonable for software receiving
>>>      a <msg-id> to change behaviour based on a theory about how
>>>      the <msg-id> was generated.
>>
>>
>> What about "<cancel.", and other schemes (maybe multiparts.)?
>>
> 
> Sorry, I don't understand. Is there software that tries to *parse*
> a  message-ID?

I think Forest is talking about the $alz convention
(http://www.killfile.org/faqs/cancel.html section IIB). I think
you're note is still OK with this as you don't parse the message-id
as much as add cancel. to what was already there. 

-- 
Graham Drabble
http://www.drabble.me.uk/

Date: Wed, 29 Jun 2005 10:11:15 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Sender: [email-masked]

On Wed, 29 Jun 2005, Harald Tveit Alvestrand wrote:
> OK, here's the proposed text for the problems I outlined yesterday...

Seems about right.

>    News is more dependent on msg-id uniqueness and fast comparision than

Should be "comparison".

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1047 Path-identity: A proposal (perhaps)
Date:  Wed, 29 Jun 2005 15:52:55 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 12
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
 
> path-address = IPv4address / no-fold-literal ; see [RFC2373]

s/2373/3986/

> recycling no-fold-literal, I've added [] around addresses...

Doesn't change the dead:beef:cafe:: problems, [ and ] are also
punctuation in 1036.
                        Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Date:  Wed, 29 Jun 2005 15:34:01 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 11
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> Is there software that tries to *parse* a message-ID?

It's a convention for spam cancel Message-IDs.  In that case
an intentional collision, one <cancel.old@id> is enough to
cancel the <old@id> spam.  In a more elaborated form it's
<cancel.secret.old@id>, because the bad guys started to send
a dummy <cancel.old@id> before the <old@id> spam, this dummy
blocked the spam cancel.  IIRC, bye, Frank

Date: Wed, 29 Jun 2005 09:30:38 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1047 Path field delimiters - feed behaviour? (fwd)
Sender: [email-masked]

On Wed, 29 Jun 2005, Harald Tveit Alvestrand wrote:
> If, in a message arriving at the old system "here", there is...
> Path: there!::FFFF:129.144.52.38!not-for-mail
> and "here" does not know about any system named "FFFF", will any change in
> handling happen?

Nope.  The Path header affects article propagation only to neighbors whose
names are mentioned in it.  If "FFFF" isn't known to be the name of a
neighbor, no effect. 

> Conversely, if a message arrives with a Path: header that says
> Path: there!::FFFF:129.144.52.38!::FFFF:129.145.23.23!not-for-mail
> will the two occurences of "FFFF" in the Path: cause anything bad to happen?

Not in the implementations I'm familiar with, and probably not in any
implementation.

                                                          Henry Spencer
                                                       [email-masked]

Date: Wed, 29 Jun 2005 09:26:08 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1047 Path-identity: A proposal (perhaps)
Sender: [email-masked]

On Wed, 29 Jun 2005, Harald Tveit Alvestrand wrote:
> If we explained it, would this be a reasonable text to insert in 3.1.6?

Looks reasonable to me...

                                                          Henry Spencer
                                                       [email-masked]

Date: Wed, 29 Jun 2005 14:00:26 +0100
To: Harald Tveit Alvestrand <[email-masked]>
Cc: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1047 Path-identity: A proposal (perhaps)
Sender: [email-masked]

In message <[email-masked]>,
Harald Tveit Alvestrand <[email-masked]> writes

> Traditional usage has had two means of generating unique identifiers:
> DNS names and asserted identities. The first gets its uniqueness from
> the DNS; the second gets its uniqueness from tradition. New servers
> should use DNS names.

Not terribly important, but since the majority of the traditional names
that occur in the "middle" of paths derive from UUCP (the ones at the
ends are more of a mixture, but still mainly UUCP) could we change the
second sentence to finish...

...  "gets its uniqueness from tradition, mainly from when articles were
carried over UUCP."

which at least gives a clue (something to Google on) as why the
tradition arose and why it is no longer sustainable.

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

Date: Wed, 29 Jun 2005 14:42:53 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: "Forrest J. Cavalier III" <[email-masked]>, [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Sender: [email-masked]

This chair has already marked ticket #1050 with "No change needed".

Please stop beating the dead horse.

--On onsdag, juni 29, 2005 08:37:51 -0400 "Forrest J. Cavalier III" 
<[email-masked]> wrote:

>
>
>> Do you want us to say that is is a required part of the protocol that
>> serving agents MUST/SHOULD construct an Xref header? I think we all agree
>> that the Xref header is a bit of a kludge, though it is a very well
>> established kludge.
>
> I am not sure we have basis for a MUST/SHOULD, but how is it a kludge
> to be able to mark cross-posts as read?
>
>
>

Date: Wed, 29 Jun 2005 14:40:52 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: "Forrest J. Cavalier III" <[email-masked]>, [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Sender: [email-masked]

--On onsdag, juni 29, 2005 08:33:34 -0400 "Forrest J. Cavalier III" 
<[email-masked]> wrote:

>
> Harald Tveit Alvestrand wrote:
>>      In no circumstance is it reasonable for software receiving a
>>      <msg-id> to change behaviour based on a theory about how the
>>      <msg-id> was generated.
>
>
> What about "<cancel.", and other schemes (maybe multiparts.)?
>

Sorry, I don't understand. Is there software that tries to *parse* a 
message-ID?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1021 USEFOR Newsgroups header - resolution proposed
Date:  Wed, 29 Jun 2005 13:42:08 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

>  Groups starting with the component "junk"
>  Groups starting with the component "to"
>  Groups containing the component "all"
>  Groups containing the component "ctl"
>  The group "control"
>  The group "junk"
>  The group "to"

How about adding...
   Groups starting with the component "control"
...for groups like control.cancel etc., and then combine the
three single groups in one line:

   Groups containing the component "all"
   Groups containing the component "ctl"
   Groups starting with the component "control"
   Groups starting with the component "junk"
   Groups starting with the component "to"
   The groups "control", "junk", and "to"

Anything else in this #1021 is fine from my POV, bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Date:  Wed, 29 Jun 2005 14:32:33 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 49
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> The main change is deleting the ABNF (unneccesary) and doing
> the import of unknown timezones in a NOTE rather than in
> normative text.

Okay.  The first statement in the note is dubious, I'd prefer
s/any/other/ resulting in:

  this specification does not change [RFC2822], which says
  that agents MUST NOT generate <date-time> constructs which
  include other zone names defined by <obs-zone>.

You already said that GMT is deprecated and MUST be accepted.
MUST NOT generate and MUST accept is a general 2822 obs-rule.
MUST NOT generate and MAY  accept is the usefor-0x  obs-rule.

If you change the latter to 2822-style for GMT it's better if
you only say "deprecated",  Besides "other zone names" can be
read as "unknown zone names", then the next statement below
your 'Note' is clearer:

> Software that accepts dates with unknown timezones SHOULD
> treat such timezones as equivalent to "-0000"
[...]

We don't want "-0000" for GMT, after s/any/other/ the "+0000"
for GMT is more obvious.

About #1022, you proposed:

| News articles MUST conform to the syntax specified in
| Section 3 of [RFC2822].  Netnews agents MAY also accept the
| obsolete syntax specified in Section 4 of [RFC2822], and
| MUST accept the "UTC" timezone (see section 3.1.2) and the
| obs-phrase construct (see [RFC2822] section 4.1) but they
| MUST NOT generate productions of such syntax.

s/"UTC"/deprecated "GMT"/ and delete "but they MUST NOT" etc.

> Trying to revoke RFC 2822's prohibition against generating
> them is, to my mind, not something we should be doing in
> this WG.

Just don't talk about it, combining MUST accept with MUST NOT
generate is odd, I prefer "SHOULD NOT generate" in such cases.

                          Bye, Frank

Date: Wed, 29 Jun 2005 08:37:51 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Sender: [email-masked]

> Do you want us to say that is is a required part of the protocol that
> serving agents MUST/SHOULD construct an Xref header? I think we all agree
> that the Xref header is a bit of a kludge, though it is a very well
> established kludge. 

I am not sure we have basis for a MUST/SHOULD, but how is it a kludge
to be able to mark cross-posts as read?

Date: Wed, 29 Jun 2005 08:33:34 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
>      In no circumstance is it reasonable for software receiving a <msg-id>
>      to change behaviour based on a theory about how the <msg-id> was
>      generated.

What about "<cancel.", and other schemes (maybe multiparts.)?

Date: Wed, 29 Jun 2005 14:33:30 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1047 Path-identity: A proposal (perhaps)
Sender: [email-masked]

At the moment, it seems that we're at an impasse partially caused by our 
documents' failure to explain what a "path-identity" is. I don't know if I 
understand it either....

At the moment, the specific format and use of path-identity is punted to 
USEPRO, where it seems to be still undefined.

If we explained it, would this be a reasonable text to insert in 3.1.6?
(I've modified the BNF too. Only modified stuff included here)
--------------------------------------------------------------------
  A path-identity identifies an agent. For the news system to work
  properly, the path-identity must be globally unique.
  Traditional usage has had two means of generating unique identifiers:
  DNS names and asserted identities. The first gets its uniqueness from
  the DNS; the second gets its uniqueness from tradition. New servers
  should use DNS names.

  In some cases, one wishes to embed something into a path-entity that
  is not a name. One common thing to embed is an IP address, serving
  as an identity when the name of the agent is not known.

   path-list       =  [FWS]
                      *( ( path-identity / path-keyword / path-address )
                      [FWS]
                      path-delimiter [FWS] )
                      tail-entry [FWS]

   path-identity   =  ( ALPHA / DIGIT )
                      *( ALPHA / DIGIT / "-" / "." / "_" )

   path-keyword    = "POSTED" / "MISMATCH"

   path-address    =  IPv4address / no-fold-literal ; see [RFC2373]

  NOTE: The path-address contains characters that some systems consider
     path-delimiters. This can cause problems; path-address should
     therefore only be used when absolutely necessary.
     See [USEPRO] for details on when to use them.
--------------------------------------------------------------------

Note that in recycling no-fold-literal, I've added [] around addresses... 
and that a naked IPv4 address is allowed separately, because people have 
said this is already present (too much for us to remove).

Does this make sense?

                      Harald

Date: Wed, 29 Jun 2005 08:29:45 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Sender: [email-masked]

Russ Allbery wrote:

>     7. It MUST remove any Xref header (F-3.2.10) from each article.  It
>        then MAY (and usually will) generate a fresh Xref header.
> 
> This is wrong for Xref slaving.  The serving agent does not remove the
> Xref header when Xref slaving is being used.

Well, is that a serving agent or a relay/transport agent?    What
if someone does a SUCK feed and Xref slaving.  Does that make the server
a relay agent?  Does a SUCK feed count as a feed in USEPRO?

As you know, I don't like all the artificial distinctions between
serving/relay/injection agents.

If anything is said, I think it should be a minor note under security
in USEAGE.  I don't think use of RFC2119 language is justified at all,
because there is no interoperability at stake, and the amount and importance
of additional information leaked across a feed in Xref is really, really small.

Downstream sites can already determine how much delay you have
internally, what feeds you get, who you get them from, how complete your
spool is.  So Xref only lets them discover approximately how old your
installation is, modulo your filters, and what newsgroups you carry.
Big deal.  They can look in Whois and look at when you filed articles of
incorporation to get a more accurate determination of when you established
They can look in archives and see when articles with your site name in Path
started appearing.

And you can probably ask them what groups they have.  They will tell you.

Date: Wed, 29 Jun 2005 08:18:37 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

Charles Lindsey wrote:
> 
> I pointed out that in the real world implementors and other readers DO
> take shortcute when reading documents. That is a fact of life. You have to
> live with it.

No I do not have to live with it.  As has been said so many times: we
need a clear, complete, unambiguous specification that is as easy to read
as possible, BUT WE HAVE NO CONCERN for people who refuse to read and
understand NORMATIVE references.  We can totally ignore such people when
writing, because they are irrelevant.

A specification is never read by unthinking sheep who have to be
bullied into compliance.  And frankly, if you write that way you
insult the people who actually do read it.

Until you understand this subtle point, this is going to keep coming up
as long as you are editor.

> 
> Therefore, people writing specifications should take care to look at what
> they have written through the eyes of those likely to read them, and ensure
> they are written in a manner sympathetic to those readers.

Right.  The disagreement and the chair's rule was in limiting the set
of who "is likely to read them."

Now we know that EVERY READER will be expected to understand RFC2822, and we are
sympathetic to them.  NO ONE will not have read RFC2822, so we do not have to
be sympathetic to people who are from "news only and do not expect the RFC2822
terminology.   Accomodating people who will not read the document is
unecessary and of no concern.  Doing so cannot justify any wording or text
of the drafts.

> 
> We are not trying to write a specification which is a challenge for
> everyone who comes to read it, so that we can say, with great glee when
> they misinterpret what we have written, "you are a moron because of your
> inability to decipher what (with superhuman attention to every detail) was
> plainly stated in the document.
> 

Excluded Middle, hostile, and insulting.  Can't you please make a dignified
argument instead?

Date: Wed, 29 Jun 2005 08:04:35 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1021 USEFOR Newsgroups header - resolution proposed
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> 
>  A newsgroup component SHOULD NOT consist of digits only, and
>  SHOULD NOT contain uppercase letters. Such components MAY be
>  used only to refer to existing groups that do not conform to this
>  naming scheme.

[snip]

> 
>  The following newsgroup names have been used for specific purposes in
> various implementations and protocols, and MUST therefore not be used
> for the names of normal newsgroups. They MAY be used for their specific
> purpose, or by local agreement.
> 
>  Groups starting with the component "junk"
>  Groups starting with the component "to"
>  Groups containing the component "all"
>  Groups containing the component "ctl"
>  The group "control"
>  The group "junk"
>  The group "to"
> 
>  NOTE: Some newsgroups violating these restrictions exist. It is not a
> protocol violation to use these names to access those newsgroups.

I don't think that permission belongs in a NOTE.  The permission to
use alt.2600 is not in a note in the first paragraph.

Why did you use different phrasing and place it in a note?

Can you change the "MUST therefore not" to "therefore MUST NOT", or
better yet put the explanation in a NOTE, like you did for digits
part.  I think there is a big improvement with more consistency in
grouping and splitting explanation and specification into body text
and NOTE, and putting the MUSTs ahead of the SHOULDs.

Can I suggest the following arrangement of text...

  The syntax of newsgroup names and components (specified in section []) are
  limited as follows.

  The group "poster" and groups starting with the component "example"
  are reserved, and MUST NOT be used for the name of a newsgroup.

  Components beginning with underline ("_") are reserved for
  future IETF specifications and otherwise MUST NOT be used by
  posting agents. Such names MUST be accepted by relaying and
  serving agents.

  Components beginning with "+" and "-" are reserved for private
  use and MUST NOT be used without a private prior agreement. Such
  names MUST be accepted by relaying and serving agents.

  Unless referring to pre-existing groups or for their specific pre-established
  purpose in implementation and protocols, names SHOULD NOT be "control",
  SHOULD NOT contain any component "all" or "ctl", and SHOULD NOT consist
  or start with the component "junk" or "to".

  Unless referring to pre-existing groups, a newsgroup component
  SHOULD NOT consist of digits only, and SHOULD NOT contain uppercase
  letters.

   NOTE: All-digit components conflict with one widely used storage
     scheme for articles. Mixed case components cause confusion because some
     systems distinguish case while others ignore it.

Date: Wed, 29 Jun 2005 13:56:05 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1047 Path field delimiters - feed behaviour? (fwd)
Sender: [email-masked]

I haven't seen an answer to this question - would anyone else like to 
answer?

                   Harald

---------- Forwarded Message ----------
Date: fredag, juni 24, 2005 09:04:27 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: John Stanley <[email-masked]>, [email-masked]
Cc:
Subject: #1047 Path field delimiters - feed behaviour?

Attempting to return to the problem at hand, rather than the discussion on
how to discuss these things - John, can you educate me a little bit about
USENET feeds?

--On 23. juni 2005 17:05 -0700 John Stanley <[email-masked]> wrote:

> It would seem to be unwise to use it in IPV6 addresses as well, since
> ::FFFF:129.144.52.38 and ::FFFF:129.145.23.23 are clearly two different
> addresses but the same failure that would make the dead:beef admin an
> idiot would make both of these people idiots. (I.e., non-conforming
> servers would match the "path-identity" of FFFF for both systems and
> feeds  would be disrupted.)

If, in a message arriving at the old system "here", there is a Path: header
that says

Path: there!::FFFF:129.144.52.38!not-for-mail

and "here" does not know about any system named "FFFF", will any change in
handling happen?

Conversely, if a message arrives with a Path: header that says

Path: there!::FFFF:129.144.52.38!::FFFF:129.145.23.23!not-for-mail

will the two occurences of "FFFF" in the Path: cause anything bad to happen?

Or is there some third circumstance that I haven't guessed at yet that will
cause something bad to happen?

I'd like to understand what you mean when you say "feeds will be
disrupted".....

                   Harald

---------- End Forwarded Message ----------

Date: Wed, 29 Jun 2005 13:47:13 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1043 USEFOR 2.1 term "header": Resolved
Sender: [email-masked]

With a few refinements based on the discussion, this item is now closed.
ROUGH consensus.

OLD:

 This specification uses the terms "header", "header name", and
 "header content" which are synonymous with the [RFC2822] terms
 "header field", "field name", and "field body" respectively.

NEW:

  NOTE: Older News specifications used the term "header" as a synonym
     for what [RFC2822] calls "header field". This document follows
     the terminology of RFC 2822 in using the term "header field",
     based on a belief that consistent terminology among specifications
     that depend on each other makes the specifications easier to use
     in the long run.

Also add one more definition to this section:

  This specification also uses the following terms:

  Header line - a line as defined in [RFC2822] section 2.1 that forms
         part of a header field.

Date: Wed, 29 Jun 2005 13:38:47 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1042 USEFOR 3.1.5 Newsgroups folding - resolution proposal
Sender: [email-masked]

Suggested addition to section 3.1.5:

Folding the Newsgroups: header field over several header lines has been 
shown to harm propagation significantly. Folded Newsgroups header fields 
SHOULD NOT be generated, but MUST be accepted.

The only valid reason identified at this time for generating folded 
Newsgroups header fields is to test propagation of articles using that 
feature.

Date: Wed, 29 Jun 2005 07:26:49 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: Chair hat ON: Free-standing documents (Re: #1043 "header": Suggested resolution)
Sender: [email-masked]

Charles Lindsey wrote:
> In <B01B60966947165B92FAEA51@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:
> 
> 
>>As I've remarked in another thread: Sometimes you need to state the obvious.
> 
> 
> But many more times you also need to state the "not so obvious".
> 
> Which was my whole point.
> 

No.  You crossed the line by saying implementors could not be expected to
read and refer to a normative draft, despite section 3 refering
to RFC2822 so many times.

There will still be occasions to highlight and repeat text from
normative references.   But now there is no merit to any argument
that the draft can be written as if people can read USEFOR-USEFOR-04
and not understand RFC2822.

Are there any other substantative arguments against matching
RFC2822 terminology?  How long it takes to edit has been debunked.
How long text has been in the draft predating MESSFOR is irrelevant
now that they finished first.

Date: Wed, 29 Jun 2005 13:24:24 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT?  (fwd)
Sender: [email-masked]

                      Harald

---------- Forwarded Message ----------
Date: onsdag, juni 08, 2005 13:32:05 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Cc:
Subject: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT?

New ticket:

 in
 will
 we

---------- End Forwarded Message ----------

Date: Wed, 29 Jun 2005 13:21:38 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1028 USEFOR 3.1.2 Date: Resolved, I think.
Sender: [email-masked]

Based on comments to Ken Murchison's text, I currently have the following 
as resolution:

 The Date header is the same as that specified in Sections 3.3 and
 3.6.1 of [RFC2822] with the added restrictions detailed in
 Section 2.2. However, the use of "GMT" as a time zone (part of <obs-
 zone>), although deprecated, is widespread in news articles today.
 Therefore, agents MUST accept <date-time> constructs which use the
 "GMT" zone.

 NOTE: this specification does not change [RFC2822], which says
  that agents MUST NOT generate <date-time> constructs which
  include any zone names defined by <obs-zone>.

  Software that accepts dates with unknown timezones SHOULD treat such
  timezones as equivalent to "-0000" when comparing dates, as specified
  in [RFC2822] section 4.3.

The main change is deleting the ABNF (unneccesary) and doing the import of 
unknown timezones in a NOTE rather than in normative text.

(Since we're quoting requirements from 2822, I think it makes sense to put 
the uppercase MUST and SHOULD in a NOTE - usually, it doesn't).

Makes sense?

                     Harald

Date: Wed, 29 Jun 2005 13:09:23 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1022 USEFOR 2.1 obs-phrase: Resolution proposal
Sender: [email-masked]

I suggest the following text change to section 2.1 of USEFOR:

OLD:

   News articles MUST conform to the syntax specified in Section 3 of
   [RFC2822].  Netnews agents MAY also accept the obsolete syntax
   specified in Section 4 of [RFC2822], but they MUST NOT generate
   productions of such syntax.

NEW:

   News articles MUST conform to the syntax specified in Section 3 of
   [RFC2822].  Netnews agents MAY also accept the obsolete syntax
   specified in Section 4 of [RFC2822], and MUST accept the "UTC" timezone
   (see section 3.1.2) and the obs-phrase construct (see [RFC2822] section
   4.1) but they MUST NOT generate
   productions of such syntax.

Reason: Of the people who have spoken up, Bruce argues strongly for as much 
alignment with RFC 2822 as possible, and Charles argues for including the 
dot.

Russ argues that a realistic parser has to parse much more weird junk than 
this construct, so it doesn't make much of a difference - but I can't see a 
strong argument for keeping this one outside the "MUST accept" set.

Trying to revoke RFC 2822's prohibition against generating them is, to my 
mind, not something we should be doing in this WG.

Makes sense?

                       Harald

Date: Wed, 29 Jun 2005 12:37:49 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1021 USEFOR Newsgroups header - resolution proposed
Sender: [email-masked]

OK, here's my "proposed text" again, incorporating feedback.

---- restricting syntax; there are no changes to the ABNF -----

 A newsgroup component SHOULD NOT consist of digits only, and
 SHOULD NOT contain uppercase letters. Such components MAY be
 used only to refer to existing groups that do not conform to this
 naming scheme.

  NOTE: All-digit components conflict with one widely used storage
    scheme for articles. Mixed case groups cause confusion between
    systems with case sensitive matching and systems with case insensiteve
    matching of newsgroup names.

----- restricting use of specific characters ------------

 Components beginning with underline ("_") are reserved for use by
 future versions of this standard and MUST NOT be generated by
 posting agents (whether in Newsgroups headers or in newgroup control
 messages [USEPRO]). However, such names MUST be accepted by
 relaying and serving agents.

 Components beginning with "+" and "-" are reserved for private
 use and MUST NOT be generated by posting agents (whether in
 Newsgroups headers or in newgroup control
 messages [USEPRO]) without a private prior agreement to do so.
 However, such names MUST be accepted by
 relaying and serving agents.

---- restricting component names ----

 The following newsgroup names are reserved, and MUST NOT be used as
the name of a newsgroup:

 Groups starting with the component "example"
 The group "poster"

 The following newsgroup names have been used for specific purposes in
various implementations and protocols, and MUST therefore not be used
for the names of normal newsgroups. They MAY be used for their specific
purpose, or by local agreement.

 Groups starting with the component "junk"
 Groups starting with the component "to"
 Groups containing the component "all"
 Groups containing the component "ctl"
 The group "control"
 The group "junk"
 The group "to"

 NOTE: Some newsgroups violating these restrictions exist. It is not a
protocol violation to use these names to access those newsgroups.

Date: Wed, 29 Jun 2005 12:35:55 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Graham Drabble <[email-masked]>, [email-masked]
Subject: Re: Definition of "private" (Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts)
Sender: [email-masked]

--On mandag, juni 27, 2005 20:56:16 +0100 Graham Drabble 
<[email-masked]> wrote:

> On 23 Jun 2005 "Charles Lindsey" <[email-masked]> wrote in
> news:[email-masked]:
>
>> But, "general" apart, I think current practice on Usenet is that
>> single-component groups are not supposed to be there, and my
>> wording was intended to reflect that. But, as I said, my wording
>> is up for debate.
>
> The more I think about this the less I'm convinced that we need to say
> anything as part of the standard (we might want to look at this again
> come USEAGE). Single component groups and groups with all numeric
> components can only harm servers that choose to carry them, if they
> simply turn up in a transited article or in a control message which is
> not actioned they aren't a problem.

I think the issue of all-numeric, mixed-case and "all" and "ctl" groups is 
that many people *can't* choose to carry them - their implementations will 
behave in unexpected ways if they try.

So it makes sense to not require everyone to be able to handle them in 
order to conform...

                    Harald

Date: Wed, 29 Jun 2005 11:44:41 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1004 USEFOR 3.2.14 - closing X-headers
Sender: [email-masked]

I found no consensus supporting either reverting to the "deprecated" text 
(Charles' suggestion) or removing X-Mailer from the list (Alexey).

So I'm moving this ticket to "Text accepted".

Reposting the proposal:

OLD (3.2.13):

 NOTE: This header supersedes the role performed redundantly by
 experimental headers such as X-Newsreader, X-Mailer, X-Posting-
 Agent, X-Http-User-Agent, and other headers previously used on
 Usenet and in Email for this purpose. Use of these experimental
 headers SHOULD be discontinued in favor of the single, standard
 User-Agent header.

 NEW:

 NOTE: Some of this information has previously been sent in
 non-standardized headers such as X-Newsreader, X-Mailer,
X-Posting-
 Agent, X-Http-User-Agent, and others.
 Once an agent uses User-Agent, it should have no need to
 send these non-standard headers.

 OLD (3.2.14):

 This header is intended to replace various currently-used but
 undocumented headers such as "NNTP-Posting-Host", "X-Trace" and
 "X-Complaints-To". These headers are thus deprecated.

 NEW:

 NOTE: Some of this information has previously been sent in
 non-standardized headers such as NNTP-Posting-Host, X-trace,
 X-Complaints-To, and others.
 Once an injecting agent uses Injection-Info, it should have no
need to
 send these non-standard headers. 

Date: Wed, 29 Jun 2005 11:36:26 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1003 USEFOR 3.1.3 Message-ID: Proposed text
Sender: [email-masked]

OK, here's the proposed text for the problems I outlined yesterday:

OLD (start of section):

   The Message-ID header contains a single unique message identifier.
   This document updates the <msg-id> construct from Section 3.6.4 of
   [RFC2822] so as to ensure that Internet Message Format Message-IDs
   are usable in widely deployed news software.  The global uniqueness
   requirement for <msg-id> in [RFC2822] is to be understood as applying
   across all protocols using such message identifiers, and across both
   Email and Netnews in particular.  A revised syntax for <msg-id> is
   given below, but the requirements and descriptive text from Section
   3.6.4 of [RFC2822] still apply.

NEW:

   The Message-ID header contains a single unique message identifier.
   News is more dependent on msg-id uniqueness and fast comparision than
   email is, and some news software would have trouble with the full range
   of possible msg-ids permitted by RFC 2822; this section therefore
   restricts the syntax of <msg-id> compared to Section 3.6.4 of
   [RFC2822].
   The global uniqueness
   requirement for <msg-id> in [RFC2822] is to be understood as applying
   across all protocols using such message identifiers, and across both
   Email and Netnews in particular.

OLD (ABNF):

   no-fold-quote   =  DQUOTE
                         ( "." mqtext /
                           *mqtext "." /
                           *mqtext mqspecial *mqtext )
                         DQUOTE

NEW:

   no-fold-quote   =  DQUOTE
                         ( "." *mqtext /
                           *mqtext "." /
                           *mqtext mqspecial *mqtext )
                         DQUOTE

OLD (end of section):

      NOTE: It is RECOMMENDED in [RFC2822] that, for ensuring global
      uniqueness, the <id-right> be some domain identifier within whose
      scope the uniqueness of the <id-left> can be guaranteed.  When
      following this recommendation, any <dot-atom-text> or <no-fold-
      literal> used for the <id-right> are to be interpreted as
      <domain>s as described in Section 3.4.1 of [RFC2822].

NEW:

   Some software will try to match the <id-right> of a msg-id in
   a case-insensitive fashion; some will match it in a case sensitive
   fashion. Implementations MUST NOT generate two message-IDs where the
   only difference is the case of characters in the <id-right> part.

   If domain literals are used, the syntax found in [RFC2821] section
   4.1.3 is RECOMMENDED.

     NOTE: [RFC2822] section 3.6.4 recommends that the <id-right> should
     be a domain name or a domain literal. Domain literals are troublesome
     since many IP addresses are not globally unique; domain names are
     more likely to generate unique message-IDs.
     In no circumstance is it reasonable for software receiving a <msg-id>
     to change behaviour based on a theory about how the <msg-id> was
     generated.

I think this makes it clear that in message-id, the question of "what" the 
RHS is matters only to the generator, not to the parsing entity; all we 
care about after generation is that the message-ids are unique.

(the 2821 reference was added when I discovered that in 2822, this was 
punted to "the transport" - this makes sense for addressing, but not for 
message-IDs, where uniqueness is the thing that matters)

Makes sense?

                    Harald

Date: Wed, 29 Jun 2005 10:12:57 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked], Frank Ellermann <[email-masked]>
Subject: Re: NEW: draft language (Was Re: Feedback on proposal)
Sender: [email-masked]

Standard practice is to write the document the way you want the result to 
end up. That includes words like "this standard" (although, in many cases, 
"this document" is better for multiple reasons - for instance, because a 
standard may consist of more than one document).

I think it's also good practice to have a "Status of this memo (to be 
deleted by the RFC Editor)" section that explains that this is a draft that 
is intended for the standards track, and is currently a work item of the 
USEFOR WG, and that comments should be directed there. State the obvious...

--On 27. juni 2005 15:06 -0400 Bruce Lilly <[email-masked]> wrote:

>
> On Mon June 27 2005 13:12, Frank Ellermann wrote:
>>
>> Bruce Lilly wrote:
>> > "conformance with our" specification (it's not a standard).
>>
>> We're going for "proposed standard", not "informational" or
>> "experimental".  Maybe BCP for USEAGE.  Modulo "with the iESG
>> you never know".
>
> "Not all RFCs are standards".  More important "The Internet-Draft should
> neither state nor imply that it has any standards status; to do so
> conflicts with the role of the RFC Editor and the IESG." (1id-guidelines).
> I think I mentioned this before; the draft(s) need to be checked
> carefully by the Editors (this should already have been done in
> accordance with the guidelines...).
>
> Specifically, the draft contains about a dozen instances of "this
> standard" (one says "this present standard"), there's a reference to
> "other standards" (implying that the draft is a standard), a reference to
> "standard User-Agent", etc.   The only occurrence that shouldn't be
> changed is the one in the IPR boilerplate.
>
>
>

Date: Wed, 29 Jun 2005 10:22:23 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Graham Drabble <[email-masked]>, [email-masked]
Subject: #1021 Single component newsgroups (Re: Definition of "private" (Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts)
Sender: [email-masked]

--On 27. juni 2005 20:56 +0100 Graham Drabble <[email-masked]> 
wrote:

> n 23 Jun 2005 "Charles Lindsey" <[email-masked]> wrote in
> news:[email-masked]:
>
>> But, "general" apart, I think current practice on Usenet is that
>> single-component groups are not supposed to be there, and my
>> wording was intended to reflect that. But, as I said, my wording
>> is up for debate.
>
> The more I think about this the less I'm convinced that we need to say
> anything as part of the standard (we might want to look at this again
> come USEAGE). Single component groups and groups with all numeric
> components can only harm servers that choose to carry them, if they
> simply turn up in a transited article or in a control message which is
> not actioned they aren't a problem. It can therefore be left up to each
> administrator to decide based on local policy whether to add these
> groups.

I think that's the consensus of the group at the moment:

It's not sensible for the article *format* standard to say anything about 
single component newsgroup names. They should be permitted in syntax.

We'll return to the topic with USEAGE (if we get that far).

                Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date:  Wed, 29 Jun 2005 05:59:15 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 34
Sender: [email-masked]

Charles Lindsey wrote:
 
>>> I'm for id-right.
 
>> And that doesn't reflect the fundamental concept <host-name>.
 
> I think that is exactly why Kai said he was "for id-right" :-)

Possible, then I missed this "option" / "idea" (?) in RfC 724.
Where is it ?

>> <id-right> could be "maybe take some random number + hash +
>> timestamp, don't forget to add an '@' somewhere in the middle"

> Yes, that is a perfectly reasonable implementation if you use
> a good random number generator and make it long enough that
> the clashes are unlikely to occur before the heat death of
> the universe.

Clashes should be restricted to your own <id-domain>.  We have
already seen one case of <number@number> where nobody knew how
it was generated.  Elsewhere I saw <number@oemcomputer>.  That
is certified madness.  Doing this in mail might work, but the
poor user is an IETF WG Chair, he sends mails to mailing lists,
and for lists it's essentially the same problem as in news.

> It does conflict with a RECOMMENDATION in RFC 2822, but that
> is no absolute bar.

<id-domain> instead of <id-right> is also no absolute bar, but
at least the semantics is then obvious.  Even for implementors.

                          Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1048 Injection-Date syntax
Date:  Wed, 29 Jun 2005 05:24:13 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 57
Sender: [email-masked]

Charles Lindsey wrote:

>> Add me, but without s/SHOULD NOT/MUST NOT/ please,
>> GMT is neither critical nor harmful.

> But our requirements would then no longer be a subset
> of RFC 2822's requirements.

Yes, that's exactly the idea of explicity mentioning GMT with
a SHOULD NOT, because Russ found it in way too many articles
for a decree to forbid it.

> As Bruce has pointed out, we should not be permitting
> things that RFC 2822 forbids.

First Bruce proposed his new concept of a "SHOULD" as a "mere
recommendation", now you add a "SHOULD NOT" as a "permission".
Why don't you take 2119 as it is ?

| SHOULD NOT
| This phrase, or the phrase "NOT RECOMMENDED" mean that there
| may exist valid reasons in particular circumstances when the
| particular behavior is acceptable or even useful, but the
| full implications should be understood and the case carefully
| weighed before implementing any behavior described with this
| label.

"Valid resons in particular circumstances" include "old UA was
implemented before some mad IETFers invented new fancy rules"

| MUST NOT
| This phrase, or the phrase "SHALL NOT", mean that the
| definition is an absolute prohibition of the specification.
...................^^^^^^^^^^^^^^^^^^^^
MUST NOT means crashes, law suits, net abuse, harm, shoot on
sight, verboten.

 [2119 keywords]
| they MUST only be used where it is actually required for
| interoperation or to limit behavior which has potential for
| causing harm (e.g., limiting retransmisssions)  For example,
| they must not be used to try to impose a particular method
| on implementors where the method is not required for
| interoperability.

GMT _cannot_ cause harm, and that's the point of explicitly
mentioning it as a SHOULD NOT.  With the same MUST NOT as in
2822 that point would be moot.

> sites that still generate GMT will be non-compliant, and it
> will take a while for the usage to fade away. There are a
> lot of things in our drafts that are like that.

All these things, if they are very much like GMT, have to be
checked, MUST and MUST NOT aren't suited for wishful thinking,
they are about real trouble and harmful misbehaviour.  Bye

To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 28 Jun 2005 19:46:27 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> Please allow for Xref slaving.  (I don't know if the context takes care
>> of that.)

> I think it is OK. If you have a farm of serving agents that keep their
> Xrefs in step, do you regard passing articles between those agents as
> "relaying"

Yes.

> IOW, is that whole farm just "one big serving agent" from the POV of our
> draft?

No.  There are reasons why you might want to Xref slave to a news feed
that isn't at your site or even run by you.  They're rare, but there's no
particular reason to disallow it.  (Consider, for example, the case where
you run a personal local news server as basically a cache for your ISP's
remote news server and want to be able to switch between servers whenever
you feel like it without renumbering.)

> If the answer is "relaying", then the present "relayers MAY delete"
> suffices.

Except that the text you included says:

    7. It MUST remove any Xref header (F-3.2.10) from each article.  It
       then MAY (and usually will) generate a fresh Xref header.

This is wrong for Xref slaving.  The serving agent does not remove the
Xref header when Xref slaving is being used.

>> I think the lack of an Xref header would be surprising to clients, so I
>> would lean towards SHOULD personally, but I haven't really thought
>> about it at any great length.

> Do you want us to say that is is a required part of the protocol that
> serving agents MUST/SHOULD construct an Xref header?

That would be my inclination, yeah.

> I think we all agree that the Xref header is a bit of a kludge, though
> it is a very well established kludge.

Eh.  I could argue that both ways.  It's a kludge from a particularly
purist standpoint of data separation, but it's not clear to me that that
degree of purity is actually useful or desireable in practice.  On the
gripping hand, it's definitely a layering violation.

Anyway, whatever we might think of it, it's not going anywhere.

To: [email-masked]
Xref: clerew local.usefor:21676
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1043 "header": Suggested resolution
Date: Tue, 28 Jun 2005 17:17:19 GMT
Lines: 56
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>Charles Lindsey wrote:
>> In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:
>> 
>> 
>>>2822 is a NORMATIVE reference.  We refer to and require people to
>>>use it.  The ENTIRE audience will need to refer and understand
>>>RFC2822.
>> 
>> 
>> But I suspect that the entire audience is not going to refer to RFC 2822
>> (because they will just use what they "remember from last time they read
>> it"). And I doubt they will fully understand it, much less the
>> ramifications of all the other MIME etc documents that we refer to
>> normatively.
>> 
>> That is a sad fact of life. If you write a document that is not free
>> standing, and/or does not contain sufficient explanatory material to
>> assist the "less than omniscient reader", then people are going to cut
>> corners when they come to read it.

>This is a preposterous position for a WG participant, and absolutely
>unacceptable position for a WG editor.

>Can I ask the chairs to issue clear statements that we are NOT
>writing "free-standing" documents, and that it will be impossible to
>create a conforming implementation without reading RFC2822?  In evidence
>that this must be so, I offer the wording in Section 3 of USEFOR-USEFOR-04.

Eh? WHen did I say that we should be writing a "free standing" document?

I pointed out that in the real world implementors and other readers DO
take shortcute when reading documents. That is a fact of life. You have to
live with it.

Therefore, people writing specifications should take care to look at what
they have written through the eyes of those likely to read them, and ensure
they are written in a manner sympathetic to those readers.

We are not trying to write a specification which is a challenge for
everyone who comes to read it, so that we can say, with great glee when
they misinterpret what we have written, "you are a moron because of your
inability to decipher what (with superhuman attention to every detail) was
plainly stated in the document.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21680
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Definition of "private" (Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts)
Date: Tue, 28 Jun 2005 18:26:21 GMT
Lines: 37
Sender: [email-masked]

In <[email-masked]> Graham Drabble <[email-masked]> writes:

>On 23 Jun 2005 "Charles Lindsey" <[email-masked]> wrote in
>news:[email-masked]: 

>> But, "general" apart, I think current practice on Usenet is that
>> single-component groups are not supposed to be there, and my
>> wording was intended to reflect that. But, as I said, my wording
>> is up for debate. 

>The more I think about this the less I'm convinced that we need to say 
>anything as part of the standard (we might want to look at this again 
>come USEAGE). Single component groups and groups with all numeric 
>components can only harm servers that choose to carry them, if they 
>simply turn up in a transited article or in a control message which is 
>not actioned they aren't a problem. It can therefore be left up to each 
>administrator to decide based on local policy whether to add these 
>groups.

What I said, in my suggested text, was that single-component groups, and a
few other things, SHOULD NOT be found on the global Usenet, though other
networks could do as they please "by mutual arrangement".

But I also said that "SHOULD" was perhaps too strong, and invited
discussion. I still think that we should say _something_, though saying it
all in USEAGE is certainly one possibility.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21679
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Tue, 28 Jun 2005 18:22:17 GMT
Lines: 49
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> Under Duties of a Serving Agent:

>>    7. It MUST remove any Xref header (F-3.2.10) from each article.  It
>>       then MAY (and usually will) generate a fresh Xref header.

>Please allow for Xref slaving.  (I don't know if the context takes care of
>that.)

I think it is OK. If you have a farm of serving agents that keep their
Xrefs in step, do you regard passing articles between those agents as
"relaying" or as some "private protocol" for their mutual cponvenience?
IOW, is that whole farm just "one big serving agent" from the POV of our
draft?

If the answer is "relaying", then the present "relayers MAY delete"
suffices. If it is "one big serving agent", then the question does not
arise. So I think the present USEPRO wording is OK.

If you want some explanatory wording, or a NOTE, then it could be done,
but I don't think it is necessary.

>> I see little merit in upgrading that MAY to a SHOULD (no
>> interoperability issue or harm arises). It is a good practice, and to be
>> commended as such, which is why that MAY is there.

>I think the lack of an Xref header would be surprising to clients, so I
>would lean towards SHOULD personally, but I haven't really thought about
>it at any great length.

Do you want us to say that is is a required part of the protocol that
serving agents MUST/SHOULD construct an Xref header? I think we all agree
that the Xref header is a bit of a kludge, though it is a very well
established kludge. Changing the wording would make it harder to introduce
innovative solutions as to how a server communicates with its clients.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21678
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: draft language (Was Re: Feedback on proposal)
Date: Tue, 28 Jun 2005 18:08:27 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>"Not all RFCs are standards".  More important "The Internet-Draft should
>neither state nor imply that it has any standards status; to do so
>conflicts with the role of the RFC Editor and the IESG." (1id-guidelines).
>I think I mentioned this before; the draft(s) need to be checked
>carefully by the Editors (this should already have been done in
>accordance with the guidelines...).

>Specifically, the draft contains about a dozen instances of "this standard"
>(one says "this present standard"), there's a reference to "other standards"
>(implying that the draft is a standard), a reference to "standard
>User-Agent", etc.   The only occurrence that shouldn't be changed is the
>one in the IPR boilerplate.

May I point out that the MESSFOR drafts that preceded RFC 2822 also used
the words "this standard" in the same manner. Also that out earlier
arictle-* drafts contained the words:

[The use of the words "this standard" within this document when
referring to itself does not imply that this draft yet has pretensions
to be a standard, but rather indicates what will become the case if and
when it is accepted as an RFC with the status of a proposed or draft
standard.]

Those words are still in USEPRO, and ought to be in USEFOR as well, for
the removal of all doubt.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21681
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Chair hat ON: Free-standing documents (Re: #1043 "header": Suggested resolution)
Date: Tue, 28 Jun 2005 18:28:36 GMT
Lines: 18
Sender: [email-masked]

In <B01B60966947165B92FAEA51@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>As I've remarked in another thread: Sometimes you need to state the obvious.

But many more times you also need to state the "not so obvious".

Which was my whole point.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21675
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Tue, 28 Jun 2005 17:10:06 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Kai Henningsen wrote:

>> I'm for id-right.

>And that doesn't reflect the fundamental concept <host-name>.

I think that is exactly why Kai said he was "for id-right" :-).

><id-right> could be "maybe take some random number + hash +
>timestamp, don't forget to add an '@' somewhere in the middle".

Yes, that is a perfectly reasonable implementation if you use a good
random number generator and make it long enough that the clashes are
unlikely to occur before the heat death of the universe.

It does conflict with a RECOMMENDATION in RFC 2822, but that is no
absolute bar.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21677
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1048 Injection-Date syntax
Date: Tue, 28 Jun 2005 18:03:47 GMT
Lines: 27
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> I still prefer GMT on its own

>Add me, but without s/SHOULD NOT/MUST NOT/ please,
>GMT is neither critical nor harmful.

But our requirements would then no longer be a subset of RFC 2822's
requirements. As Bruce has pointed out, we should not be permitting
things that RFC 2822 forbids.

Yes, sites that still generate GMT will be non-compliant, and it will take
a while for the usage to fade away. There are a lot of things in our
drafts that are like that.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 28 Jun 2005 22:33:23 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Decision procedures (Re: #1043 "header": Suggested resolution)
Sender: [email-masked]

--On 28. juni 2005 09:45 -0400 Bruce Lilly <[email-masked]> wrote:

>
> On Mon June 27 2005 14:53, Charles Lindsey wrote:
>
>> if head counting is required.
>
> May I remind the co-chairs that there is still the unresolved
> question as to the method that is to be used to determine
> consensus; is it in fact a head count (in which case those of us
> who have been using reasoned arguments can adopt the simpler
> method of making a non-sequitur comment), or is it to be based
> on reasoned arguments as was the case in the Resnick-Gierth era,
> or something else?

RFC 2418 section 3.3:

   The other case is where the discussion has been held entirely over
   the mailing list.  The determination of the level of consensus may be
   harder to do in this case since most people subscribed to mailing
   lists do not actively participate in discussions on the list. It is
   left to the discretion of the working group chair how to evaluate the
   level of consensus.

Alexey and I decide.

                          Harald 

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 msg-id issues - attempt to focus
Date:  Tue, 28 Jun 2005 22:10:35 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 18
Sender: [email-masked]

Frank Ellermann wrote:

> P.S.:  Just for fun I test it with this article.

Here's what I got back from GMaNe:

http://article.gmane.org/gmane.ietf.usenet.format:29206:raw
Message-ID: <[email-masked]>

And that's what I posted (truncated header):

Message-ID: <"\=1B[7;0m"@xyzzy.claranet.de>
Date: Tue, 28 Jun 2005 20:57:02 +0200
Content-Type: text/plain; charset=3Dus-ascii
Content-Transfer-Encoding: 7bit

NetNews one Bruce nil.  Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  NEW USEPRO 7.9 gateway (was: LTRU questions)
Date:  Tue, 28 Jun 2005 21:59:49 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 24
Sender: [email-masked]

In <[email-masked]>
Harald wrote...

| So far, no tickets have been opened on USEPRO.

...therefore I test this procedure now, what do we expect
from x2news gateways trying to inject articles with an 
"unfoldable" subject or similar header field body line ?

> should we just offer a way to force compatibility with
> our rules by adding a "dummy" word in the 1st line of
> the Subject ?

Read "1st line of a folded subject header field body" if
that's your preferred style.
 
> Subject: =?utf-8?Q?=EF=BF=BE?=

That's a dummy BOM if I got it right.  Ugly like hell but
better than silently rejecting "unfoldable" articles.  It
could be also (ab)used to enforce a non-empty subject. 

> what should the poor mail2news gateway do ?  Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 msg-id issues - attempt to focus
Date:  Tue, 28 Jun 2005 20:57:02 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 10
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> Is this a fair statement of the issues being discussed?

Yes.  Minor nit, "there exist valid 2822 message-iDs that
are not valid News message-ids" - that's AFAIK in theory,
in practice I've never seen such Message-IDs.  Bye, Frank

P.S.:  Just for fun I test it with this article.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1043 "header": Suggested resolution
Date:  Tue, 28 Jun 2005 20:47:01 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 13
Sender: [email-masked]

Bruce Lilly wrote:
 
> May I remind the co-chairs that there is still the unresolved
> question as to the method that is to be used to determine
> consensus; is it in fact a head count

It's a "judgement call".  You have repeatedly posted articles
explaining why one sound argument can be more important than
several "heads".  Now what's new about this, are you testing
Scott's "trouble" filter ?  Then you have to write "voting" ;-)

                            Bye, Frank

Date: Tue, 28 Jun 2005 09:38:21 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>Add "even if it confuses the rest of the world, and leads them to complain
>you are stuck in your ivory tower and would rather you had a document that
>was pedantically correct rather than one of use to the people working at
>the coal face".

Your argument is filled with so many silly assumptions, unsupported 
assertions and insulting implications that it is hard to determine where 
to start debunking it.

What will confuse the "rest of the world" is telling them to read RFC2822 
and then use terms that are different than RFC2822.

It is not an "ivory tower" to be consistent with the rest of the world.

Using the correct terminology will not make our standard any less usefull
to anyone.

People who implement news software are not "at the coal face". and they 
are not idiots who won't be able to understand the terms used in RFC2822.

And it still won't take an hour using any reasonable editing tool to make 
the changes.

>And I suspect that "in the long run" the terminology that the marketplace
>uses will eventually prevail, even for email.

I have no idea what you mean by "prevail" or by "the marketplace".  These
are meaningless terms. The slang that people use in everyday speech is
outside the scope of our discussion, and the people who write the next
standard are free to use whatever terminology they feel appropriate,
assuming they have some justification for changes from the existing
standards, which we do not.

>You could also argue that, since this WG has consistenly used the "header"
>terminology since the earliest days, that it should require an actual
>consensus to make the change before reversing that decision.

I find it amusing that you make this argument given the large number
of changes you've made to the drafts based on nothing more than your own
feelings about it. If "since the earliest days" is now insufficient cause
for correcting a mistake, then it was insufficient for each and every
one of the other corrections you have made over the years. Please be 
consistent; if you want to make this argument now, I expect to see the 
long-standing text regarding References headers replaced. No, not your 
"followups MUST/everything else MAY" version, but the "followups 
MUST/everything else MUST NOT" that used to exist.

Cc: [email-masked]
To: Frank Ellermann <[email-masked]>
Date: Mon, 27 Jun 2005 18:01:24 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Sender: [email-masked]

>  [IESG process failure, quoting me:]
> > after approving technically incompatible [...]

> Maybe in the very same telechat (minutes not yet available).

> But I wasn't talking about NNTP -27.  It was about abusing
> v=spf1 sender policies for the incompatible "PRA" causing
> bogus FAIL as well as bogus PASS results for received mail,
> "unprofessional and unethical".  NOT RECOMMENDED and SHOULD
> simultaneously.  Broken by design instead of engineering.

Yes, well, I have to say that out of all the stuff that's going on right now,
this is the one that has me the most worried.

				Ned

Date: Tue, 28 Jun 2005 17:00:04 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1003 USEFOR 3.1.3 msg-id issues - attempt to focus
Sender: [email-masked]

I see that the subject of message-id is still controversial.

It is, however, less obvious what needs resolving and how to resolve it.

I can see the following issues:

1 - Is News message-ID different from 2822 message-id?
  As far as I can see, consensus YES - there exist valid 2822 message-iDs 
that are not valid News message-ids, but not vice versa (News msgid is a 
strict subset of 2822 msg-id). And current text (with bug fixed) is a 
reasonable description of the subset.

2 - Is the RHS of the message-ID a domain, a domain|ipv4literal or a 
domain|ipv4literal|ipv6literal, or an "opaquestring"?

  As far as I can see, there is no clear-cut consensus.

3 - Is the RHS of the message-ID case sensitive or case insensitive?

  Here, I think consensus is clear, based on existing implementations' 
behaviour: Once generated, you cannot change them, and making 2 message-IDs 
with the same LHS and only different in case on the RHS is NOT a good idea.

[deep breath]
Is this a fair statement of the issues being discussed?

If so, I'll try to make suggestions for text changes.....

                        Harald

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Date: Tue, 28 Jun 2005 10:19:39 -0400
Cc: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

On Mon June 27 2005 14:53, Charles Lindsey wrote:

> since this WG has consistenly used the "header"
> terminology since the earliest days

Charles, please stop promulgating unsupported lies; text from
our past drafts has been cited which uses standard terminology.
Moreover, it has been stated (with supporting evidence) that
such use was self-inconsistent.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Tue, 28 Jun 2005 10:04:45 -0400
Sender: [email-masked]

On Mon June 27 2005 15:50, Charles Lindsey wrote:

> And USEPRO already says what you want.
> 
> Under Duties of a Relaying Agent:
> 
>    8. It MAY delete any Xref header that is present.

If you fail to understand not only that "SHOULD NOT" and "MAY" differ,
but that the meanings under BCP 14 differ markedly, you shouldn't be
editing I-Ds intended for the Standards Track.

If you do in fact understand the implications, then please stop wasting
everybody's time by promulgating lies; "MAY" != "SHOULD NOT".  Such lies
are not conducive to rational discussion and are frankly insulting.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1047: Path field delimiters and components
Date: Tue, 28 Jun 2005 09:59:47 -0400
Sender: [email-masked]

On Mon June 27 2005 15:39, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >On Fri June 24 2005 12:44, Charles Lindsey wrote:
> 
> >> Yes, there is a theoretical possibility of a backward incompatibility
> 
> >There is nothing "theoretical" about it; you know this because it has
> >been discussed at length, with supporting documentation -- we know for
> >a fact that B news, C news, and INN treat colon as a delimiter, and
> >that reassignment will therefore unquestionably break backward
> >compatibility.  It is disingenuous of you, and frankly insulting to
> >others, for you to categorize this as "theoretical".
> 
> On the contrary, I stand by every word that I said. It is highly
> improbable that any currenly existing <path-identity> will run into any
> problem when ":" ceases to be a path-delimiter.

That will happen when every instance of currently conforming news software
handling the Path field is replaced.  We know from past experience that
that will take a very long time.  Indeed, if anything can be said to be
"theoretical", it is that hypothetical wholesale replacement of such
software.

> >That wording is inadequate.
> 
> Then you are welcome to propose alternative wordings.

No change would be better than that change.

> OK, you want a NOTE warning would-be generators of identities to avoid all
> hex-digit-only names?

No, I want clear semantics and a corresponding mechanism that address
the issue, leaving no known technical omission.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Tue, 28 Jun 2005 09:53:43 -0400
Cc: [email-masked]
Sender: [email-masked]

On Mon June 27 2005 15:26, Charles Lindsey wrote:

> They clearly say nothing about the semantics of <id-right>.

2822 clearly says "the domain name (or a domain literal IP address"; it
does not say "some random string" or anything like it.  Nor does it say
that the semantics have changed from RFC 822.

[...]
> >Please cite RFC 2822 text supporting your "opaque labels" assertion.
> 
> Please cite the RFC text supporting your assertion that they are anything
> else than "opaque" (once generated).

Sorry, Charles; your claim, your burden of proof.  Prove or retract.

Date: Tue, 28 Jun 2005 14:53:23 +0100
From: Alexey Melnikov <[email-masked]>
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

Charles Lindsey wrote:

>You could also argue that, since this WG has consistenly used the "header"
>terminology since the earliest days, that it should require an actual
>consensus to make the change before reversing that decision.
>  
>
I was under impression that this is exactly what this thread is all about.

Alexey

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Tue, 28 Jun 2005 09:49:00 -0400
Cc: [email-masked]
Sender: [email-masked]

On Mon June 27 2005 15:01, Charles Lindsey wrote:

> Simply changing the name of some syntax rule cannot possibly change the
> meaning of the document.

Ergo, by your own logic, "id-right" does not change the meaning of
the RHS of a msg-id from that of a domain name or literal.  Nor
does it change the fact that domain names are case-insensitive.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Date: Tue, 28 Jun 2005 09:45:14 -0400
Sender: [email-masked]

On Mon June 27 2005 14:53, Charles Lindsey wrote:

> if head counting is required.

May I remind the co-chairs that there is still the unresolved
question as to the method that is to be used to determine
consensus; is it in fact a head count (in which case those of us
who have been using reasoned arguments can adopt the simpler
method of making a non-sequitur comment), or is it to be based
on reasoned arguments as was the case in the Resnick-Gierth era,
or something else? 

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Tue, 28 Jun 2005 09:38:15 -0400
Cc: [email-masked]
Sender: [email-masked]

On Mon June 27 2005 12:59, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

> >While you might not be aware of problems, they certainly exist. Although
> >the UUCP mapping project has good intentions and was intended to avoid
> >clashes, it failed to do so.  I happen to know for a fact (and have
> >previously mentioned here) that there have been unresolved duplicates;
> >one that I can recall off the top of my head even after a decade is
> >multiple sites claiming the name "wolf".  I am not aware that the duplicates
> >were ever resolved.  Do you have evidence to support your claim that "they
> >have been sorted out amicably"?
> 
> Do you have evidence that they have not?

1. I do; the issue was reported ages ago, and not resolved after years.

2. Reasoned debate is predicated on a claimant supporting his or her
   assertions (ideally, *when* they are made), not in saying "here's
   my assertion; you prove me wrong".  You have made the claim that
   such issues "have been sorted out amicably" -- with to date no
   supporting evidence, and you have been asked to support *your*
   claim.  The burden of proof for *your* claim is on *you*, not me.
 
> It is clearly the reponsibility of anyone adopting an identity not based
> on DNS of IP addresses to choose something that is unique. Our drafts do
> not, cannot and should not specify how that is ensured.

Au contraire, requiring uniqueness but failing to indicate a suitable
method by which uniqueness can be assured is a "known technical omission"
(see BCP 9).

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Tue, 28 Jun 2005 09:29:55 -0400
Cc: [email-masked]
Sender: [email-masked]

On Mon June 27 2005 12:33, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >We need to specify the semantics of a "path-identity" as one of:
> >o an opaque label
> >o a public name
> >o domain names, IP-addresses, etc.
> 
> No we don't.
> 
> 1. I am not clear what is the distinction between "opaque label" and
> "public name", or whether it matters for our purpose.

See RFC 1958.
 
> 2. <path-identity>s are used for various purposes. They are used by
> relayers to prevent articles being sent where they have already been sent.
> Performance issues require that, for such use, regarding them as "opaque
> labels" (enabling them to be compared octet by octet) is essential. Our
> drafts must impose no requirements that would prevent that opaque
> interpretation. They do not currently impose any such.
> 
> 3. <path-identity>s are also useful for humans, who may well use the fact
> that they are domain-names or IP-addresses for a variety of purposes
> (including generating email to "[email-masked]"). They may even be
> interpreted as such by software outside of the critical code within
> relayers, if implementors find that useful (for example, when constructing
> lists of identities used by their various peers).
> 
> >It is unsatisfactory, because it is self-inconsistent, to give contradictory
> >semantics.
> 
> I see nothing contradictory in treating a <path-identity> as opaque for
> some purposes, but not for others. I also see nothing unsatisfactory about
> it either, particularly as that is the way is has been for the last 20
> years or more.

So you see no problem in some random site in some random place with
some random affiliation using the opaque label "clerew.man.ac.uk".
 
> >> Also, in order to detect MISMATCHes, you will likely need to know whether
> >> the IP address the NNTP packet
> 
> >NNTP is not the only transport mechanism.
> 
> So?

So a mismatch in source of UUCP data requires no IP address information,
and your argument equating a specific case with the general case fails.

Date: Tue, 28 Jun 2005 15:25:55 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Admin: Ticket status
Sender: [email-masked]

I'm listing the tickets in numeric order.

1002 USEFOR 6. References - Updates needed

 Resolution proposed on June 14, no objection to content.
 Status: "text accepted".

1003 USEFOR 3.1.3 - Cleanup ABNF for msg-id
 Status: "text needed"
 I think I can produce a proposed resolution, but it requires wading
 through the message log again.

1004 USEFOR 3.2.14 - Can we deprecate X- headers?
 Status: Text proposed. Some debate on X-Mailer.

1008 USEFOR 1.5: relation between a followup and a References: header?
  Status: "Text accepted".

1021 USEFOR 3.1.5 Newsgroups header ABNF and description needs cleanup
  Status: "Text proposed" - debate. Need to review.

1022 USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
  Status: "No consensus" - still only 2 voices.

1028 USEFOR 3.1.2 Date: What zones should be on the MUST accept list?
  Status: "Text proposed". I'll send out a message; I think we're
  finished with it.

1029 USEFOR 3.2.2 References: Should comments be a MUST NOT?
  Status: "No consensus". I'm not sure that's right; will review.

1030 USEFOR general: Backwards compatibility and handling incompatibilities
  Status: "No consensus". Unless specific suggestions come from it, I'll
  change the status to "No change needed".

1031 USEFOR general: Standard terminology and text
  Status: "No consensus". Unless further specific suggestions come from
  this once the "header" issue is settled, I'll change the status to
  "No change needed".

1032 USEFOR general: Document changes from RFC 1036
  Status: "Text needed"

1042 USEFOR 3.1.5: Newsgroups folding?
  Status: "Text needed". I think there's consensus that at this time,
  we cannot recommend folding newsgroups.

1043 USEFOR 2.1 use of term "header"
  Status: "Text proposed"

1046 USEFOR 5. MIME boundary security considerations
  Status: "Text needed". Principle seems uncontroversial.....

1047 USEFOR 3.1.6: Path field delimiters and components
  Status: "Text needed"

1048 USEFOR 3.2.1 Injection-date syntax
  Status: "Proposed no change"

1050 USEFOR 3.2.11: Xref field for local use only?
  Status: "No change needed"

So far, no tickets have been opened on USEPRO.

Date: Tue, 28 Jun 2005 10:12:50 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: "Forrest J. Cavalier III" <[email-masked]>, [email-masked]
Subject: Chair hat ON: Free-standing documents (Re: #1043 "header": Suggested resolution)
Sender: [email-masked]

--On 27. juni 2005 23:37 -0400 "Forrest J. Cavalier III" 
<[email-masked]> wrote:

>> That is a sad fact of life. If you write a document that is not free
>> standing, and/or does not contain sufficient explanatory material to
>> assist the "less than omniscient reader", then people are going to cut
>> corners when they come to read it.
>
> This is a preposterous position for a WG participant, and absolutely
> unacceptable position for a WG editor.
>
> Can I ask the chairs to issue clear statements that we are NOT
> writing "free-standing" documents, and that it will be impossible to
> create a conforming implementation without reading RFC2822?

We are not writing "free-standing" documents, and it will be impossible to 
create a conforming implementation without reading RFC 2822. (1)

As I've remarked in another thread: Sometimes you need to state the obvious.

                        Harald

(1) nitpicking mode ON: it's possible that some freestanding piece of 
software that performs a single function, and which uses libraries written 
by someone who's carefully studied RFC 2822, can be conforming. It's just 
quite unlikely.

Date: Tue, 28 Jun 2005 00:05:18 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Steve Walker <[email-masked]> wrote:
> Henry Spencer wrote:

>> That is their problem, not ours. 
>
> It becomes everyone's problem.  What are people going to do when 
> some idiot starts sending 6 million cancel messages per day.

Yawn, ignore them and him, and depeer him.

Just like the last time.

Seth

To: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 27 Jun 2005 20:45:50 -0700
Sender: [email-masked]

Steve Walker <[email-masked]> writes:
> Henry Spencer wrote:

>> The reason why yenc isn't compatible with MIME is that the yenc people
>> don't care about anything but their own little world.  *They* are not
>> interested in providing for future adaptation and growth. That is their
>> problem, not ours.

> It becomes everyone's problem.  What are people going to do when some
> idiot starts sending 6 million cancel messages per day.

You start filtering out their messages until their feed gets terminated.
What does that have to do with yEnc, or for that matter, with
standardization in general?

> I have a network/servers that can handle this load, but the smaller
> sites may not.  If Usenet doubles every 2 years in ten years you are
> looking at >100 million non-compliant article per day.

None of which are carried on the text servers anyway.  Besides, what
difference does that make?  What's the point of an argument from volume?
Probably 90% of HTML web pages don't comply with the HTML standard either;
does that mean that the HTML standard is a waste of time or that all those
pages should be declared compliant?  Of course not.

Date: Mon, 27 Jun 2005 22:44:20 -0500
From: Steve Walker <[email-masked]>
To: Henry Spencer <[email-masked]>
CC: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

Henry Spencer wrote:

> The reason why yenc isn't compatible with MIME is that the yenc people
> don't care about anything but their own little world.  *They* are not
> interested in providing for future adaptation and growth. 
> 
> That is their problem, not ours. 

It becomes everyone's problem.  What are people going to do when 
some idiot starts sending 6 million cancel messages per day.  I 
have a network/servers that can handle this load, but the smaller 
sites may not.  If Usenet doubles every 2 years in ten years you 
are looking at >100 million non-compliant article per day.

Steve Walker
NNTPServer.com

Date: Mon, 27 Jun 2005 23:39:03 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

Charles Lindsey wrote:

> In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:
> 
> 
>>2822 is a NORMATIVE reference.  We refer to and require people to
>>use it.  The ENTIRE audience will need to refer and understand
>>RFC2822.
> 
> 
> But I suspect that the entire audience is not going to refer to RFC 2822
> (because they will just use what they "remember from last time they read
> it"). And I doubt they will fully understand it, much less the
> ramifications of all the other MIME etc documents that we refer to
> normatively.
> 
> That is a sad fact of life. If you write a document that is not free
> standing, and/or does not contain sufficient explanatory material to
> assist the "less than omniscient reader", then people are going to cut
> corners when they come to read it.

I object to all parts of the drafts written or inserted because
of an assumption that implementors are uncooperative and lazy.

Adopting this position officially will clear the way for removal of
inappropriate uses of MUST/NOT imperatives, result in a shorter document,
and I believe will improve the tone and usefulness of several sections.

(This mostly affects USEPRO, I think.)

Date: Mon, 27 Jun 2005 23:37:12 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:
> 
> 
>>2822 is a NORMATIVE reference.  We refer to and require people to
>>use it.  The ENTIRE audience will need to refer and understand
>>RFC2822.
> 
> 
> But I suspect that the entire audience is not going to refer to RFC 2822
> (because they will just use what they "remember from last time they read
> it"). And I doubt they will fully understand it, much less the
> ramifications of all the other MIME etc documents that we refer to
> normatively.
> 
> That is a sad fact of life. If you write a document that is not free
> standing, and/or does not contain sufficient explanatory material to
> assist the "less than omniscient reader", then people are going to cut
> corners when they come to read it.

This is a preposterous position for a WG participant, and absolutely
unacceptable position for a WG editor.

Can I ask the chairs to issue clear statements that we are NOT
writing "free-standing" documents, and that it will be impossible to
create a conforming implementation without reading RFC2822?  In evidence
that this must be so, I offer the wording in Section 3 of USEFOR-USEFOR-04.

To: [email-masked]
Subject: Re: Feedback on proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 27 Jun 2005 20:34:22 -0700
Sender: [email-masked]

Steve Walker <[email-masked]> writes:

> I think most people are agreeing that a mime compliant yEncoder standard
> isn't possible in the near future.  If you want mime and you don't want
> to break existing practice, the only choice I see is to not require
> mime.  It's the only way not to break stuff.

Well, I vote for requiring MIME and letting non-MIME articles just be
non-compliant.  It's not like IETF standards are some sort of government
law, or that software that doesn't comply with them will magically stop
functioning.

We need to make sure that nothing in the standard requires software to
refuse to deal with non-compliant articles and then leave the degree to
which people want to deal with them up to the individual software authors,
exactly the way that yEnc is now.

Date: Mon, 27 Jun 2005 22:35:37 -0500
From: Steve Walker <[email-masked]>
To: Russ Allbery <[email-masked]>
CC: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Russ Allbery wrote:
> Steve Walker <[email-masked]> writes:
> 
> 
>>The existing practice is posting non-mime articles.  Today the bulk may
>>be yEncoded and next year it may be something else. It's certain that
>>Usenet will adapt and grow.  Forcing absolute restrictions on the
>>article format is a bad idea.
> 
> 
> Steve, this is the sort of statement that causes people involved in
> message format standardization to tear their hair out.  This is exactly
> the *point* of MIME -- it's an *extensible* framework for allowing the
> message format to change and evolve.  This is the whole reason why we're
> *using* it.  MIME takes care of many of the corner cases that make
> changing the article format hard.

I can certainly see this point and I'm willing to bet that the 
yEncoder author would also see the value of MIME if they took the 
time.  It doesn't change the fact that we have a huge amount of 
non-mime articles to deal with today and for the foreseeable 
future.  We just need a little black box to shove all the 
non-mime articles into until everyone either uses mime or it's 
replaced with something better.

I think most people are agreeing that a mime compliant yEncoder 
standard isn't possible in the near future.  If you want mime and 
you don't want to break existing practice, the only choice I see 
is to not require mime.  It's the only way not to break stuff.

Steve Walker
NNTPServer.com

Date: Mon, 27 Jun 2005 23:17:30 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
cc: Steve Walker <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

On Mon, 27 Jun 2005, Steve Walker wrote:
> The existing practice is posting non-mime articles.

Depends very much on where you look.  Per earlier comments by Russ, I ran
some numbers on a sample of just under 54,000 articles, from most of the
*text* newsgroups in the traditional Usenet hierarchies, and found 58% had
a MIME-Version header.  Existing practice in the text groups *is*
increasingly to use MIME. 

See earlier comments about the binary groups being basically a separate
community. 

> Today the 
> bulk may be yEncoded and next year it may be something else. 
> It's certain that Usenet will adapt and grow...

Exactly.  And precisely because we can't predict just what formats will be
in use in the future, there is every reason to want some kind of standard
labeling scheme, so you can *tell* what format an article is in and
whether you know how to decode it, rather than having to guess based on
heuristics.  The labeling system needs to be extensible, so it covers data
formats we don't know about today.  And it should be non-intrusive, so it
doesn't inject garbage into the Subject header or other headers which
already have other purposes.

And that's what MIME is.  It's a little more complicated than it needs to
be, partly because there were some difficult backward-compatibility
constraints.  But by and large, MIME is a well-designed framework to
support future adaptation and growth, and make it less painful.

The reason why yenc isn't compatible with MIME is that the yenc people
don't care about anything but their own little world.  *They* are not
interested in providing for future adaptation and growth. 

That is their problem, not ours. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Feedback on proposal
Date:  Tue, 28 Jun 2005 05:13:21 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 11
Sender: [email-masked]

Steve Walker wrote:

> RFC 1036 does not even include the word MIME.

That's no surprise, 1036 < 1341 < 1521 < 2045.

OTOH I get 79 occurences of MIME in 78 lines
of s-o-1036.  Only 26 in 24 lines of usefor-04.

              Bye, Frank

Date: Mon, 27 Jun 2005 22:07:16 -0500
From: Steve Walker <[email-masked]>
To: Henry Spencer <[email-masked]>
CC: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

Henry Spencer wrote:

> As you've been told repeatedly, just because a client is MIME-capable
> doesn't mean that it insists on handling only MIME messages.  Most
> plain-text articles on Usenet today do not have MIME-Version headers.
> A Usenet client which ignored any article lacking a MIME-Version header
> would be useless. 

If your intent is to allow millions of non-mime articles to flow 
freely each day through the Usenet why would you require all 
Usenet clients to support MIME.  The support may even be a two 
edged sword in that the client who supported mime wouldn't be 
able to post non-mime and still call themselves a compliant news 
client.

Why would software developers bother supporting any requirements, 
like line length and 7bit headers, it they are still going to be 
called non-compliant.  Having hundreds of clients each making 
their own rules would be bad for Usenet in general.

 From your charter (website):
> The Goal of this working group is to publish a standards-track
> successor to RFC 1036 that with particular attention to
>  backward compatibility

How is requiring mime backwards compatible with out providing 
some type of out (meaning SHOULD instead of MUST) for the 
existing articles that are not, and can not be, mime compliant?

RFC 1036 does not even include the word MIME.

Steve Walker
NNTPServer.com

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1048 Injection-Date syntax
Date:  Tue, 28 Jun 2005 04:42:22 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 7
Sender: [email-masked]

Charles Lindsey wrote:

> I still prefer GMT on its own

Add me, but without s/SHOULD NOT/MUST NOT/ please,
GMT is neither critical nor harmful.  Bye, Frank

To: [email-masked]
Subject: Re: Feedback on proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 27 Jun 2005 19:38:52 -0700
Sender: [email-masked]

Steve Walker <[email-masked]> writes:

> The existing practice is posting non-mime articles.  Today the bulk may
> be yEncoded and next year it may be something else. It's certain that
> Usenet will adapt and grow.  Forcing absolute restrictions on the
> article format is a bad idea.

Steve, this is the sort of statement that causes people involved in
message format standardization to tear their hair out.  This is exactly
the *point* of MIME -- it's an *extensible* framework for allowing the
message format to change and evolve.  This is the whole reason why we're
*using* it.  MIME takes care of many of the corner cases that make
changing the article format hard.

This sort of massive missing of the point is a lot of the reason why I
have a hard time caring about the binary Usenet any more.  It's exactly
what made the original yEnc discussions so frustrating.  I and others put
a lot of time into trying to get yEnc to be deployable in the right way,
and was basically told that our input was not of interest, that
standardization was a waste of time, and that they found efforts to make
things interoperate properly offensive.

Not responsible for advice not taken.  We can't standardize the work of
people who are actively opposed to standardization.  As far as I'm
concerned, enough of that reaction and one forfeits one's seat at the
table.  If the yEnc folks are happy to be unstandardized, they can go
right ahead and continue to be unstandardized after this working group
finishes its work, as far as I'm concerned.

There is *way* too much NIH syndrome and inferior reinventions of the
wheel on Usenet historically.

In the meantime, I have problems that I want to solve, and I'm not going
to stop trying to solve them just because other people are uninterested in
cooperation.  Nor do I think that we should bend over backwards to
accomodate the needs of people who have chosen to stay actively uninvolved
in the process.

To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 27 Jun 2005 19:30:27 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Under Duties of a Serving Agent:

>    7. It MUST remove any Xref header (F-3.2.10) from each article.  It
>       then MAY (and usually will) generate a fresh Xref header.

Please allow for Xref slaving.  (I don't know if the context takes care of
that.)

> I see little merit in upgrading that MAY to a SHOULD (no
> interoperability issue or harm arises). It is a good practice, and to be
> commended as such, which is why that MAY is there.

I think the lack of an Xref header would be surprising to clients, so I
would lean towards SHOULD personally, but I haven't really thought about
it at any great length.

> And I take issue with your use of the term "local system". I cannot
> accept that currently used terminology would equate a server plus all
> its thousands of clients scattered around the globe with a "local
> system".

Now if you'd just apply that same argument to "cooperating subnet"...  :)

To: [email-masked]
Subject: Re: #1047: Path field delimiters and components
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 27 Jun 2005 19:28:35 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> On the contrary, I stand by every word that I said. It is highly
> improbable that any currenly existing <path-identity> will run into any
> problem when ":" ceases to be a path-delimiter.

Ceasing to use it as a path delimiter is not the problem; in practice, no
one uses anything other than "!".  Starting to use it for something *else*
is the problem, since sites will continue treating it as a path delimiter.

I do agree that the possibility of colliding with a <=4 character path
identity isn't exactly the sort of risk that would keep me up at night.

> And how many sites do you suppose there exist on Usenet which consist of
> exactly four Hex Digits?

Pedant point:  one to four hex digits, as you can omit the zero padding in
IPv6 address specifications.

Date: Mon, 27 Jun 2005 21:33:26 -0500
From: Steve Walker <[email-masked]>
To: Seth Breidbart <[email-masked]>
CC: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Seth Breidbart wrote:
> Steve Walker <[email-masked]> wrote:
> 
> 
>>The percentage, which is greater than 65% and growing, of 
>>non-conformance is too large to just be ignored.
> 
> 
> Where are you getting that number?  Are you including plaintext
> articles that have only ASCII characters and lack MIME headers?  Those
> _are_ valid MIME.

I am counting only yEncoded articles in that number and only the 
ones that bother to include the word yEnc in the Subject header. 
  The number of non-conforming articles is higher than 65%.  I 
just choose to use yEnc'ed because it's the most popular article 
format, conforming or not.

> It's trying to standardize those existing practices which can be
> standardized.  Apparently yEnc can't be; at least, the efforts so far
> have failed, and this is the wrong group to work on standardizing yEnc
> anyway, for any number of reasons.

The existing practice is posting non-mime articles.  Today the 
bulk may be yEncoded and next year it may be something else. 
It's certain that Usenet will adapt and grow.  Forcing absolute 
restrictions on the article format is a bad idea.

> If you want us to point to a yEnc standard in addition to the MIME
> standard, show us the one to point to.

Even if there was a fixed yEncoding standard I would still not 
suggest that it be used.  The article format need to remain 
flexible so that Usenet can continue to grow and adapt.   There 
needs to be someway to allow posting content without necessary 
restrictions on the exact message structure.

Steve Walker
NNTPServer.com

To: [email-masked]
Subject: Re: Feedback on proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 27 Jun 2005 19:25:09 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Yes, the Yenc people have been told, repeatedly, that they could start
> using

>     Content-Transfer-Encoding: x-yenc

> tomorrow without breaking any rules. But they just don't want to listen.
> What is abundantly clear is nothing which requires special keywords in
> the Subject header for it to work is _ever_ going to get to be an IETF
> standard.

> Before reading Russ's message, I was about to propose:

>     Content-Type: application/octet-stream
>     Content-Transfer-Encoding: 8bit

> as a way of legitimizing current Yenc practice (see remark from Frank, I
> believe, that Yenc encourages lines to be kept well within 998).

Bruce's message is entirely correct; they should really be using:

    MIME-Version: 1.0
    Content-Type: image/jpeg
    Content-Transfer-Encoding: x-yenc

or whatever the actual content of the thing is so that software knows what
this blob of data is supposed to decode to.

All the additional metadata and multipart stuff is still a problem,
though.  If it's actually a multipart message, I don't think there's any
way to get around needing application/x-yenc right now (and of course,
yEnc doesn't have any way of embedding the file type because the author of
yEnc didn't understand the need, so you have to play extension guessing
games to figure it out after that, with all the problems that go with
that).

To: [email-masked]
Xref: clerew local.usefor:21610
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Mon, 27 Jun 2005 19:26:02 GMT
Lines: 77
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Fri June 24 2005 11:55, Charles Lindsey wrote:
>> 
>> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
>> 
>> >On Wed June 22 2005 15:18, Charles Lindsey wrote:
>> >> 
>> >> In RFC 2822 an <id-right> is a string of characters satisfying some ABNF.
>> >> It has no semantics attached to it (except that a `\` in a <quoted-pair>
>> >> is "invisible").
>> 
>> >No, the semantics are "the domain name (or a domain literal IP address)".
>> 
>> I do not see where RFC 2822 says that.

>It says so (in those words) in the part which I quoted from 2822 and which
>you have snipped.

Here are the snipped words:
> >   Since the msg-id has
> >   a similar syntax to angle-addr (identical except that comments and
> >   folding white space are not allowed), a good method is to put the
> >   domain name (or a domain literal IP address) of the host on which the
> >   message identifier was created on the right hand side of the "@", and
> >   put a combination of the current absolute date and time along with
> >   some other currently unique (perhaps sequential) identifier available
> >   on the system (for example, a process id number) on the left hand
> >   side.
> 
They clearly say nothing about the semantics of <id-right>. Note that the
word "semantics" implies something that tells you the "meaning".

The text in question gives sound (even RECOMMENDED) advice on how to
*generate* an <id-right>, in order to fulfil the REQUIREMENT that what
you generate is to be globally unique. It tells you nothing about the
meaning of the object you have generated, or what it is to be used for, or
how (though there are instructions elsewhere in RFC 2822 as to how to use
it in constructing a References field). In particular, there is nothing to
say that its semantics (meaning) is in any way related to the semantics
(meaning) of a <domain>.

If you read elsewhere in RFC 2822, you will find that, semantically, the
<msg-id> is intended to identify a particular version of a particular
message, and also that certain rules regarding quoted-pairs are relevant
(though that bit does not apply in USEFOR). And that is all the semantics
that is given.

>  John Stanley seems to have identified some problem
>related to editing (unless of course you were exaggerating about "an hour
>or so" (N.B. there was a "smiley" notation indicating sadness, but none
>indicating intended humor or exaggeration));

John thinks the edit could be done in 10 minutes. He might well be right,
but the remaining 50 minutes, "or so", would be needed to remove all the
false positives that such a naive approach would undoubtedly throw up :-).

>> All of <path-identity>, <id-left> and <id-right> are "opaque 
>> labels", whether you initially construct them out of domain-names or
>> domain literals, or whatever else.

>Please cite RFC 2822 text supporting your "opaque labels" assertion.

Please cite the RFC text supporting your assertion that they are anything
else than "opaque" (once generated).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21603
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Mon, 27 Jun 2005 16:59:01 GMT
Lines: 65
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Fri June 24 2005 11:31, Charles Lindsey wrote:

>OK; we're agreed that there is a uniqueness requirement. N.B. requirement
>as in "necessary", not "wouldn't it be nice if...".
> 
>> >  That in turn requires either
>> >a hierarchical system (like DNS) or a central authority (like the
>> >UUCP mapping project was supposed to be, though it never worked
>> >well).
>> 
>> But that does not follow at all. Usenet has been working quite happily for
>> many years now just by admins using their common sense when choosing
>> <path-identity>s. I am not aware of serious clashes that have occurred,
>> and if they have then they have been sorted out amicably (because it is in
>> everybody's interest to do so).

>While you might not be aware of problems, they certainly exist. Although
>the UUCP mapping project has good intentions and was intended to avoid
>clashes, it failed to do so.  I happen to know for a fact (and have
>previously mentioned here) that there have been unresolved duplicates;
>one that I can recall off the top of my head even after a decade is
>multiple sites claiming the name "wolf".  I am not aware that the duplicates
>were ever resolved.  Do you have evidence to support your claim that "they
>have been sorted out amicably"?

Do you have evidence that they have not? Do you have evidence that the
various sites called "wolf" have actually used that name in their
path-identities? Have you ever seen dicussion in the various news.admin.*
newsgroups discussing problems arising from such causes (I have not, that
I recall). Has Usenet been brought to its kneed due to this cause (I often
hear of the "death of the net predicted" for a variety of causes, but
never from that one).

In short, have you any evidence that the present setup (with no Cabal and
not even any UUCP maps anymore) is in any sense "broken"?

Clearly, use of domain-names or IP addresses is an excellent way of
ensuring that you have acquired a unique path-identity, but not all sites
necessarily even have an internet connection (for example, those using
UUCP or carrier pigeons).

It is clearly the reponsibility of anyone adopting an identity not based
on DNS of IP addresses to choose something that is unique. Our drafts do
not, cannot and should not specify how that is ensured. It is a problem
for that site to sort out as it sees fit. All out drafts have ever said
(and this is now in USEPRO) is:

|  3. An arbitrary name believed to be unique and registered at least
|     with all other news-servers receiving articles directly from the
|     given one.

If you do not like that, then you are free to propose alternatives.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21602
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Mon, 27 Jun 2005 16:33:05 GMT
Lines: 64
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>We need to specify the semantics of a "path-identity" as one of:
>o an opaque label
>o a public name
>o domain names, IP-addresses, etc.

No we don't.

1. I am not clear what is the distinction between "opaque label" and
"public name", or whether it matters for our purpose.

2. <path-identity>s are used for various purposes. They are used by
relayers to prevent articles being sent where they have already been sent.
Performance issues require that, for such use, regarding them as "opaque
labels" (enabling them to be compared octet by octet) is essential. Our
drafts must impose no requirements that would prevent that opaque
interpretation. They do not currently impose any such.

3. <path-identity>s are also useful for humans, who may well use the fact
that they are domain-names or IP-addresses for a variety of purposes
(including generating email to "[email-masked]"). They may even be
interpreted as such by software outside of the critical code within
relayers, if implementors find that useful (for example, when constructing
lists of identities used by their various peers).

>It is unsatisfactory, because it is self-inconsistent, to give contradictory
>semantics.

I see nothing contradictory in treating a <path-identity> as opaque for
some purposes, but not for others. I also see nothing unsatisfactory about
it either, particularly as that is the way is has been for the last 20
years or more.

>> Also, in order to detect MISMATCHes, you will likely need to know whether
>> the IP address the NNTP packet

>NNTP is not the only transport mechanism.

So? It is widely used, and current implementations regularly take note of
the source IP of incoming packets for a variety of purposes. It is
expected that those purposes will, in future, include detection of
MISMATCHes (indeed, some servers even do so currently).

>Irrelevant to the discussion at hand, which is interpretation of the
>"path-identity" *as* an IP address, not interpretation of an IP address
>obtained from another source.

The expectation is that those two will be the same, and newsadmins should
not be accepting differences without knowing why (there are, of course,
lots of valid reasons why it might be so, and also lots of reason why it
might indicate malpractice).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21611
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047: Path field delimiters and components
Date: Mon, 27 Jun 2005 19:39:22 GMT
Lines: 53
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Fri June 24 2005 12:44, Charles Lindsey wrote:

>> Yes, there is a theoretical possibility of a backward incompatibility

>There is nothing "theoretical" about it; you know this because it has
>been discussed at length, with supporting documentation -- we know for
>a fact that B news, C news, and INN treat colon as a delimiter, and
>that reassignment will therefore unquestionably break backward
>compatibility.  It is disingenuous of you, and frankly insulting to
>others, for you to categorize this as "theoretical".

On the contrary, I stand by every word that I said. It is highly
improbable that any currenly existing <path-identity> will run into any
problem when ":" ceases to be a path-delimiter. And even if it does it
will be of limited impact, and will easily be fixed. Hence "theoretical
possibility" was an appropriate way to describe it.

>> I have already proposed a wording (in reply to John) for inclusion in
>> USEFOR to advise care when choosing <path-identity>s.

>That wording is inadequate.

Then you are welcome to propose alternative wordings.

>Sigh (again).  Charles, do you honestly fail to recognize, after it has
>been pointed out and after Frank has also mentioned "cafe" etc., that
>"dead" and "beef" are composed entirely of characters representing
>hexadecimal digits (and moreover DEADBEEF is a well-known set of such
>heaxadecimal digits) and therefore might well appear in an IPv6 address
>literal?

And how many sites do you suppose there exist on Usenet which consist of
exactly four Hex Digits?

Perhaps somebody with access to a large server and some spare machine
cycles to burn would care to do a grep for us.

OK, you want a NOTE warning would-be generators of identities to avoid all
hex-digit-only names?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21612
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Mon, 27 Jun 2005 19:50:31 GMT
Lines: 51
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>RFC 1036 says, regarding the Xref field:

>    This is only of value to the local system, so it should not be
>    transmitted.

>There is no corresponding text in the usefor draft version -04.  Best
>current practice seems to be to keep track of local data in a separate
>database rather than in a non-transmitted addition to the article
>header.  I therefore propose that the following text (subject to discussion
>of course) be added to the usefor draft Xref text:

Gentlemen,

This is a protocol issue, and therfore it belongs in USEPRO, and most
certainly not in USEFOR.

And USEPRO already says what you want.

Under Duties of a Relaying Agent:

   8. It MAY delete any Xref header that is present.

Under Duties of a Serving Agent:

   7. It MUST remove any Xref header (F-3.2.10) from each article.  It
      then MAY (and usually will) generate a fresh Xref header.

I see little merit in upgrading that MAY to a SHOULD (no interoperability
issue or harm arises). It is a good practice, and to be commended as such,
which is why that MAY is there.

>  The optional Xref field has been used as a place to store
>  cross-reference information of use to a local system.  Other
>  methods of handling such metadata include separate databases.

And I take issue with your use of the term "local system". I cannot accept
that currently used terminology would equate a server plus all its
thousands of clients scattered around the globe with a "local system".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21607
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1043 "header": Suggested resolution
Date: Mon, 27 Jun 2005 18:53:37 GMT
Lines: 34
Sender: [email-masked]

In <29379DD0F7F08A6E55C5B833@[192.168.1.113]> Harald Tveit Alvestrand <[email-masked]> writes:

>  NOTE: Older News specifications used the term "header" as a synonym
>     for what [RFC2822] calls "header field". This document follows
>     the terminology of RFC 2822 in using the term "header field",
>     based on a belief that consistent terminology among specifications
>     that depend on each other makes the specifications easier to use
>     in the long run.

Add "even if it confuses the rest of the world, and leads them to complain
you are stuck in your ivory tower and would rather you had a document that
was pedantically correct rather than one of use to the people working at
the coal face".

And I suspect that "in the long run" the terminology that the marketplace
uses will eventually prevail, even for email.

I am still with Henry on this one, and I see that Graham Drabble has now
joined us, if head counting is required.

You could also argue that, since this WG has consistenly used the "header"
terminology since the earliest days, that it should require an actual
consensus to make the change before reversing that decision.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21606
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1043 "header": Suggested resolution
Date: Mon, 27 Jun 2005 18:45:17 GMT
Lines: 40
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>2822 is a NORMATIVE reference.  We refer to and require people to
>use it.  The ENTIRE audience will need to refer and understand
>RFC2822.

But I suspect that the entire audience is not going to refer to RFC 2822
(because they will just use what they "remember from last time they read
it"). And I doubt they will fully understand it, much less the
ramifications of all the other MIME etc documents that we refer to
normatively.

That is a sad fact of life. If you write a document that is not free
standing, and/or does not contain sufficient explanatory material to
assist the "less than omniscient reader", then people are going to cut
corners when they come to read it.

That is why Son-of-1036 was so long, and it is also why our earlier
article-nn drafts were so long, and it is a lesson I and my colleagues
learnt the hard way many years ago when writing the Revised Algol 68 Report.

And yet Usefor is excessively short and brief (with no examples, and few
NOTEs). Some people seem to think that is a Good Thing. We shall see.

>I made this point before.  I do not recall any reply addressing
>the point.  Maybe we shouldn't use RFC2822 as a base.

There is much merit in that, but we have already changed horses in
mid-stream once. Let us not do it again.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21605
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1048 Injection-Date syntax
Date: Mon, 27 Jun 2005 18:32:14 GMT
Lines: 35
Sender: [email-masked]

In <7A11E70C35850D512E2E732F@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>I have created ticket #1048 for this issue, with a "Discussion:" field of 
>"Proposed no change", since that's what I ended up parsing Bruce's comment 
>as.

>This may be uncontroversial......

>                 Harald

Yes. I think the last thing we want is different syntax for the various
headers that use <date-time>. By all means, if sites want to use whatever
"MAY accept" discretion we give them differently for different headers,
then that is their choice.

AFAICS, the only two options open to us are to leave the syntax as Harald
recently proposed it, with just "GMT" on its own. Or else to replace that
"GMT" with the whole of <obs-zone>. Oh, and the generation of GMT or
<obs-zone) should be MUST NOT rather than the SHOULD NOT that is there at
the moment. Apart from that, the general "MAY accept obs-syntax" gives
everything else that is required.

I still prefer GMT on its own, and I think that was the consensus view as
Harald determined it when he opened this ticket. But it is Harald's call.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21604
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Feedback on proposal
Date: Mon, 27 Jun 2005 17:09:24 GMT
Lines: 37
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Actually, I believe that:

>    MIME-Version: 1.0
>    Content-Type: application/x-yenc
>    Content-Transfer-Encoding: x-yenc

>does pretty much what you would want from the perspective of a MIME client
>that doesn't understand yEnc.  The unrecognized CTE pretty much tells
>everything "hands off" in a particularly thorough manner. ...

Yes, the Yenc people have been told, repeatedly, that they could start using
    Content-Transfer-Encoding: x-yenc
tomorrow without breaking any rules. But they just don't want to listen.
What is abundantly clear is nothing which requires special keywords in
the Subject header for it to work is _ever_ going to get to be an IETF
standard.

Before reading Russ's message, I was about to propose:

    Content-Type: application/octet-stream
    Content-Transfer-Encoding: 8bit

as a way of legitimizing current Yenc practice (see remark from Frank, I
believe, that Yenc encourages lines to be kept well within 998).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21613
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1043 "header": Suggested resolution
Date: Mon, 27 Jun 2005 19:57:40 GMT
Lines: 34
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>I'd also like to add two more definitions, but those are less important:

>  This specification also uses the following terms:

>  Header line - a line as defined in [RFC2822] section 2.1 that forms
>         part of a header field.

I think that depends on whether and how the term gets used when the
problems you identified earlier get fixed (I hope Ken is owrking on that
one).

>  Header field content - the field body of a header field, not including
>         leading or trailing whitespace, or comments.

>I'm not sure whether the last one is what Charles wanted for his "semantic 
>content" construct - but I'm sure he'll tell us if it's not.

Please leave me to define the term in USEPRO, where it will be used. My
next job is to trawl through all the places in USEPRO that would use the
term, in order to see exactly what properties I need in order to make it
work.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21608
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Our web site
Date: Mon, 27 Jun 2005 18:59:24 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> "Charles Lindsey" <[email-masked]> writes:

>In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>>Unfortunately the online WG mailing list archive is incomplete.

>I eventually got around to updating my Perl script to grab them from the
>Landfield site, and even managed to download a couple of months' worth.

>But next day I found the site down, and I have not been able to catch it
>up since. But I shall keep trying.

And I have at last caught that site "up", and my script to download the
missing archive is chugging away, even as we speak.

>And I was going to put the USEFOR-04 draft up at the same time, which is
>why that has never happened as of yet.

And I have now done that.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21609
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Mon, 27 Jun 2005 19:01:47 GMT
Lines: 20
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>So for these two details the rough consensus is IMHO clear.
>Putting it together the consequence _should_ be also clear:

>  msg-id = "<" id-local "@" id-domain ">"

Simply changing the name of some syntax rule cannot possibly change the
meaning of the document.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 27 Jun 2005 20:03:34 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

On Tue, 28 Jun 2005, Frank Ellermann wrote:
> > a checksum (on the end line), typically CRC16 based.
> 
> More dark and bad memories, and now they tell us that MD5
> is obsolete.

MD5 is still an excellent choice when the concern is accidental
corruption, rather than deliberate human malice.

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Feedback on proposal
Date:  Tue, 28 Jun 2005 00:42:16 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 38
Sender: [email-masked]

Ned Freed wrote:

> an octal protection mask

Yes, now I recall it, user group world, odd was "dangerous".
When my OS/2 box dies I'll proably meet these bits again.

> a checksum (on the end line), typically CRC16 based.

More dark and bad memories, and now they tell us that MD5
is obsolete.

> at the time there were a fair number of systems that used
> national variants of ASCII

Yes, the German variant was very ugly.  I also didn't like
the trigraph stuff in C.

> plenty of format variations, including some really wild
> ones, in the 4-in-5 space.

One I vaguely recall is "FIFfy" (four in five) used for 
binary file transfers in German videotext (like Prestel).

>> I didn't read the old CTE-drafts, did one of them cover
>> the CTE-part of "yenc" for some value of n <= 997 ?
 
> Yes, that's exactly what my draft does (or did).

Then getting an RfC for it is a good idea.  I'm only lost
why it should integrate gzip, compression algorithms come
and go, they never satisfy all users or all applications.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date:  Mon, 27 Jun 2005 23:50:26 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Bruce Lilly wrote:

 [IESG process failure, quoting me:]
> after approving technically incompatible [...]

Maybe in the very same telechat (minutes not yet available).

But I wasn't talking about NNTP -27.  It was about abusing
v=spf1 sender policies for the incompatible "PRA" causing
bogus FAIL as well as bogus PASS results for received mail,
"unprofessional and unethical".  NOT RECOMMENDED and SHOULD
simultaneously.  Broken by design instead of engineering.

> Use msg-id as defined in 2822

No, it's broken, it's too broad for all protocols where it's
used, we need an interoperable subset.  And the maximal set
of syntactically well-formed <message-id>s is already clear.

                      Bye, Frank

Cc: [email-masked]
To: Frank Ellermann <[email-masked]>
Date: Mon, 27 Jun 2005 14:03:30 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

> >> UUENCODE variants, but I've long since lost track of it.
 
> > Maybe http://www.alvestrand.no/ietf/uuencode-no.html ?

I actually think someone took this and generalized it at some point, but I
cannot seem to track down the reference in my email archives and google isn't
being helpful today.

> Nice, thanks, that zoo was fortunately replaced by B64 and
> MIME.  For "yenc" it's the same zoo again, trying to solve
> several problems.  Ned wrote:

> >> Actually, it is clear that uuencode is a CTE.

> The main part of it, binary IN, some text OUT, could be a
> CTE.  But it also had some kind of a Content-Disposition
> (specifying filename), IIRC it also allowed the indication
> of "executable".

Uuencode typically allows the optional specification of a file name and an
octal protection mask. What's actually allowed as a file name varies, as does
what's allowed in the protection field. There are variants that also include a
checksum (on the end line), typically CRC16 based.

But none of this makes UUENCODE ineligable as a CTE. If it did base64 would
also have been ineligable, as the original specification of it allowed for
embeeded comments (and maybe some variably in terminator representation - I no
longer recall). In the case of base64 we simply dropped the comment facility,
had we gone with base64 we would have dealt with the filename, protection, and
other gunk by calling for it to be assigned a fixed value on encode and ignored
on decode.

We elected to go with base64 and not with uuencode for two main reasons: (1)
Uuencode's character repetoire exceeded the minimal safe subset of ASCII, and
at the time there were a fair number of systems that used national variants of
ASCII and had trouble with characters that uuencode uses and (2) All the format
variations were already causing interop problems, and even if we nailed things
down in our specificaiton the temptation to use just existing software would
surely have led to more problems.

Similar issues apply to the 4-in-5 encodings. In particular, there just aren't
enough safe characters in the minimal safe subset of ASCII to do base85. And
there are plenty of format variations, including some really wild ones, in the
4-in-5 space.

In retrospect I kind of wish we'd ignored the minimal safe subset issue and
gone with a 4-in-5 (preferably the one that's built into PostScript since it is
fully and completely specified), since 20% overhead is a lot more attactive
than 33%. But you have to remember that the entire MIME effort started not as
an attempt to send non-text material through email. Rather, the main goal was
to internationalize text email. It is unlikely in the extreme that given the
original purpose of the effort we would have allowed to ignore what was then a
fairly critical factor in the handling of internationalized text.

> I didn't read the old CTE-drafts, did one of them cover the
> CTE-part of "yenc" for some value of n <= 997 ?  Bye, Frank

Yes, that's exactly what my draft does (or did).

				Ned

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: NEW: draft language (Was Re: Feedback on proposal)
Date:  Mon, 27 Jun 2005 22:33:33 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 19
Sender: [email-masked]

Bruce Lilly wrote:
 
> the draft contains about a dozen instances of "this standard"

Okay, that's wrong, minus a case introduced by the boilerplate.

But we're still going for "proposed standard", and saying "our
standard" in a WG discussion is perfectly okay, because we've
to make sure that all works as soon as it is a standard.

Declaring all pre-2231 UAs as "non-conformant" is an odd idea,
normally that's a case for a SHOULD meaning "old UAs have an
excuse, but new UAs need another _good_ excuse if they don't
support 2231".  So why a MUST in this case, jumping from 1036
to MUST 2231 is wrong.  Jumping from s-o-1036 to MUST 2049 is
okay from my POV, but Steve says that even this is too much :-(
 
                      Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Feedback on proposal
Date:  Mon, 27 Jun 2005 22:12:59 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Bruce Lilly wrote:

>> UUENCODE variants, but I've long since lost track of it.
 
> Maybe http://www.alvestrand.no/ietf/uuencode-no.html ?

Nice, thanks, that zoo was fortunately replaced by B64 and
MIME.  For "yenc" it's the same zoo again, trying to solve
several problems.  Ned wrote:

>> Actually, it is clear that uuencode is a CTE.

The main part of it, binary IN, some text OUT, could be a
CTE.  But it also had some kind of a Content-Disposition
(specifying filename), IIRC it also allowed the indication
of "executable".

I didn't read the old CTE-drafts, did one of them cover the
CTE-part of "yenc" for some value of n <= 997 ?  Bye, Frank

To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 27 Jun 2005 13:02:33 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:
> On Mon June 27 2005 14:32, Frank Ellermann wrote:

>> In fact NNTP -27 was approved, 1*248 VCHAR excl. ">" within
>> angle brackets is settled.

> Somebody recently said "The IESG is already at its lowest point
> of unprofessional and unethical behaviour after approving
> technically incompatible [...]"...

The NNTP draft defines the minimum syntactic restrictions on message ID
required to work with NNTP software and explicitly states that additional
syntactic restrictions will be imposed by the relevant article standard.
It is compatible with our proposed message ID definition in that it
doesn't add additional restrictions.  It is not fully compatible with the
RFC 2822 message ID syntax for the reasons previously stated here
(primarily whitespace and multiple representations for the same message
ID), since making it so (even apart from whether that would be a good
idea) would have been significantly incompatible with existing NNTP
practice and hence outside our WG charter.

Please note that NNTP is not used exclusively with Usenet messages, and
was not written limiting itself to Usenet messages.

To: [email-masked]
Subject: Re: Definition of "private" (Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts)
From: Graham Drabble <[email-masked]>
Date: Mon, 27 Jun 2005 20:56:16 +0100
Organization: Home
Sender: [email-masked]

On 23 Jun 2005 "Charles Lindsey" <[email-masked]> wrote in
news:[email-masked]: 

> But, "general" apart, I think current practice on Usenet is that
> single-component groups are not supposed to be there, and my
> wording was intended to reflect that. But, as I said, my wording
> is up for debate. 

The more I think about this the less I'm convinced that we need to say 
anything as part of the standard (we might want to look at this again 
come USEAGE). Single component groups and groups with all numeric 
components can only harm servers that choose to carry them, if they 
simply turn up in a transited article or in a control message which is 
not actioned they aren't a problem. It can therefore be left up to each 
administrator to decide based on local policy whether to add these 
groups.

-- 
Graham Drabble
http://users.ox.ac.uk/~sjoh1646/

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Mon, 27 Jun 2005 15:52:16 -0400
Cc: [email-masked]
Sender: [email-masked]

> In fact NNTP -27 was approved, 1*248 VCHAR excl. ">" within
> angle brackets is settled.

Somebody recently said "The IESG is already at its lowest point
of unprofessional and unethical behaviour after approving
technically incompatible [...]"...

> > The (now past) WG co-chairs clearly indicated our direction:
> > http://www.imc.org/ietf-usefor/2003/Mar/0305.html
> 
> That article says that USEFOR produces its fourth and final
> I18N RFC in November 2003.

That was the suggested ("here's what we suggest as a schedule")
for the timeline.
 
> > with our work products to deal only with articles that
> > conform to 2822 and MIME syntax.
> 
> Sure, "<" <id-local> "@" <id-domain> ">" or Charles' version
> are a proper subset of what 2822 / MIME / NNTP say.

Fine, Use msg-id as defined in 2822 and add a note saying what
other protocol imposes limitations, what those limitations are,
etc. as suggested by Mark.

> > N.B. not to *redefine* 2822 or MIME syntax.
> 
> Now what's that, avoiding <id-left> "@" <id-right> is not
> good enough,

Not "not good enough", a redefinition is harmful (as detailed
earlier).

> do you maybe want a new term for a <msg-id> ? 
> How about the good old term <message-id> as in s-o-1036 ?

Where would you use it?  Message-ID and References fields are defined
in RFC 2822, Likewise for Received and In-Reply-To.  Supersedes is
defined in RFC 2156.  Content-ID is defined in RFC 2045.  Don't mess
with those definitions -- add restrictions if there's good reason.
 
> True, but updating only its <msg-id> syntax is an option after
> this WG is closed.

Meanwhile we should use the definition as it is, not as one or
two people would like to see it.
 
> > MIME,
> 
> Except from a clear "don't mess with the boundary parameter"
> plus some more elaborated security considerations I don't
> see what you'd want to update there - okay, fix the errata,
> use ABNF while you're at it, and avoid any prose in syntax.

*I* don;t want to change it -- you apparently want to screw up
the definition of the Content-ID field.
 
> > You have yet (see above) to indicate a specific technical
> > problem.
> 
> "Does not work with NNTP" is good enough.

What is the specific issue with RFC 977 (NNTP)?

> But just for fun 
> here's another obvious reason found in RfC 2046, please read
> also the surrounding text:
> 
> | Aside from these conventions, any use of the control
> | characters or DEL in a body must either occur
> |
> | (1)   because a subtype of text other than "plain"
> |       specifically assigns some additional meaning, or
> [... (private convention) ...]

That's for the charset parameter of a text media type and pertains
to the text media body content.  We're discussing header field
components.
 
> Many NetNews articles are text/plain US ASCII and routinely
> mention some <message-id>s in the body.

If it hurts, don't do it.  I have no objection to a note (presumably
in USEAGE) pointing out the problem (note that body text can be
encoded using e.g. quoted-printable).  But that would amount to
fretting about nothing (see below).

> That won't work for 
> NO-WS-CTL (excl. the conventional HT and FF), it's illegal.

See quoted-printable.  All of which is academic because while "dtext"
syntax allows such things to accommodate future formats, STD 3 and
the various IPv6 schemes don't, so it won't happen until and unless
some future scheme provides explicitly for it.
 
> Deja-vu, IIRC we already discussed this.
> 
> > Can you accurately predict what future versions of IP
> > address literals will look like?
> 
> Yes, it will be a subset of the STD 66 <IPvFuture> syntax.

That's one opinion; RFC 3986 pertains to URIs, which is irrelevant
to message format header field content except where that content
specifically specifies a URI (which is not the case under discussion).
It's also hard to take seriously a document that has changed the
definition of "reserved" characters as drastically and frequently
as the URI specification has.

> > there isn't agreement for IPv6 (RFC 2821 "IPv6:" tag, for
> > example).
> 
> Sorry, I fail to see any NO-WS-CTL in the string "IPv6:", and
> otherwise it's the same as in STD 66 <IPv6address>.

The "IPv6:" tag is a crucial difference.

Date: Mon, 27 Jun 2005 18:18:29 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

--On 27. juni 2005 09:17 -0400 Bruce Lilly <[email-masked]> wrote:

> On Mon June 27 2005 04:56, Harald Tveit Alvestrand wrote:
>>
>> --On 26. juni 2005 18:47 -0400 Bruce Lilly <[email-masked]> wrote:
>
>> > The purpose of adding MIME is to acknowledge that MIME RFCs also use
>> > the terms "header" (as in MIME-part[ -]header) and "header field".
>>
>> Uses. Does not define.
>
> I believe you are mistaken.  RFC 2045:

Defines "MIME-part-headers", not "header field". The sentence you quote 
uses the term "header field", it does not define it.
 The paragraph I'm asking to have inserted in USEFOR talks about the 
definition of the term "header field".

Since nobody else appears interested in this debate, I'll stop responding 
now.

>     (2)   In a MIME body part header within a multipart
>           construct.
>
>    The formal definition of these header fields is as follows:
>
> [...]
>      MIME-part-headers := entity-headers
>                           [ fields ]
>                           ; Any field not beginning with
>                           ; "content-" can have no defined
>                           ; meaning and may be ignored.
>                           ; The ordering of the header
>                           ; fields implied by this BNF
>                           ; definition should be ignored.
>
> That says it is a formal definition and is not covered by [2]822.
>
>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date:  Mon, 27 Jun 2005 21:42:21 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 30
Sender: [email-masked]

Kai Henningsen wrote:

> the idea goes back at least to RFC 624 (obsoleted by 733,

Okay, 733 says 724, let's see:

| The uniqueness of the message identifier is guaranteed by
| each host.

And that <host-name> is the part of an <host-indicator>,
which is either "at" or "@" followed by the name, an <atom>
or a <decimal-host-address>.

> we're debating are implementation techniques and problems,
> not the fundamental concept.

I'm not convinced that that's the case, Charles repeatedly
said that it's no domain (i.e. <host-name> in this old RFC).

In <[email-masked]> (see ?) you said:

> I'm for id-right.

And that doesn't reflect the fundamental concept <host-name>.

<id-right> could be "maybe take some random number + hash +
timestamp, don't forget to add an '@' somewhere in the middle".

                           Bye, Frank

To: [email-masked]
Subject: Re: Feedback on proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 27 Jun 2005 12:34:35 -0700
Sender: [email-masked]

Ken Murchison <[email-masked]> writes:
> Ned Freed wrote:

>> It was the last word from me because basically all I got for my trouble
>> was a bunch of (mostly private) hate mail and precious little public
>> (or private) support for the idea.

> Just out of curiosity, did you ever look at Jeremy Nixon's work on this?

> http://www.exit109.com/~jeremy/news/binaries/

> It doesn't include compression, but it does seem to retain backwards
> compatibility.

I don't really see a lot of point in the compression aspect of this,
actually.  At least for the Usenet problem, the data is already
compressed, and converting to base64 and then compressing seems like a lot
of hassle.

The real point of yEnc is that it specifies the minimum possible munging
of binary data to allow it to be transferred through Usenet.  (And
probably most mail servers, but that I haven't investigated at all.)  As a
result, it saves a lot of encoding space without requiring the harder work
of switching to a full binary transport.

One of the reasons why standardizing that particular good idea has been so
hard is that the author of that very solid idea then went on to add a
bunch of uuencode-inspired cruft that reinvents MIME multiparts poorly
(albeit with a few good ideas that could usefully be added to MIME
multiparts), and he was insisting on dragging along all of that cruft in
any standardization effort.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Ned Freed <[email-masked]>
Subject: Re: Feedback on proposal
Date: Mon, 27 Jun 2005 15:19:25 -0400
Cc: Frank Ellermann <[email-masked]>, [email-masked]
Sender: [email-masked]

On Mon June 27 2005 14:23, Ned Freed wrote:

> There used to be a document floating around that described some, but not all,
> of the UUENCODE variants, but I've long since lost track of it.

Maybe http://www.alvestrand.no/ietf/uuencode-no.html ? 

Date: Mon, 27 Jun 2005 15:17:47 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Ned Freed <[email-masked]>
CC: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Ned Freed wrote:

> It was the last word from me because basically all I got for my trouble was a
> bunch of (mostly private) hate mail and precious little public (or private)
> support for the idea.

Ned,

Just out of curiosity, did you ever look at Jeremy Nixon's work on this?

http://www.exit109.com/~jeremy/news/binaries/

It doesn't include compression, but it does seem to retain backwards 
compatibility.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: Feedback on proposal
Date: Mon, 27 Jun 2005 15:13:54 -0400
Cc: [email-masked]
Sender: [email-masked]

On Mon June 27 2005 13:12, Frank Ellermann wrote:
> 
> Bruce Lilly wrote:
> 
> >> AFAIK yenc is 8bit minus NUL, CR, LF, and optional tricks
> >> to avoid trailing SP or HT.
> 
> > The key here is "AFAIK".
[...]
> > The problem is that "n" (from your other message) isn't
> > specified to be constrained to 998 or less
> 
> For the mentioned "typical values" 128 or 256 I don't see any
> problem,

The problem is that n = 999, n= 1000, ... n = 99999999999999, ...
are permitted.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: NEW: draft language (Was Re: Feedback on proposal)
Date: Mon, 27 Jun 2005 15:06:46 -0400
Cc: [email-masked]
Sender: [email-masked]

On Mon June 27 2005 13:12, Frank Ellermann wrote:
> 
> Bruce Lilly wrote:
> > "conformance with our" specification (it's not a standard).
> 
> We're going for "proposed standard", not "informational" or
> "experimental".  Maybe BCP for USEAGE.  Modulo "with the iESG
> you never know".

"Not all RFCs are standards".  More important "The Internet-Draft should
neither state nor imply that it has any standards status; to do so
conflicts with the role of the RFC Editor and the IESG." (1id-guidelines).
I think I mentioned this before; the draft(s) need to be checked
carefully by the Editors (this should already have been done in
accordance with the guidelines...).

Specifically, the draft contains about a dozen instances of "this standard"
(one says "this present standard"), there's a reference to "other standards"
(implying that the draft is a standard), a reference to "standard
User-Agent", etc.   The only occurrence that shouldn't be changed is the
one in the IPR boilerplate.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date:  Mon, 27 Jun 2005 20:32:36 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 95
Sender: [email-masked]

Bruce Lilly wrote:

>> | The Group shall also aid and/or oversee the production
>> | of other Usenet related Internet Drafts and Standards.

>> That covers NNTP and the two news-nntp-uri drafts.
> That's covered by a separate WG.

In fact NNTP -27 was approved, 1*248 VCHAR excl. ">" within
angle brackets is settled.

> The (now past) WG co-chairs clearly indicated our direction:
> http://www.imc.org/ietf-usefor/2003/Mar/0305.html

That article says that USEFOR produces its fourth and final
I18N RFC in November 2003.

> with our work products to deal only with articles that
> conform to 2822 and MIME syntax.

Sure, "<" <id-local> "@" <id-domain> ">" or Charles' version
are a proper subset of what 2822 / MIME / NNTP say.

> N.B. not to *redefine* 2822 or MIME syntax.

Now what's that, avoiding <id-left> "@" <id-right> is not
good enough, do you maybe want a new term for a <msg-id> ?
How about the good old term <message-id> as in s-o-1036 ?

> So far nobody has made a convincing case about no-WS-CTLs;
> we know that you don't like them, but you haven't given a
> technical argument against their use.

You probably missed one or two of the several hundreds (?)
of articles in the various "msg-id" threads.  That point
is moot, the proposed standard NNTP -27 has VCHAR excl. ">",
and it's not exactly a new concept (1036, 1738, etc.).

> There is a separate effort to update URIs.  Not our job

Obviously Charles and I volunteered after Paul said that he's
not more interested.  And so it's "our" job to educate these
volunteers about our ideas for a <message-id>, it there are
any differences from 1036 / 1738 (hint: yes).

> we don't have the time or temperament to rewrite 2822

True, but updating only its <msg-id> syntax is an option after
this WG is closed.

> MIME,

Except from a clear "don't mess with the boundary parameter"
plus some more elaborated security considerations I don't
see what you'd want to update there - okay, fix the errata,
use ABNF while you're at it, and avoid any prose in syntax.

> You have yet (see above) to indicate a specific technical
> problem.

"Does not work with NNTP" is good enough.  But just for fun
here's another obvious reason found in RfC 2046, please read
also the surrounding text:

| Aside from these conventions, any use of the control
| characters or DEL in a body must either occur
|
| (1)   because a subtype of text other than "plain"
|       specifically assigns some additional meaning, or
[... (private convention) ...]

Many NetNews articles are text/plain US ASCII and routinely
mention some <message-id>s in the body.  That won't work for
NO-WS-CTL (excl. the conventional HT and FF), it's illegal.

Deja-vu, IIRC we already discussed this.

> Can you accurately predict what future versions of IP
> address literals will look like?

Yes, it will be a subset of the STD 66 <IPvFuture> syntax.

> there isn't agreement for IPv6 (RFC 2821 "IPv6:" tag, for
> example).

Sorry, I fail to see any NO-WS-CTL in the string "IPv6:", and
otherwise it's the same as in STD 66 <IPv6address>.  Another
deja-vu, at least Charles and I looked into all these details.

Until Alexey said that we better don't try to define all the
details of <id-literal> or <no-fold-literal> resp., just the
set of allowed characters adding \\, \[, and \] just in case.

                          Bye, Frank

Cc: [email-masked]
To: Frank Ellermann <[email-masked]>
Date: Mon, 27 Jun 2005 11:23:22 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

> > The problem with both is that there is no satisfactory
> > indication of the media type.

> Yes, isn't that the idea of application/octet-stream ?
> Some binary crap.  Actually the same case as UUE in a
> plain text part, the only difference, it's not 7bit.

> > YEnc is clearly a transfer encoding, not a media type.

> If that's "clear" then it would be also "clear" for UUE.

Actually, it is clear that uuencode is a CTE. It just isn't a very good one -
there are numerous variations, some compatible, others not. Issues include line
length, the use or non-use of spaces versus tildes, how incomplete lines are
handled, and the format ofd both the begin and end lines.

Messages with a CTE of X-UUENCODE were fairly common at one point, as a matter
of fact.

Similar issues surround the class of 4-in-5 encodings: btoa, base85, and so on.

There used to be a document floating around that described some, but not all,
of the UUENCODE variants, but I've long since lost track of it.

				Ned

Cc: [email-masked]
To: Frank Ellermann <[email-masked]>
Date: Mon, 27 Jun 2005 11:11:23 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

> >> AFAIK yenc is 8bit minus NUL, CR, LF, and optional tricks
> >> to avoid trailing SP or HT.

> > The key here is "AFAIK".

> You can check it at <http://www.yenc.org> - this page starts:

> | 02.April.2003:
> | A new encoding has been specified by Ned Fried.
> | gZip-8Bit is an extension to yEnc with MIME and integrated
> | compression.

> | The next round in standardization of Binary Usenet messages
> | has been started.

> I've no idea why that's apparently the last word about yenc.

It was the last word from me because basically all I got for my trouble was a
bunch of (mostly private) hate mail and precious little public (or private)
support for the idea.

				Ned

To: [email-masked]
Subject: Re: "System" (Re: NEW: USEFOR -04 3.2.11 Xref vs. transport)
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 27 Jun 2005 11:00:05 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:

> o there is some purpose in one "serving agent" sending an Xref field to
>   another "serving agent"?

As I believe I previously mentioned, there is indeed some purpose in one
serving agent sending an Xref field to another serving agent, namely Xref
slaving.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: "System" (Re: NEW: USEFOR -04 3.2.11 Xref vs. transport)
Date: Mon, 27 Jun 2005 13:26:42 -0400
Cc: [email-masked]
Sender: [email-masked]

On Mon June 27 2005 11:21, Harald Tveit Alvestrand wrote:
> 
> --On 27. juni 2005 09:31 -0400 Bruce Lilly <[email-masked]> wrote:

> >From the charter:
> 
> > 1. Produce an Internet Draft (or series of drafts) that describes the
> >   core standards for a Usenet article and the features that all Usenet
> >   software should take account of.

Something seems to be wrong with the attribution...
 
> If you want to argue that Usenet readers are not Usenet software

I don't.

> Article formats passed in standardized protocols between a client ("posting 
> agent" or "reading agent") and a server ("injecting agent" and "serving 
> agent") are within scope.

That's nice.  Are you suggesting that:
o a posting agent should generate and transmit an Xref field?
o an injecting agent should generate and transmit an Xref field?
o there is some purpose in one "serving agent" sending an Xref field to
  another "serving agent"?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Feedback on proposal
Date:  Mon, 27 Jun 2005 19:26:00 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 18
Sender: [email-masked]

Bruce Lilly wrote:
 
> The problem with both is that there is no satisfactory
> indication of the media type.

Yes, isn't that the idea of application/octet-stream ?
Some binary crap.  Actually the same case as UUE in a
plain text part, the only difference, it's not 7bit.

> YEnc is clearly a transfer encoding, not a media type.

If that's "clear" then it would be also "clear" for UUE.
The latter isn't the case, your premise is utter dubious.

Both are not designed to fit nicely into a MIME framework,
no surprise for UUE.  But the framework is still capable
to transport this crap as is without causing havoc.  Bye

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Feedback on proposal
Date:  Mon, 27 Jun 2005 19:12:30 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 57
Sender: [email-masked]

Bruce Lilly wrote:

>> AFAIK yenc is 8bit minus NUL, CR, LF, and optional tricks
>> to avoid trailing SP or HT.

> The key here is "AFAIK".

You can check it at <http://www.yenc.org> - this page starts:

| 02.April.2003:
| A new encoding has been specified by Ned Fried.
| gZip-8Bit is an extension to yEnc with MIME and integrated
| compression.

| The next round in standardization of Binary Usenet messages
| has been started.

I've no idea why that's apparently the last word about yenc.

> The problem is that "n" (from your other message) isn't
> specified to be constrained to 998 or less

For the mentioned "typical values" 128 or 256 I don't see any
problem, apparently any "n" up to 997 goes (they don't allow
to split an escaped NUL, HT, CR, LF, SP, or "=" by CRLF, and
so they need "n+1" for such cases),

> not formally specified in a stable specification suitable as
> the basis for an IETF encoding

Yes, I'm guessing what it means.  OTOH this part of yenc is
simple and clear enough to get it right for NNTP.  But when
they start with =ybegin, multiple parts, and Subject tags I'm
not more interested, after all I'm not planning to implement
it.  JFTR, I also don't plan to implement message/partial.

But I have an application claiming to create or at lest read
yenc (among others like UUE and XXE).  Not message/partial -
I've never seen any message/partial, whatever that means, it
could be because I rarely look into binary groups, not once
for at least 20 months.

> the proponents of yEnc seem to be unwilling to standardize
> the specification in the IETF (as noted by several people)

<shrug />  If Steve really wants it he's free to try it, my
interest is limited to the question of our various MUSTs, and
to a certain degree to MIME itself.  In your 822-discussion
limited to the one case _without_ gzip.

> "conformance with our" specification (it's not a standard).

We're going for "proposed standard", not "informational" or
"experimental".  Maybe BCP for USEAGE.  Modulo "with the iESG
you never know".
                 Bye, Frank

Date: Mon, 27 Jun 2005 12:55:42 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

On Mon, 27 Jun 2005, Russ Allbery wrote:
> > ...Most
> > plain-text articles on Usenet today do not have MIME-Version headers...
> 
> I haven't run more complete statistics, but assuming that my randomly
> chosen text newsgroup is indeed a decent random sample, this is
> interestingly not the case.  Around 60% of articles had a MIME-Version
> header.

I ran a quick check in a couple of groups I follow regularly, and got
numbers broadly agreeing with that.  I'm surprised; I hadn't thought it
was that high yet.

                                                          Henry Spencer
                                                       [email-masked]

To: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 27 Jun 2005 09:47:27 -0700
Sender: [email-masked]

Henry Spencer <[email-masked]> writes:

> As you've been told repeatedly, just because a client is MIME-capable
> doesn't mean that it insists on handling only MIME messages.  Most
> plain-text articles on Usenet today do not have MIME-Version headers.  A
> Usenet client which ignored any article lacking a MIME-Version header
> would be useless.

I haven't run more complete statistics, but assuming that my randomly
chosen text newsgroup is indeed a decent random sample, this is
interestingly not the case.  Around 60% of articles had a MIME-Version
header.

I'd have to do more work to see if this applies across the rest of the
text Usenet.

(The rest of your point, of course, I agree with completely.)

Date: Mon, 27 Jun 2005 12:33:14 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

On Mon, 27 Jun 2005, Steve Walker wrote:
> The percentage, which is greater than 65% and growing, of 
> non-conformance is too large to just be ignored.  It is not in 
> the best interest of Usenet to write a standard that software 
> client developers can not conform to even if they would like to.

All other issues aside, there is simply no way we can standardize the yenc
stuff without cooperation and participation from the community that uses
it... and as others have noted, in the past that community has been
conspicuously uninterested. 

This isn't "65% and growing" of Usenet; this is, for all practical
purposes, a separate network which happens to use a few of the same
protocols and some of the same servers. 

> Is this group trying to writing something that standardizes on 
> existing practices or is it trying to re-invent a whole new 
> protocol.

We're trying to standardize existing practices, specifically the ones
derived from the earlier quasi-standards for news, notably RFC 1036. 
There is a limit to what we can include under that umbrella; the yenc
people have, to put it bluntly, gone their own way with complete disregard
for standards.  There is every reason to believe that they will continue
to do so, and in that context, contorting standards to accommodate them
appears rather pointless. 

> The non-binary focused clients would support mime because it 
> works well for text and the binary focused clients most likely 
> would not implement a full MIME parser.  The mime clients should 
> be looking for the header MIME-Version anyway and they could 
> ignore articles without it.

As you've been told repeatedly, just because a client is MIME-capable
doesn't mean that it insists on handling only MIME messages.  Most
plain-text articles on Usenet today do not have MIME-Version headers.
A Usenet client which ignored any article lacking a MIME-Version header
would be useless. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Feedback on proposal
Date:  Mon, 27 Jun 2005 18:28:09 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 35
Sender: [email-masked]

Henry Spencer wrote:

> How many times do I have to say "non-standard"? :-)

Until you've submitted s-o-1036 as FYI... <gd&r>

Joke off, from my POV non-standard is the gap between
SHOULD and MUST.  But violating a MUST is not only
"non-standard", it's broken and known to be harmful
in some cases.  Not all cases, that would be OBVIOUS,
and OBVIOUS is a 2119-keyword that MUST NOT be used -
consequently it's not mentioned in 2119.

Unfortunately Bruce's keywords are slightly different,
SHOULD is a "mere recommendation", MUST is important,
and violating a MUST is a real problem if the brown
shirts get you.  Otherwise it's probably "let's see
what happens".

No surprise that we have serious difficulties to come
to a common understanding of what "UAs MUST conform
to RfC 2049 and MUST support RfC 2231" actually means.

 From my POV it's perfectly sane to reject articles if
they use 8bit without MIME.  It's the idea of saying
MUST instead of SHOULD.

 From your or Bruce's POV this MUST apparently works
like a SHOULD in my terminology, if it's broken you
own the pieces, no other consequences.  This includes
many fine points like "no justification to reject or
clean such articles", "no valid reason for an abuse
report", etc.
                       Bye, Frank

To: [email-masked]
Subject: Re: "System" (Re: NEW: USEFOR -04 3.2.11 Xref vs. transport)
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 27 Jun 2005 09:27:35 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:

> I did quote the corresponding text from RFC 1036 at the very beginning of
> the discussion.  I repeat it for convenience:

>     This is only of value to the local system, so it should not be
>     transmitted.

> Now many of us would agree that 1036 is somewhat vague, but that specific
> text in 1036 obviously hasn't prevented a particular (NNTP) server from
> communicating cross-reference information to its clients, nor would the
> suggested text for usefor.

Mostly because everyone ignored that part of RFC 1036.

To: [email-masked]
Subject: Re: Feedback on proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 27 Jun 2005 09:25:36 -0700
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:
> Russ Allbery wrote:

>> and yEnc breaks the line length requirement.

> I'm not sure, seen in file YENC_1_3.TXT (2002-07-22 on my box):

> | To facilitate transmission via existing standard protocols
> | (most notably NNTP), carriage return/linefeed pairs should be
> | written to the output stream after every n characters, where
> | n is the desired line length.  Typical values for n are 128
> | and 256.

> That's smaller than 998.  So it should be okay for 8bit MIME,
> although anything else is more like UUE than MIME.  Bye, Frank

Oh, hm, that's interesting.  I hadn't realized that was in there (maybe
it's a change from the original protocol?).  That would seem to indicate
you could get away with content-type application/x-yenc, CTE: 8bit if you
really wanted to (and as I recall, yEnc includes a variety of other data
besides the actual message, in an attempt to -- poorly -- duplicate MIME
multipart features, so it's arguably not purely a CTE either).

Date: Mon, 27 Jun 2005 12:19:35 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Steve Walker <[email-masked]> wrote:

> The percentage, which is greater than 65% and growing, of 
> non-conformance is too large to just be ignored.

Where are you getting that number?  Are you including plaintext
articles that have only ASCII characters and lack MIME headers?  Those
_are_ valid MIME.

> Is this group trying to writing something that standardizes on 
> existing practices or is it trying to re-invent a whole new 
> protocol.

It's trying to standardize those existing practices which can be
standardized.  Apparently yEnc can't be; at least, the efforts so far
have failed, and this is the wrong group to work on standardizing yEnc
anyway, for any number of reasons.

If you want us to point to a yEnc standard in addition to the MIME
standard, show us the one to point to.

Seth

Date: Mon, 27 Jun 2005 18:12:54 +0200
From: Sebastian Brocks <[email-masked]>
To: Steve Walker <[email-masked]>
CC: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Steve Walker schrieb:
> The mime clients should 
> be looking for the header MIME-Version anyway and they could 
> ignore articles without it.

No they could not ignore articles without it, those articles are just as
valid as MIME articles if they do not contain 8 Bit Characters.

Date: Mon, 27 Jun 2005 17:21:57 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: "System" (Re: NEW: USEFOR -04 3.2.11 Xref vs. transport)
Sender: [email-masked]

--On 27. juni 2005 09:31 -0400 Bruce Lilly <[email-masked]> wrote:

> Now many of us would agree that 1036 is somewhat vague, but that specific
> text in 1036 obviously hasn't prevented a particular (NNTP) server from
> communicating cross-reference information to its clients, nor would the
> suggested text for usefor.  Indeed, such issues are outside of the scope
> of our work.  The scope of our work does include transmission of message
> (article) content from one "site" (used but not defined in the draft) or
> "server" (ditto) to another.

Now, that's a surprising idea....

>From the charter:

> 1. Produce an Internet Draft (or series of drafts) that describes the
>   core standards for a Usenet article and the features that all Usenet
>   software should take account of.

If you want to argue that Usenet readers are not Usenet software, and 
therefore what goes on between the reader and the server (or "agent" as the 
I'm sure that it would make our work simpler, but in this case, I'm pretty 
sure you're not reflecting a consensus view.

Article formats passed in standardized protocols between a client ("posting 
agent" or "reading agent") and a server ("injecting agent" and "serving 
agent") are within scope.

                        Harald

Date: Mon, 27 Jun 2005 10:52:29 -0500
From: Steve Walker <[email-masked]>
To: [email-masked]
CC: Frank Ellermann <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

Bruce Lilly wrote:
> No, there are no IETF brown shirts forcing conformance at gunpoint.
> No matter what we say (given the constraint that what we say has to
> pass IESG muster), there will be non-conforming software.  Our task
> is to specify format, protocol, and best practice for those who wish
> to build conforming implementations.

The percentage, which is greater than 65% and growing, of 
non-conformance is too large to just be ignored.  It is not in 
the best interest of Usenet to write a standard that software 
client developers can not conform to even if they would like to.

Is this group trying to writing something that standardizes on 
existing practices or is it trying to re-invent a whole new 
protocol.  In that case, a proposal for Usenet-Version: 2.0 is in 
order and you would not have to keep backward compatibility.

> 2.3  MIME Conformance
> 
>    User agents MUST meet the definition of MIME-conformance in [RFC2049]
>    and MUST also support [RFC2231].  This level of MIME Conformance
>    provides support for internationalization and multimedia in message
>    bodies ([RFC2045], [RFC2046], [RFC2231]), and support for
>    internationalization of headers ([RFC2047], [RFC2231]).  Note that
>    [Errata] currently exist for [RFC2046] and [RFC2231].

Would changing the above to the following harm anything?

User agents SHOULD meet the definition of MIME-conformance in 
[RFC2049] and SHOULD also support [RFC2231].

The non-binary focused clients would support mime because it 
works well for text and the binary focused clients most likely 
would not implement a full MIME parser.  The mime clients should 
be looking for the header MIME-Version anyway and they could 
ignore articles without it.

Steve Walker
NNTPServer.com

Date: Mon, 27 Jun 2005 11:08:24 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

On Mon, 27 Jun 2005, Frank Ellermann wrote:
> > The passages you note cover only articles containing
> > non-ASCII text.
> 
> Yes, AFAIK yenc is 8bit minus...

Correct.  I was speaking more generally, not of the specific question of
reconciling yenc with MIME. 

> >> how should it post this stuff ?
> > Non-standardly.  Here, I was discussing software support for
> > conventions, not standards conformance
> 
> That's not allowed in usefor-0?, all UAs MUST follow RfC 2049...

How many times do I have to say "non-standard"? :-)  Here I was talking
about what is technologically possible, not what is standard-compliant. 

> > Clients can have non-standard modes as well as standard ones
> 
> Then they'd need a SHOULD instead of a MUST in usefor to _post_
> such articles.

If they're in a non-standard mode, what's in Usefor is irrelevant. 
Standards cannot dictate what happens when the standard is not
followed.

                                                          Henry Spencer
                                                       [email-masked]

Cc: Frank Ellermann <[email-masked]>, [email-masked]
To: Bruce Lilly <[email-masked]>
Date: Mon, 27 Jun 2005 07:42:20 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

> On Mon June 27 2005 04:50, Frank Ellermann wrote:

> > Yes, AFAIK yenc is 8bit minus NUL, CR, LF, and optional tricks
> > to avoid trailing SP or HT.

> The key here is "AFAIK". The problem is that "n" (from your other
> message) isn't specified to be constrained to 998 or less, and in
> any event is not formally specified in a stable specification
> suitable as the basis for an IETF encoding.  That has led to
> interoperability problems (as noted by Russ), and the proponents
> of yEnc seem to be unwilling to standardize the specification in
> the IETF (as noted by several people).

Bingo. And in this discussion we now have a proposal to identify this stuff
with either 'Content-type: raw/subtype' or 'NNTP-Format: raw/subtype' label and
a refusal to consider using a CTE. The problems with both are obvious and don't
bear repeating.

I give up. It is clear that there's no real interest in moving forward on this
in a way that's compatible with existing standards. So, if any proposals for
additional CTEs move forward, they won't do so as part of an attempt to bridge
this gap in usenet, but rather based solely on their own merit.

				Ned

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: "System" (Re: NEW: USEFOR -04 3.2.11 Xref vs. transport)
Date: Mon, 27 Jun 2005 09:31:14 -0400
Cc: [email-masked]
Sender: [email-masked]

On Mon June 27 2005 05:07, Harald Tveit Alvestrand wrote:

> usefor-04 does not contain a definition of "system".

OK.
 
> In fact the word is used in 2 different senses: "a system" (something that 
> passes articles) and "the news system".
> 
> Unusually for you, you're using controversial terms without defining them 
> first.

I did quote the corresponding text from RFC 1036 at the very beginning of
the discussion.  I repeat it for convenience:

    This is only of value to the local system, so it should not be
    transmitted.

Now many of us would agree that 1036 is somewhat vague, but that specific
text in 1036 obviously hasn't prevented a particular (NNTP) server from
communicating cross-reference information to its clients, nor would the
suggested text for usefor.  Indeed, such issues are outside of the scope
of our work.  The scope of our work does include transmission of message
(article) content from one "site" (used but not defined in the draft) or
"server" (ditto) to another.  Given the context of the discussion (viz.
Xref field), it should be clear that the issue is related to the field
body component
   server-name     =  path-identity

So it appears that the usefor draft does not contain a suitably defined
term.  That's another new issue.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1043 "header": Suggested resolution
Date: Mon, 27 Jun 2005 09:17:14 -0400
Cc: [email-masked]
Sender: [email-masked]

On Mon June 27 2005 04:56, Harald Tveit Alvestrand wrote:
> 
> --On 26. juni 2005 18:47 -0400 Bruce Lilly <[email-masked]> wrote:

> > The purpose of adding MIME is to acknowledge that MIME RFCs also use the
> > terms "header" (as in MIME-part[ -]header) and "header field".
> 
> Uses. Does not define.

I believe you are mistaken.  RFC 2045:

    (2)   In a MIME body part header within a multipart
          construct.

   The formal definition of these header fields is as follows:

[...]
     MIME-part-headers := entity-headers
                          [ fields ]
                          ; Any field not beginning with
                          ; "content-" can have no defined
                          ; meaning and may be ignored.
                          ; The ordering of the header
                          ; fields implied by this BNF
                          ; definition should be ignored.

That says it is a formal definition and is not covered by [2]822.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Mon, 27 Jun 2005 09:13:08 -0400
Cc: [email-masked]
Sender: [email-masked]

On Sun June 26 2005 12:47, Frank Ellermann wrote:
> 
> Bruce Lilly wrote:
>  
> > The WG charter gives no authorization to modify RFC 2822.

> [...]
> | The Group shall also aid and/or oversee the production of
> | other Usenet related Internet Drafts and Standards.
> 
> That covers NNTP and the two news-nntp-uri drafts.

That's covered by a separate WG.

Note also that the charter hasn't been substantively updated in
years, even after a specific call to recharter (which resulted
in the addition of a note at the top claiming that "a charter
rewrite/update is underway" and some milestones (including
Apr 05 recharter or conclude) at the bottom.

The (now past) WG co-chairs clearly indicated our direction:
http://www.imc.org/ietf-usefor/2003/Mar/0305.html
with our work products to deal only with articles that conform
to 2822 and MIME syntax.  N.B. not to *redefine* 2822 or MIME
syntax. 

> > o CID and MID URIs (RFC 2392) seem to work fine.
> 
> That's essentially a percent-encoded RfC 822 <addr-spec>, and
> we already know that we don't want WSP, NO-WS-CTL, or comments
> in a <msg-id>.  Not one of the RfC 2392 examples discusses or
> uses such constructs, it's about %25 for % etc. in URLs based
> on RfC 1738.

1. 2822 obsoletes 822 and forbids CFWS.  So far nobody has made a
   convincing case about no-WS-CTLs; we know that you don't like
   them, but you haven't given a technical argument against their
   use.
2. There is a separate effort to update URIs.  Not our job, and
   we don't have the time or temperament to rewrite 2822, MIME,
   and RFC 2392; we way behind schedule on getting the first of
   two Standards Track documents done.

> > o there are mechanisms other than syntax changes (e.g.
> >   implementation notes) to address specific issues.
> 
> If we don't get the <msg-id> [...] syntax right
> this WG should be dissolved.  Intentionally hiding this syntax
> problem in prose is cowardly and no option.  

1. You have yet (see above) to indicate a specific technical problem.
2. msg-id (and date-time and zone) are defined in RFC 2822 and are
   used in a number of standardized fields (Message-ID, References,
   In-Reply-To, and Superseded for msg-id, Date, Received, Resent-Date
   for date-time and zone).  Conflicting syntax definitions are
   unsatisfactory:
   a. They are a problem for implementers of combined mail/news UAs
   b. They are a problem for implementers of Standards Track protocols
      that handle messages of all sorts (e.g. IMAP)
   c. Conflicting syntax of field components (msg-id, date-time, zone)
      affect fields which this WG has no authority over (i.e. those
      mentioned above
   d. It affects other applications based on the Internet Message
      Format (voice mail, fax, EDI, etc.)
   e. It affects other Standards Track protocols based on the field
      components defined in [2]822 (e.g. MIME Content-ID, HTTP, SIP). 

We can, without causing conflict and remaining within the bounds
established in "the plan of action" provide suitable implementation
notes adding specific restrictions based on technical issues as suggested
in detail by Mark Crispin.  We are also free to define whatever separate
syntax we desire for news-specific fields such as Injection-Date.

> | A word about non-whitespace controls, since there seems to
> | be some confusion: aside from appearance in qtext and ctext,
> | they also appear in the 2822 grammar for dtext.  The reason
> | for that as I understand it (private correspondence with
> | Pete Resnick) is that at the time, IDN had not been fleshed
> | out, and the provision is to accommodate possible schemes.
> | Of course at the present time, a domain name, including IDN,
> | consists solely of letters, digits, and hyphens plus
> | separating dots.  So while the syntax has provision for
> | NO-WS-CTLS in dtext, those won't actually occur because of
> | restrictions in other specifications (notably RFCs 1034, 1035).
> 
> Your communication with the [MESSFOR] author is not good enough
> for a future <id-domain>.

Can you accurately predict what future versions of IP address
literals will look like?  Heck, there isn't agreement for
IPv6 (RFC 2821 "IPv6:" tag, for example).

> And this dtext problem hits also the 
> <General-address-literal> in RfC 2821.  It is _our_ job, if we
> don't get the <msg-id> right this WG has absolutely no business
> to mess with NetNews at all.

You are free to suggest addition of a (redundant) note pointing
out that at this time IPv4 address literals are limited to dotted
decimal quads as specified in STD 3.  You can add, if you wish,
that there is no single agreed-upon way to represent IPv6 address
literals.
  
> > o NNTP and URIs are handled by other groups.
> 
> See the WG Charter quoted above.  State of the art in nntp -27:

This WG doesn't have a "nntp -27".
 
[...]
> > Wording: "fixed" implies that something is "broken"
> 
> Yes, and the least common denominators are printable (syntax)
> and a domain allowing for a domain-literal as RHS (semantics).

RFC 2822 msg-id permits all of that.  If you have a specific case
of a standards-compliant external protocol issue that adds some
restriction, please specify it along the lines suggested by Mark
Crispin.  We can't rewrite 2822.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: Feedback on proposal
Date: Mon, 27 Jun 2005 08:15:39 -0400
Cc: [email-masked]
Sender: [email-masked]

On Mon June 27 2005 04:50, Frank Ellermann wrote:

> Yes, AFAIK yenc is 8bit minus NUL, CR, LF, and optional tricks
> to avoid trailing SP or HT.

The key here is "AFAIK". The problem is that "n" (from your other
message) isn't specified to be constrained to 998 or less, and in
any event is not formally specified in a stable specification
suitable as the basis for an IETF encoding.  That has led to
interoperability problems (as noted by Russ), and the proponents
of yEnc seem to be unwilling to standardize the specification in
the IETF (as noted by several people).

> OTOH it's also not really binary, it's some kind of 8bit.

iff n <= 998
 
> >> how should it post this stuff ?
> 
> > Non-standardly.  Here, I was discussing software support for
> > conventions, not standards conformance
> 
> That's not allowed in usefor-0?, all UAs MUST follow RfC 2049,
> and 2049 says "post either MIME or ASCII".  That's what Steve
> was talking about, conformance with our standard includes this
> one MUST.

N.B. "conformance with our" specification (it's not a standard).
 
> > Clients can have non-standard modes as well as standard ones
> 
> Then they'd need a SHOULD instead of a MUST in usefor to _post_
> such articles.  Bye, Frank

No, there are no IETF brown shirts forcing conformance at gunpoint.
No matter what we say (given the constraint that what we say has to
pass IESG muster), there will be non-conforming software.  Our task
is to specify format, protocol, and best practice for those who wish
to build conforming implementations.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: Feedback on proposal
Date: Mon, 27 Jun 2005 08:01:18 -0400
Cc: [email-masked]
Sender: [email-masked]

On Sun June 26 2005 22:40, Frank Ellermann wrote:

> How about this:
> 
>  MIME-Version: 1.0
>  Content-Type: text/x-yenc; charset=unknown-8bit (RfC 1428)
>  Content-Transfer-Encoding: 8bit
[...]
> Or why not this:
> 
>  MIME-Version: 1.0
>  Content-Type: application/x-text-with-yenc
>  Content-Transfer-Encoding: 8bit

The problem with both is that there is no satisfactory indication
of the media type.  In the first case, you're claiming the the
unencoded content is text of some sort. In the second, you're
claiming application data.  In practice, yEnc is used for image,
video, and audio media, possibly also application data; I doubt
that is has been used for text.  The idea (granted, not 100% clear
from the specifications) is that a MIME media type identifies what
the (unencoded) content is, and Content-Transfer-Encoding
identifies two things:
1. An encoding method (possibly an "identity" encoding)
2. The characteristics ("domain" in RFC 2045) of the encoded
   data (7bit, 8bit, binary).

YEnc is clearly a transfer encoding, not a media type.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Feedback on proposal
Date:  Mon, 27 Jun 2005 11:07:28 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 23
Sender: [email-masked]

Russ Allbery wrote:

> you can't use CTE 8bit for yEnc data.  RFC 2045 says:
> "8bit data" refers to data that is all represented as
>  relatively short lines with 998 octets or less between
>  CRLF line separation sequences [RFC-821]), but octets
>  with decimal values greater than 127 may be used.  As
>  with "7bit data" CR and LF octets only occur as part
>  of CRLF line separation sequences and no NULs are allowed.

> and yEnc breaks the line length requirement.

I'm not sure, seen in file YENC_1_3.TXT (2002-07-22 on my box):

| To facilitate transmission via existing standard protocols
| (most notably NNTP), carriage return/linefeed pairs should be
| written to the output stream after every n characters, where
| n is the desired line length.  Typical values for n are 128
| and 256.

That's smaller than 998.  So it should be okay for 8bit MIME,
although anything else is more like UUE than MIME.  Bye, Frank

Date: Mon, 27 Jun 2005 11:07:58 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: "System" (Re: NEW: USEFOR -04 3.2.11 Xref vs. transport)
Sender: [email-masked]

--On 25. juni 2005 18:55 -0400 Bruce Lilly <[email-masked]> wrote:

>> So the wording "of use to a local system" is not accurate.  Consider
>> what you are proposing: clients cannot be expected to grab a message by
>> ID, and then only have a newsgroups header to go on.
>
> For clarification, as there seems to be continued confusion, the proposal
> has been carefully considered; "local system" refers to a server *and*
> its clients.

Bruce,

usefor-04 does not contain a definition of "system".

In fact the word is used in 2 different senses: "a system" (something that 
passes articles) and "the news system".

Unusually for you, you're using controversial terms without defining them 
first.

Date: Mon, 27 Jun 2005 10:58:43 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1050: USEFOR 3.2.11 Xref vs. transport
Sender: [email-masked]

Ticket #1050 created. Based on the following discussion, I have put it into 
state "no change needed".

--On 25. juni 2005 11:14 -0400 Bruce Lilly <[email-masked]> wrote:

>
> RFC 1036 says, regarding the Xref field:
>
>     This is only of value to the local system, so it should not be
>     transmitted.
>
> There is no corresponding text in the usefor draft version -04.  Best
> current practice seems to be to keep track of local data in a separate
> database rather than in a non-transmitted addition to the article
> header.  I therefore propose that the following text (subject to
> discussion of course) be added to the usefor draft Xref text:
>
>   The optional Xref field has been used as a place to store
>   cross-reference information of use to a local system.  Other
>   methods of handling such metadata include separate databases.
>
> That is to clarify the semantics of the field.  When we get to the usepro
> document, text similar to the following could appear there:
>
>   Because the Xref field is only of use to a particular system, it
>   SHOULD NOT be transmitted to other systems.
>
>
>

Date: Mon, 27 Jun 2005 10:56:47 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

--On 26. juni 2005 18:47 -0400 Bruce Lilly <[email-masked]> wrote:

> On Sun June 26 2005 15:28, Harald Tveit Alvestrand wrote:
>
>> I thought I was making a note that concerned only the use of the term
>> "header", while your suggestion of adding MIME to the mix seems
>> appropriate  if the purpose of the note was to say that we were going to
>> use all the  terminology from MIME and RFC 2822 (and others).
>
> The purpose of adding MIME is to acknowledge that MIME RFCs also use the
> terms "header" (as in MIME-part[ -]header) and "header field".

Uses. Does not define. And (naturally) refers to RFC 822, not 2822.
RFC 2045 section 1:

   All of the header fields defined in this document are subject to the
   general syntactic rules for header fields specified in RFC 822.  In
   particular, all of these header fields except for Content-Disposition
   can include RFC 822 comments, which have no semantic content and
   should be ignored during MIME processing.

We're dealing with definitions here. We should just refer to 2822.

Your argument has been made. Unless others strongly wish to support the 
inclusion, this change will not be made.

                 Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Feedback on proposal
Date:  Mon, 27 Jun 2005 10:50:33 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 28
Sender: [email-masked]

Henry Spencer wrote:

> The passages you note cover only articles containing
> non-ASCII text.

Yes, AFAIK yenc is 8bit minus NUL, CR, LF, and optional tricks
to avoid trailing SP or HT.  All octects %x01..%xFF reserving
CRLF for line ends, and using "=" as an escape character.  So
that's certainly not 7bit or ASCII, whatever else it might be.

OTOH it's also not really binary, it's some kind of 8bit.

>> how should it post this stuff ?

> Non-standardly.  Here, I was discussing software support for
> conventions, not standards conformance

That's not allowed in usefor-0?, all UAs MUST follow RfC 2049,
and 2049 says "post either MIME or ASCII".  That's what Steve
was talking about, conformance with our standard includes this
one MUST.

> Clients can have non-standard modes as well as standard ones

Then they'd need a SHOULD instead of a MUST in usefor to _post_
such articles.  Bye, Frank

Date: Mon, 27 Jun 2005 00:00:23 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

On Mon, 27 Jun 2005, Frank Ellermann wrote:
> > There's nothing which requires a MIME-supporting client to
> > use MIME in all articles posted through it.
> 
> There is, see 2049...

The passages you note cover only articles containing non-ASCII text. 
Nothing compels MIME use for articles containing only ASCII characters
(possibly as a result of using higher-level conventions to encode
non-ASCII material in ASCII). 

> > just because a client supports MIME doesn't mean it can't
> > support other conventions as well.
> 
> For UUE and other variations of 7bit no problem, but for any
> 8bit conventions, how should it post this stuff ?

Non-standardly.  Here, I was discussing software support for conventions,
not standards conformance, refuting the bizarre contention that a client
which supports MIME couldn't possibly support any alternative.  Clients
can have non-standard modes as well as standard ones; the non-standard
modes might even be the defaults.  This is arguably a Bad Thing, but it is
unquestionably possible. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Feedback on proposal
Date:  Mon, 27 Jun 2005 05:12:22 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 30
Sender: [email-masked]

Henry Spencer wrote:
 
> There's nothing which requires a MIME-supporting client to
> use MIME in all articles posted through it.

There is, see 2049:

| (8) Conformant user agents are required, if they provide
|     non-standard support for non-MIME messages employing
|     character sets other than US-ASCII, to do so on
|     received messages only. Conforming user agents must not
|     send non-MIME messages containing anything other than
|     US-ASCII text.

|     In particular, the use of non-US-ASCII text in mail
|     messages without a MIME-Version field is strongly
|     discouraged as it impedes interoperability when sending
|     messages between regions with different localization
|     conventions. Conforming user agents MUST include proper
|     MIME labelling when sending anything other than plain
|     text in the US-ASCII character set.

Steve got us, both MUST in usefor-04 (2.3) are utter dubious.

> just because a client supports MIME doesn't mean it can't
> support other conventions as well.

For UUE and other variations of 7bit no problem, but for any
8bit conventions, how should it post this stuff ?  Bye, Frank

To: [email-masked]
Subject: Re: Feedback on proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sun, 26 Jun 2005 19:59:38 -0700
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> How about this:

>  MIME-Version: 1.0
>  Content-Type: text/x-yenc; charset=unknown-8bit (RfC 1428)
>  Content-Transfer-Encoding: 8bit

> If I understand RfC 2046 correctly that should have a similar
> effect like application/octet-stream for any newsreader not
> knowing what to do with it, see "4.1.4 Unrecognized Subtypes".

Actually, I believe that:

    MIME-Version: 1.0
    Content-Type: application/x-yenc
    Content-Transfer-Encoding: x-yenc

does pretty much what you would want from the perspective of a MIME client
that doesn't understand yEnc.  The unrecognized CTE pretty much tells
everything "hands off" in a particularly thorough manner.  RFC 2045:

   Implementors may, if necessary, define private Content-Transfer-
   Encoding values, but must use an x-token, which is a name prefixed by
   "X-", to indicate its non-standard status, e.g., "Content-Transfer-
   Encoding: x-my-new-encoding". [...]

   Unlike media types and subtypes, the creation of new Content-
   Transfer-Encoding values is STRONGLY discouraged, as it seems likely
   to hinder interoperability with little potential benefit

[...]

   Any entity with an unrecognized Content-Transfer-Encoding must be
   treated as if it has a Content-Type of "application/octet-stream",
   regardless of what the Content-Type header field actually says.

The only thing it doesn't do is help a MIME client actually understand the
body as opposed to realize that it can't deal with it.

Structurally, yEnc is actually a multipart, but since it doesn't use MIME
bounaries, one can't tag it that way.  (Not to mention that multiparts
aren't allowed to use other CTEs.)

Anyway, you can't use CTE 8bit for yEnc data.  RFC 2045 says:

   "8bit data" refers to data that is all represented as relatively
   short lines with 998 octets or less between CRLF line separation
   sequences [RFC-821]), but octets with decimal values greater than 127
   may be used.  As with "7bit data" CR and LF octets only occur as part
   of CRLF line separation sequences and no NULs are allowed.

and yEnc breaks the line length requirement.  The only already-defined CTE
you can use for yEnc data is binary, which of course NNTP and most news
software can't handle since it requires nul-cleanliness and places no
special meaning on CR and LF.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Feedback on proposal
Date:  Mon, 27 Jun 2005 04:40:05 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 40
Sender: [email-masked]

Steve Walker wrote:

> Some form of meta data is needed to officially separate the
> MIME articles from the 5 million non-MIME articles that
> arrive every day.

How about this:

 MIME-Version: 1.0
 Content-Type: text/x-yenc; charset=unknown-8bit (RfC 1428)
 Content-Transfer-Encoding: 8bit

If I understand RfC 2046 correctly that should have a similar
effect like application/octet-stream for any newsreader not
knowing what to do with it, see "4.1.4 Unrecognized Subtypes".

Or why not this:

 MIME-Version: 1.0
 Content-Type: application/x-text-with-yenc
 Content-Transfer-Encoding: 8bit

> If you push a standard that says everything must be mime
> and 85% of articles are non-mime there will be problems.

If adding one of the above header-blocks is asking too much
from these UAs, then they'll be treated as "non-conformant"
violating the first MUST in usefor-04 section 2.3

It also says "MUST support 2231" - I'm 100% sure that my UA
doesn't support 2231.  Or to say it more clearly, I very much
doubt that even ONE user-agent supporting all 2231-oddities
in a MIME boundary parameter exists.

Now that asks for a new ticket, the minimal MIME conformance is
defined in 2049, and a "MUST support 2231" rendering precisely
ALL (= my guess) UAs as non-conformant is obviously wrong.  At
best it can be a SHOULD.
                        Bye, Frank

Date: Sun, 26 Jun 2005 21:28:23 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Steve Walker <[email-masked]> wrote:

> I'd also like to point out that no one person here has commented 
> on the suggestion of requiring article numbers to be 64 bit.

OK, I'll comment on it.  What's an "article number"?  Is that some
local thing that different sites do differently and nobody non-local
needs to care about?

Seth

Date: Sun, 26 Jun 2005 19:30:36 -0500
From: Steve Walker <[email-masked]>
To: Kai Henningsen <[email-masked]>
CC: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Kai Henningsen wrote:
> As far as I'm concerned - and I expect I'm not representing any kind of  
> majority here - binary traffic is a completely uninteresting part of  
> Usenet. In fact, my position may be better characterized by saying that  
> binary traffic isn't part of Usenet, at least in general.

I can understand someone not liking the expenses involved in 
running a news server due to the traffic caused by binaries.  I 
can also understand someone not liking what someone posts, be it 
binary or text.  But I can't imagine how anyone could say that a 
binary article is not Usenet.  If one person shares a written 
story and another shares an audio book of the same story, what 
makes one Usenet and the other not?  Is it the size of the 
article or the file type?  Or is it the person?

Steve Walker
NNTPServer.com

Date: Sun, 26 Jun 2005 19:17:57 -0500
From: Steve Walker <[email-masked]>
To: Ned Freed <[email-masked]>
CC: Henry Spencer <[email-masked]>, Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

Ned Freed wrote:
> The use of media types to identify encodings is specifically prohibited. 
> This
> is a transfer encoding, not a media type.

I was not referring to an official MIME type at all.  I was 
suggesting a new header that signifies the following body is not 
MIME.  If Content-type is too much like MIME then use something 
else like NNTP-Format.  Some form of meta data is needed to 
officially separate the MIME articles from the 5 million non-MIME 
articles that arrive every day.

If you push a standard that says everything must be mime and 85% 
of articles are non-mime there will be problems.  If people 
expect non-mime articles they will write better software.

Steve Walker
NNTPServer.com

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1043 "header": Suggested resolution
Date: Sun, 26 Jun 2005 18:47:36 -0400
Cc: [email-masked]
Sender: [email-masked]

On Sun June 26 2005 15:28, Harald Tveit Alvestrand wrote:

> I thought I was making a note that concerned only the use of the term 
> "header", while your suggestion of adding MIME to the mix seems appropriate 
> if the purpose of the note was to say that we were going to use all the 
> terminology from MIME and RFC 2822 (and others).

The purpose of adding MIME is to acknowledge that MIME RFCs also use the
terms "header" (as in MIME-part[ -]header) and "header field".  Usefor
draft -04 also refers to MIME fields in section 2.3 and 3.2.  There are
differences between a message header and a MIME-part header, and the two
sets of RFCs are disjoint (I have already pointed out 2822s "out of scope"
text; RFC 2045 says "these documents are largely orthogonal to [...] RFC
822").  I don't know that we're going to use *all* of the MIME terminology,
but we certainly do refer to:
o "the character set for headers [...] MIME mechanisms"
o "MIME conformance [...] internationalization of headers"
o "MIME extension headers"
o "The MIME headers MIME-Version [...]"
o "MIME message and multipart parts"
o "[...] headers and bodies [...] using MIME mechanisms"
RFC 2047 specifically (sect. 5) pertains to MIME-part (therein called
"body part") header fields.  MIME extension fields by RFC 2045 rules begin
with "Content-" and are specifically permitted in a MIME-part header (indeed,
only those fields explicitly have meaning there; RFC 2046 sect. 5.1).
MIME-part headers of course appear in what is considered an RFC 822/2822
body.  So specifically related to use of the terms "header" and "header
field", we do use (directly and indirectly) the relevant MIME terms as
they relate to MIME-part headers.

Date: Sun, 26 Jun 2005 23:56:23 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: gzip/deflate compression/encoding
Cc: Dan Kohn <[email-masked]>, Bruce Lilly <[email-masked]>, Ned Freed <[email-masked]>, [email-masked]
Sender: [email-masked]

IMNSHO, a protocol extension will succeed only if implementing it brings 
value to the user. If implementing e.g the SMTP MUMBLE extension 
doesn't benefit either the implementor or the user, and does benefit 
someone else greatly, then that extension isn't going to be implemented 
very much.

I may be wrong, of course. But let's suppose I'm right. In that case:

1. Mobile MUAs get considerable benefit from compressing messages as 
they go from/to the server, so an SMTP/POP/IMAP extension to provide 
compression for all messages or all attachments ought to be fairly 
widely implemented by authors of such MUAs (and likely seldom by 
others).

2. Even mobile MUAs don't get much benefit from c-t-e: deflate, since 
that works only when both the message sender and all receivers support 
it, which'll be fairly seldom. Authors of clients who assume ethernet 
to the server almost never get any benefit.

Arnt

Date: 26 Jun 2005 16:46:00 +0200
From: [email-masked] (Kai Henningsen)
To: [email-masked]
Subject: Re: Feedback on proposal
Organization: Organisation? Me?! Are you kidding?
Comment: Unsolicited commercial mail will incur an US$100 handling fee per received mail.
Sender: [email-masked]

[email-masked] (Steve Walker)  wrote on 24.06.05 in <[email-masked]>:

> For example, why doesn't the "standard" specify that 64-bit
> integers are used for article counters?  A real-world problem
> facing NNTP developers is what happens when the 31/32 bit counter
> rolls over?  I highly doubt that most clients can deal with this
> issue.  Some clients are going to roll at 2^31 and some at 2^32.

Wrong working group. NNTPEXT is <--- thataway. (Incidentally, the update  
to the base standard just passed IESG review, so you're a little late for  
any changes; on the other hand, towards the end, the group was nearly  
exclusively manned by server software authors ...)

MfG Kai

Date: 26 Jun 2005 14:47:00 +0200
From: [email-masked] (Kai Henningsen)
To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Organization: Organisation? Me?! Are you kidding?
Comment: Unsolicited commercial mail will incur an US$100 handling fee per received mail.
Sender: [email-masked]

[email-masked] (Frank Ellermann)  wrote on 21.06.05 in <[email-masked]>:

> RfC 2822 missed the idea of Message-IDs, a globally unique
> identifier forever.

I'm seriously fed up by this completely unfounded assertion that seems to  
be impossible to kill no matter how false.

In fact, the idea goes back at least to RFC 624 (obsoleted by 733, which  
was obsoleted by 822, which is obsoleted by 2822), 1977-05-12.

All we're debating are implementation techniques and problems, not the  
fundamental concept.

MfG Kai

Date: 26 Jun 2005 16:42:00 +0200
From: [email-masked] (Kai Henningsen)
To: [email-masked]
Subject: Re: Feedback on proposal
Organization: Organisation? Me?! Are you kidding?
Comment: Unsolicited commercial mail will incur an US$100 handling fee per received mail.
Sender: [email-masked]

[email-masked] (Steve Walker)  wrote on 24.06.05 in <[email-masked]>:

> Sebastian Brocks wrote:
> > Steve Walker schrieb:
> >
> >
> >>You will not get the other 96% to
> >>switch to mime just because this "standard" is published.
> >
> >
> > The vast majority of that 96% is binary traffic, which does not
> > care about standards anyway.
>
> So you write something knowing no one will follow and call it the
> standard.  Why don't we write a "standard" that follows standard
> practices.

As far as I'm concerned - and I expect I'm not representing any kind of  
majority here - binary traffic is a completely uninteresting part of  
Usenet. In fact, my position may be better characterized by saying that  
binary traffic isn't part of Usenet, at least in general.

In any case, software authors and users of that are certainly on record as  
not caring for IETF standards. See, for one of many examples, the yEnc  
discussions.

My only interest is in text Usenet.

MfG Kai

Date: 26 Jun 2005 14:15:00 +0200
From: [email-masked] (Kai Henningsen)
To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Organization: Organisation? Me?! Are you kidding?
Comment: Unsolicited commercial mail will incur an US$100 handling fee per received mail.
Sender: [email-masked]

[email-masked] (Frank Ellermann)  wrote on 14.06.05 in <[email-masked]>:

> Harald Tveit Alvestrand wrote:
>
> > I'm going to resolve the ticket as "just replace mqtext with
> > *mqtext".
>
> > OK?
>
> There's a dispute about the meaning of <id-right>.  I say it's
> in fact an <id-domain>, a proper name space with an owner.
>
> TTBOMK Charles says, whatever it is, in a msg-id it's case
> sensitive.  These positions aren't incompatible, but somewhat
> different.  I'm obsessed with the (2)822 semantics, Charles is
> more interested in a visual similarity to RfC 2822 <id-right>.

If we insist on at least as much different semantics from a domain in an  
address as 2822 (it's not clear to me what 2822 implies about this), then  
we certainly shouldn't be any closer to the notation of a domain in an  
address.

I'm for id-right.

MfG Kai

Cc: Henry Spencer <[email-masked]>, Usefor Mailing List <[email-masked]>
To: Steve Walker <[email-masked]>
Date: Sun, 26 Jun 2005 13:52:12 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

> Henry Spencer wrote:
> > The only place where we deal with any form of article numbering is in an
> > explanatory note on our discussion of the Xref header, where we note that
> > Xref "article locators" are commonly decimal numbers.  We say nothing
> > about how long those numbers might be, since we're dealing with them in
> > ASCII text, not in a binary representation.

> My apologies, I am in the wrong group for this topic.

> On the other issue, has anyone proposed using headers similar to
> 'Content-type: raw/subtype' or 'NNTP-Format: raw/subtype'.

The use of media types to identify encodings is specifically prohibited. This
is a transfer encoding, not a media type.

> Which
> would mean that \r\n\0 are escaped and the line length limits
> were kept, but other than that no further restrictions on format
> or content of the article was made.

Bruce already pointed out that not only is it perfectly permissible to define
additionaal content encodings for this sort of thing, at least two drafts have
been written that propose doing exactly this. Both have been allowed to
exprire. I cannot speak for Dan's draft but I let mine expire due to there
being little if any apparent interest in pursuing it from the community.

				Ned

To: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sun, 26 Jun 2005 13:52:29 -0700
Sender: [email-masked]

Steve Walker <[email-masked]> writes:

> On the other issue, has anyone proposed using headers similar to
> 'Content-type: raw/subtype' or 'NNTP-Format: raw/subtype'.  Which would
> mean that \r\n\0 are escaped and the line length limits were kept, but
> other than that no further restrictions on format or content of the
> article was made.

This would be a Content-Transfer-Encoding (CTE), not a Content-Type, and
yes, people have.  I tried to work with the author of yEnc on such a
proposal, but unfortunately didn't get anywhere.  There are reasons why
the IETF is reluctant to bless any additional CTEs, but I think that this
one could have a future and could make it through the process.  In the
absence of any interest or cooperation from the motivating software
package, however, I gave up.

Date: Sun, 26 Jun 2005 15:53:33 -0500
From: Steve Walker <[email-masked]>
To: Henry Spencer <[email-masked]>
CC: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

Henry Spencer wrote:
> The only place where we deal with any form of article numbering is in an
> explanatory note on our discussion of the Xref header, where we note that
> Xref "article locators" are commonly decimal numbers.  We say nothing
> about how long those numbers might be, since we're dealing with them in
> ASCII text, not in a binary representation. 

My apologies, I am in the wrong group for this topic.

On the other issue, has anyone proposed using headers similar to
'Content-type: raw/subtype' or 'NNTP-Format: raw/subtype'.  Which 
would mean that \r\n\0 are escaped and the line length limits 
were kept, but other than that no further restrictions on format 
or content of the article was made.

This would allow yEncoding and any future applications to used in 
a way that would not cause harm to others. Clients that didn't 
want to process articles with an unspecified format would just 
ignore them.  Specialized clients would only process articles 
with a content-type of raw looking for their own signatures.

It seems like an extremely simple solution and I believe that you 
may get the client developers to add one simple header if it 
means complying with a new standard.

Steve Walker.
NNTPServer.com

Date: Sun, 26 Jun 2005 21:28:07 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

--On 25. juni 2005 11:52 -0400 Bruce Lilly <[email-masked]> wrote:

>> If there was ever a conflict discovered between MIME and 2822 about what
>> a  "header field" was, this would make our "note" ambiguous.
>> And if there is no conflict, the change adds no information.
>
> The information added relates to MIME-part headers and the fields
> contained therein, as defined in RFCs 2045 and 2046 and used in other MIME
> specifications, which as noted in 2822 itself (sect. 1.1) "are outside of
> the scope of" 2822.  Although there is no conflict, there is additional
> information.  [rather than asserting that a specific suggested change,
> which can be assumed to have been suggested for a reason, is "useless",
> "harmful", "ambiguous", and "adds no information", I suggest that in
> future a more professional and productive approach would be to ask a
> question if you are unclear about the reason for the suggestion]

After reading this a couple of times, I think I get the point.
When I suggested the note:

  NOTE: Older News specifications used the term "header" as a synonym
     for what [RFC2822] calls "header field". This document follows
     the terminology of RFC 2822.

I thought I was making a note that concerned only the use of the term 
"header", while your suggestion of adding MIME to the mix seems appropriate 
if the purpose of the note was to say that we were going to use all the 
terminology from MIME and RFC 2822 (and others). To make it clearer what I 
intended to say, I'll amend my suggested note as follows:

  NOTE: Older News specifications used the term "header" as a synonym
     for what [RFC2822] calls "header field". This document follows
     the terminology of RFC 2822 in using the term "header field",
     based on a belief that consistent terminology among specifications
     that depend on each other makes the specifications easier to use
     in the long run.

That's also as far as I think it's appropriate to go in what Henry Spencer 
calls "apologizing for the change".

Date: Sun, 26 Jun 2005 14:08:56 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

On Fri, 24 Jun 2005, Steve Walker wrote:
> For example, why doesn't the "standard" specify that 64-bit 
> integers are used for article counters?

Our draft says nothing about the number of bits used in an article
counter, because we don't have the concept of an article counter. 

The only place where we deal with any form of article numbering is in an
explanatory note on our discussion of the Xref header, where we note that
Xref "article locators" are commonly decimal numbers.  We say nothing
about how long those numbers might be, since we're dealing with them in
ASCII text, not in a binary representation. 

> A real-world problem facing NNTP developers...

Discuss this with the NNTP WG, please.  It's not an article-format or
article-processing issue, so it's outside our scope.

                                                          Henry Spencer
                                                       [email-masked]

Date: Sun, 26 Jun 2005 13:59:17 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

On Fri, 24 Jun 2005, Steve Walker wrote:
> ...In the real world, if all clients must support mime 
> how do non-mime articles get posted.

There's nothing which requires a MIME-supporting client to use MIME in all
articles posted through it.

For one thing, an article which is written in plain ASCII text doesn't
need to be a MIME message.  Some clients might throw in

	MIME-Version: 1.0
	Content-Type: text/plain; charset=us-ascii

just on general principles, but this doesn't actually affect the content
or interpretation of the article.  (That Content-Type header is completely
redundant -- text/plain is the default is the default content type, and
us-ascii is the default character set for text/plain.  The MIME-Version
header merely adds a -- rather weak -- assurance that in the absence of a
Content-Type header, the article really is plain human-readable text.)

For another, just because a client supports MIME doesn't mean it can't
support other conventions as well.  MIME might not even be the default;
the MIME support might be activated only by explicit user request.  All
our drafts require is that MIME support be present. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date:  Sun, 26 Jun 2005 18:47:19 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 108
Sender: [email-masked]

Bruce Lilly wrote:
 
> The WG charter gives no authorization to modify RFC 2822.

If you look at http://tools.ietf.org/wg/usefor/ you'd find
among others two expired drafts especially about the msg-id.

You'd also find a link to the WG Charter stating:

| A draft update to RFC 1036 ("Son of RFC 1036" ) was released
| by Henry Spencer in June 1994 but this was not further
| pursued and is now itself out of date. Currently a
| combination of this and RFC 1036 are regarded as the de-facto
| standard.
[...]
| The Group shall also aid and/or oversee the production of
| other Usenet related Internet Drafts and Standards.

That covers NNTP and the two news-nntp-uri drafts.  If the WG
failed to fix the msg-id error in [MESSFOR] it's sad, but at
least it's not forced to repeat this error.

> o CID and MID URIs (RFC 2392) seem to work fine.

That's essentially a percent-encoded RfC 822 <addr-spec>, and
we already know that we don't want WSP, NO-WS-CTL, or comments
in a <msg-id>.  Not one of the RfC 2392 examples discusses or
uses such constructs, it's about %25 for % etc. in URLs based
on RfC 1738.

> o there are mechanisms other than syntax changes (e.g.
>   implementation notes) to address specific issues.

If we don't get the <msg-id> or <newsgroup-name> syntax right
this WG should be dissolved.  Intentionally hiding this syntax
problem in prose is cowardly and no option.  

The IESG is already at its lowest point of unprofessional and
unethical behaviour after approving technically incompatible
"mail experiments".  That's no excuse for you and me to follow
their "worst common practice".

For an example how this problem should be _not_ solved see e.g.
<http://article.gmane.org/gmane.org.w3c.uri/569/match=dtext> :

| A word about non-whitespace controls, since there seems to
| be some confusion: aside from appearance in qtext and ctext,
| they also appear in the 2822 grammar for dtext.  The reason
| for that as I understand it (private correspondence with
| Pete Resnick) is that at the time, IDN had not been fleshed
| out, and the provision is to accommodate possible schemes.
| Of course at the present time, a domain name, including IDN,
| consists solely of letters, digits, and hyphens plus
| separating dots.  So while the syntax has provision for
| NO-WS-CTLS in dtext, those won't actually occur because of
| restrictions in other specifications (notably RFCs 1034, 1035).

Your communication with the [MESSFOR] author is not good enough
for a future <id-domain>.  And this dtext problem hits also the
<General-address-literal> in RfC 2821.  It is _our_ job, if we
don't get the <msg-id> right this WG has absolutely no business
to mess with NetNews at all.

> o NNTP and URIs are handled by other groups.

See the WG Charter quoted above.  State of the art in nntp -27:

| A message-id MUST begin with "<" and end with ">", and MUST
| NOT contain the latter except at the end.
| A message-id MUST be between 3 and 250 octets in length.
| A message-id MUST NOT contain octets other than printable
| US-ASCII characters.
[...]
|      message-id = "<" 1*248A-NOTGT ">"
[...]
|      A-NOTGT    = %x21-3D / %x3F-7E  ; exclude ">"

For the purposes of NNTP the <id-local> "@" <id-domain> details
are irrelevant, as long as the combination is unique, worldwide
and forever.  For other purposes like news-URLs, or designing a
msg-id generator based on proper name spaces, the structure is
important.

> Not our problem.

Read the Charter.  Of course it's our problem, it affects also
<unique>@<full_domain_name> in RfC 1738, or rather its syntax:

article = 1*[ uchar | ";"| "/" | "?" | ":" | "&" | "=" ] "@" host

The two proposed news-nntp-uri-00 successors use either:

    message-id      = printable-ascii "@" printable-ascii
    printable-ascii = 1*( %d33-61 / %d63-126 ) ; excludes ">"
or:
    message-id = unique "@" host               ; see RFC 1036
    unique     = 1*( %d33-61 / %d63-126 )      ; excludes ">"

Some details still TBD, at the moment <unique> is plain wrong,
because it has to be percent-encoded in an URL.

> Wording: "fixed" implies that something is "broken"

Yes, and the least common denominators are printable (syntax)
and a domain allowing for a domain-literal as RHS (semantics).

                          Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date:  Sun, 26 Jun 2005 16:42:50 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 16
Sender: [email-masked]

Russ Allbery wrote:
 
> I object to saying that removal of Xref is a SHOULD for
> servers.
[...] 
> I'm going to now assume that people understand my position
> on this proposal without these sorts of small clarifications.

FWIW "add me".  Bruce's definition of a "local system" as in
'"local system" refers to a server *and* its clients' doesn't
match my concept of NetNews, where a "leafnode" with more than
one "node" is in theory just another "node".

BTW, that's also the reason why I hunted the word "downstream"
in an earlier draft until it was finally removed.  Bye, Frank

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Forrest J. Cavalier III" <[email-masked]>
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Sat, 25 Jun 2005 18:55:53 -0400
Cc: [email-masked]
Sender: [email-masked]

On Sat June 25 2005 16:06, Forrest J. Cavalier III wrote:

> If a client obtains an article by Msg-Id instead of newsgroup:number,
> it needs the Xref information to properly mark seen/unseen articles,
> even if the message is in the CURRENT newsgroup.
> 
> So the wording "of use to a local system" is not accurate.  Consider
> what you are proposing: clients cannot be expected to grab a message by
> ID, and then only have a newsgroups header to go on.

For clarification, as there seems to be continued confusion, the proposal
has been carefully considered; "local system" refers to a server *and*
its clients.  The proposal is to state that Xref is used to store
metadata on a local system, and that there are security implications
involved in transmitting that local data with an article to remote
systems, where "remote systems" means a different server where the field
body -- if the optional field is used on that remote system -- would
necessarily have to be different.

> I also don't agree with your suggestion that it is BCP which would
> make any improvement in complexity of implementation or CPU load.
> 
> I expect that most server software accepting a feed does Path header
> slicing and Xref header replacement in the same section of code, using
> similar code.  INN works this way.  The work to cut information out of
> the header is about the same in both cases, and we certainly can't drop
> Path splicing.

You missed the critical point: the work to "cut information out of the
header" is zero for Xref if there is no Xref field.  Many servers already
index information by message-id, and I see no reason why this sort of
cross-reference information, which exists when an article is stored
locally, could not be part of such data.

> Software complexity is not going to be reduced by this SHOULD NOT either.
> A SHOULD NOT added to USEFOR

The "SHOULD NOT" was for usepro, not usefor, and exists because of
security issues, not "complexity".  See separate discussions.

> A note about how Xref leaks information might be something for USEAGE.

Security issues related to the protocols should be discussed in the
protocol document; that's not a matter that can be deferred to a mere
informative RFC.

From: Matthias Andree <[email-masked]>
To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Sun, 26 Jun 2005 00:07:12 +0200
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:

> RFC 1036 says, regarding the Xref field:
>
>     This is only of value to the local system, so it should not be
>     transmitted.
>
> There is no corresponding text in the usefor draft version -04.  Best
> current practice seems to be to keep track of local data in a separate
> database rather than in a non-transmitted addition to the article
> header.  I therefore propose that the following text (subject to discussion
> of course) be added to the usefor draft Xref text:
>
>   The optional Xref field has been used as a place to store
>   cross-reference information of use to a local system.  Other
>   methods of handling such metadata include separate databases.
>
> That is to clarify the semantics of the field.  When we get to the usepro
> document, text similar to the following could appear there:
>
>   Because the Xref field is only of use to a particular system, it
>   SHOULD NOT be transmitted to other systems.

The Xref field is usually (i. e. with all upstream servers I can access,
three different software packages and versions) provided through the
RFC-2980-style XOVER, also in the current NNTPEXT-base draft #27 as
OVER, and is somewhat useful to sort out postings that are excessively
crossposted without looking at the HEAD.

Clients might use it by marking a cross-posted article read in
.newsrc-style files in other groups without searching for the
Message-ID.

About transmission to other systems, I'd expect a receiving system that
uses the Xref header to replace it by its own data. Leafnode for
instance does that, and INN appears to do that, too. I wonder if this
issue is so important as to deserve reglementation or strong
recommendations at all.

-- 
Matthias Andree

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Henry Spencer <[email-masked]>
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Sat, 25 Jun 2005 17:16:41 -0400
Cc: [email-masked]
Sender: [email-masked]

On Sat June 25 2005 15:38, Henry Spencer wrote:
> 
> On Sat, 25 Jun 2005, Bruce Lilly wrote:

> > Because...?  (N.B. the BCP 14 meaning of SHOULD NOT is a recommendation,
> > not an imperative).
> 
> By author's intent and wide community consensus, SHOULD NOT is an
> *emphatic* recommendation, not to be disregarded without specific and
> important reasons.

I make no claim about the BCP 14 author's intent or about consensus for a
specific interpretation which is not present in the approved document's text.
Here, verbatim, is what the document actually says regarding SHOULD NOT:

4. SHOULD NOT   This phrase, or the phrase "NOT RECOMMENDED" mean that
   there may exist valid reasons in particular circumstances when the
   particular behavior is acceptable or even useful, but the full
   implications should be understood and the case carefully weighed
   before implementing any behavior described with this label.

That text, indeed the full text of BCP 14, fails to include the words
"emphatic", "specific", or "important", with or without "scare" asterisks.
It says what it says, and I have quoted that section verbatim above.
While I would have been happy to defer detailed discussion of suggested
usepro text until we get to that document, I chose the "SHOULD NOT" phrase
carefully and specifically in the context of the suggested text.

> BCP 14 itself calls for using its keywords sparingly, 
> adding:  "they must not be used to try to impose a particular method on
> implementors where the method is not required for interoperability".

The context of that sentence fragment comes from section 6 of BCP 14,
which reads:

6. Guidance in the use of these Imperatives

   Imperatives of the type defined in this memo must be used with care
   and sparingly.  In particular, they MUST only be used where it is
   actually required for interoperation or to limit behavior which has
   potential for causing harm (e.g., limiting retransmisssions)  For
   example, they must not be used to try to impose a particular method
   on implementors where the method is not required for
   interoperability.

"they" in the quoted sentence fragment refers to "Imperatives", and
"SHOULD NOT" is no imperative.  See
http://dictionary.reference.com/search?q=imperative for definitions,
which include the words "command", "obligation", "an order", "compels
a certain behavior", in contrast to the specified meaning of BCP 14
"SHOULD NOT".  In any event, the phrase "SHOULD NOT" in the suggested
text was used sparingly and where security considerations apply; it
imposes no particular method (i.e. imperative or not, the guidelines
for use have been satisfied).  BCP 14 goes on to note:

7. Security Considerations

   These terms are frequently used to specify behavior with security
   implications.  The effects on security of not implementing a MUST or
   SHOULD, or doing something the specification says MUST NOT or SHOULD
   NOT be done may be very subtle. Document authors should take the time
   to elaborate the security implications of not following
   recommendations or requirements as most implementors will not have
   had the benefit of the experience and discussion that produced the
   specification.

While, as noted, I would have been happy to defer details to usepro, I
have taken the time to elaborate the security implications associated with
the suggested recommendation, and have pointed out that it has not been
posed as a requirement.

> The correct way to make a mild recommendation is not to use a keyword at
> all.

The intent is not "a mild recommendation"; it is intended to convey the
fact that there are (perhaps subtle) security effects and that "the full
implications should be understood and the case carefully weighed before
implementing" transmission of an Xref field containing local system data
to a remote system, precisely as BCP 14 indicates for use of SHOULD NOT.

> (And as noted in previous mail, this particular recommendation would 
> be commanding the tide not to come in, so it seems pointless.)

Your (unsubstantiated by facts, I note) claim about B news appears to
be a false claim.  Statements dependent on the veracity of a false claim
are dubious at best.  There was in fact a point to the specific text
proposed, and there was a reason why "SHOULD NOT" specifically was
proposed, not "MUST NOT" or "MAY" or "should not", and why I have taken
the effort to clarify mischaracterizations of what has been proposed,
at length and in detail.

So to also answer Russ' question, yes there is a point in clear, precise
communication and specifically there is a point in clarifying that the
suggested text is intended as a recommendation with security implications,
but falls short of an imperative (requirement or prohibition).

As I have said a few times, I would have preferred deferring discussion
of this to usepro, but there now seems little more to say on that part
of the suggestion.  That "little" may include the acknowledgment, here
given, that Russ' specific issues indeed qualify as "valid reasons in
particular circumstances when the particular behavior is acceptable or
even useful", but that does not change my position, and indeed is
consistent with it.

Let us now please return to the suggested text for usefor, where I still
don't know if "might be" is acceptable to Russ.

Date: Sat, 25 Jun 2005 16:45:05 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Sender: [email-masked]

On Sat, 25 Jun 2005, Bruce Lilly wrote:
>    #ifdef DOXREFS
>         if ( wr ==1 && *hp->xref)
>                 fprintf(fp, "Xref: %s\n", hp->xref);
>    #endif /* DOXREFS */
> If DOXREFS is not defined, nothing happens with Xref fields.  When
> defined (the default), wr (the third argument) is checked, and if and
> only if its value is one, the field is written.  As noted, the value
> is zero when transmitting an article to a remote system.

*If* you use the built-in single-article transmission in rfuncs2.c.  Which
nobody has done since *very* early in B News history, because sending
single articles via UUCP -- or most any other store-and-forward path -- is
grossly inefficient.  You're looking at unused vestigial code.

Almost all real B News sites used batch transmission exclusively.  (Bear
in mind that most B News sites joined Usenet only after batching became
standard practice.)  The B News *batcher* never did Xref exclusion; it
sent a byte-for-byte copy of the on-disk article, Xref and all.

                                                          Henry Spencer
                                                       [email-masked]

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Henry Spencer <[email-masked]>
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Sat, 25 Jun 2005 16:25:28 -0400
Cc: Usefor Mailing List <[email-masked]>
Sender: [email-masked]

On Sat June 25 2005 15:29, Henry Spencer wrote:
> Even B News,
> despite its alleged close relationship to RFC 1036, never did that.

B news source:

file rfuncs2.c:

   /*
    * Transmit file to system.
    */
   transmit(sp, file)

...

        if (index(sp->s_flags, 'A') == NULL)
                hwrite(&hh, ofp);

...

The 'A' flag, incidentally, is only present for A news format.
hwrite is defined as a macro in file header.h:

   #define hwrite(hp,fp)   ihwrite(hp,fp,0)
   #define lhwrite(hp,fp)  ihwrite(hp,fp,1)

note that the 3rd argument to ihwrite is zero.

file header.c:

   /*
    * Write header at 'hp' on stream 'fp' in B+ format.  Include received date
    * if wr is 1.  Leave off sysname if wr is 2.
    */
   #ifndef DOXREFS
   /*ARGSUSED*/
   #endif /* !DOXREFS */
   ihwrite(hp, fp, wr)
   register struct hbuf *hp;
   register FILE *fp;
   int     wr;
...

   #ifdef DOXREFS
        if ( wr ==1 && *hp->xref)
                fprintf(fp, "Xref: %s\n", hp->xref);
   #endif /* DOXREFS */
...

If DOXREFS is not defined, nothing happens with Xref fields.  When
defined (the default), wr (the third argument) is checked, and if and
only if its value is one, the field is written.  As noted, the value
is zero when transmitting an article to a remote system.

Date: Sat, 25 Jun 2005 16:06:51 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Sender: [email-masked]

Bruce Lilly wrote:
> RFC 1036 says, regarding the Xref field:
> 
>     This is only of value to the local system, so it should not be
>     transmitted.
> 
> There is no corresponding text in the usefor draft version -04.  Best
> current practice seems to be to keep track of local data in a separate
> database rather than in a non-transmitted addition to the article
> header.  I therefore propose that the following text (subject to discussion
> of course) be added to the usefor draft Xref text:
> 
>   The optional Xref field has been used as a place to store
>   cross-reference information of use to a local system.  Other
>   methods of handling such metadata include separate databases.
> 
> That is to clarify the semantics of the field.  When we get to the usepro
> document, text similar to the following could appear there:
> 
>   Because the Xref field is only of use to a particular system, it
>   SHOULD NOT be transmitted to other systems.
> 
> 

If a client obtains an article by Msg-Id instead of newsgroup:number,
it needs the Xref information to properly mark seen/unseen articles,
even if the message is in the CURRENT newsgroup.

So the wording "of use to a local system" is not accurate.  Consider
what you are proposing: clients cannot be expected to grab a message by
ID, and then only have a newsgroups header to go on.  Getting the message
numbers would require getting the entire XOVERs for all the articles
in all the groups listed in Newsgroups.  So servers really MUST serve
out Xref to clients.  It isn't a local system issue.

If the servers have to serve it out to clients, they have
to store it on spool, for all the same reasons I just mentioned.  It
would be very difficult for a server to generate Xref by cross-
referencing a message ID to a database, and then getting all the
newsgroup:number pairings and inserting the data.  Or do you think
that there should be a way to go from message-ID to overview record?
Keeping history is hard enough!

I also don't agree with your suggestion that it is BCP which would
make any improvement in complexity of implementation or CPU load.

I expect that most server software accepting a feed does Path header
slicing and Xref header replacement in the same section of code, using
similar code.  INN works this way.  The work to cut information out of
the header is about the same in both cases, and we certainly can't drop
Path splicing.

Software complexity is not going to be reduced by this SHOULD NOT either.
A SHOULD NOT added to USEFOR is not going to change the necessity
of having Xref stripping capability in the software.  There are going
to be legacy peers around for a long time, and a SHOULD NOT permits
persistence in the behavior even for peers updated to conform.

And as discussed above, the change isn't going to save that many CPU cycles,
and could actually waste them depending on how you decided to store and
strip Xref on the way out to your N peers, and insert it on the way out
to M clients.

A note about how Xref leaks information might be something for USEAGE.
But peers can gather a lot more information about the internals of
your systems in other ways.

Date: Sat, 25 Jun 2005 15:58:06 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Sender: [email-masked]

On Sat, 25 Jun 2005, Bruce Lilly wrote:
> >   2822 msg-id syntax has to be fixed for News, NNTP, and URLs
> 
> The WG charter gives no authorization to modify RFC 2822.

Nobody is talking about modifying RFC 2822.  The extent to which 2822
syntax applies to news, and specifically what restrictions might have to
be imposed to make it work for news, is legitimately our decision.  An
arbitrary 2822-conforming message is *not* a valid news article, so there
*are* added restrictions; the only questions are what they are and how
they are best explained. 

Quite plausibly some of them are best explained by using text and/or ABNF
which is a rewritten version of parts of 2822 (although this needs to be
done with care to ensure that our version doesn't legalize something that
2822 outlaws). 

Please stop playing this broken record, Bruce. 

                                                          Henry Spencer
                                                       [email-masked]

Date: Sat, 25 Jun 2005 15:38:03 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Sender: [email-masked]

On Sat, 25 Jun 2005, Bruce Lilly wrote:
> > I do object to this.
> 
> Because...?  (N.B. the BCP 14 meaning of SHOULD NOT is a recommendation,
> not an imperative).

By author's intent and wide community consensus, SHOULD NOT is an
*emphatic* recommendation, not to be disregarded without specific and
important reasons.  BCP 14 itself calls for using its keywords sparingly,
adding:  "they must not be used to try to impose a particular method on
implementors where the method is not required for interoperability". 

The correct way to make a mild recommendation is not to use a keyword at
all.  (And as noted in previous mail, this particular recommendation would
be commanding the tide not to come in, so it seems pointless.)

                                                          Henry Spencer
                                                       [email-masked]

Date: Sat, 25 Jun 2005 15:29:55 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Sender: [email-masked]

On Sat, 25 Jun 2005, Russ Allbery wrote:
> >   Because the Xref field is only of use to a particular system, it
> >   SHOULD NOT be transmitted to other systems.
> 
> I do object to this.

Concur.  It's a nice idea in abstract, but the real world has never worked
that way, and it's time to stop pretending that it does.  (Even B News,
despite its alleged close relationship to RFC 1036, never did that.)

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sat, 25 Jun 2005 11:52:38 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:

> That does not affect transmission between servers.  What happens between
> a server and a client is a separate matter.

I already covered this too.  We're really just repeating ourselves at this
point.

> On Sat June 25 2005 14:21, Russ Allbery wrote:
>> I object to requiring Xref be removed from outgoing articles

> Fine.  No proposal for a *requiring* removal has been made, as far as I
> know.

I object to saying that removal of Xref is a SHOULD for servers.

(Was it really necessary for you to make me say that?  Was not your
message and this message really a waste of WG attention and bandwidth?
I'm going to now assume that people understand my position on this
proposal without these sorts of small clarifications.)

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Russ Allbery <[email-masked]>
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Sat, 25 Jun 2005 14:35:50 -0400
Cc: [email-masked]
Sender: [email-masked]

On Sat June 25 2005 14:21, Russ Allbery wrote:
> 
> Bruce Lilly <[email-masked]> writes:
> 
> > Since this is local data, how does interoperability apply?
> 
> I already answered that.  I'm fairly certain that at least some news
> readers use this data.

That does not affect transmission between servers.  What happens between
a server and a client is a separate matter.

> I object to requiring Xref be
> removed from outgoing articles

Fine.  No proposal for a *requiring* removal has been made, as far as I know.

To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sat, 25 Jun 2005 11:21:53 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:

> Since this is local data, how does interoperability apply?

I already answered that.  I'm fairly certain that at least some news
readers use this data.  There's also Xref-slaving, of course.

I think I've said everything I have to say.  I object to requiring Xref be
removed from outgoing articles for the reasons previously stated, mostly
because it's not worth the trouble and the justifications are not
compelling enough to change existing software, and do not find Bruce's
counter-arguments convincing.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Russ Allbery <[email-masked]>
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Sat, 25 Jun 2005 14:17:59 -0400
Cc: [email-masked]
Sender: [email-masked]

On Sat June 25 2005 13:36, Russ Allbery wrote:
> 
> Bruce Lilly <[email-masked]> writes:
> > On Sat June 25 2005 12:20, Russ Allbery wrote:
> 
> >> No, Xref is very widely used in practice.
> 
> > Perhaps I was unclear.  I didn't imply that Xref was not used, perhaps
> > even widely so.  Simply that *best* practice is to put local data in a
> > local database.
> 
> This depends on what definition you're using for "best."

In terms of the principle of keeping strictly local (and therefore variable
between systems) data separate from invariant data (the article content).

> This is 
> certainly not the most interoperable practice.

Since this is local data, how does interoperability apply?

> Because putting an Xref header in the message is how much Usenet software
> works, and I don't see a compelling reason to change that.

Security is one issue.  Sending local data exposes that local data,
constituting unauthorized data disclosure.  The threat consequences
may include a threat action of inference, specifically traffic
analysis.  See the security glossary, BCP 36 (a.k.a. RFC 2828).  In
common terms, it's a violation of the security principle of restricting
disclosure of data to those with a "need to know".

Although security is not mentioned (at all!) in RFC 1036, the principle
of not exposing local data espoused in the RFC 1036 text is sound.

To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sat, 25 Jun 2005 10:36:29 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:
> On Sat June 25 2005 12:20, Russ Allbery wrote:

>> No, Xref is very widely used in practice.

> Perhaps I was unclear.  I didn't imply that Xref was not used, perhaps
> even widely so.  Simply that *best* practice is to put local data in a
> local database.

This depends on what definition you're using for "best."  This is
certainly not the most interoperable practice.

>> I do object to this.

> Because...?  (N.B. the BCP 14 meaning of SHOULD NOT is a recommendation,
> not an imperative).

Because putting an Xref header in the message is how much Usenet software
works, and I don't see a compelling reason to change that.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Russ Allbery <[email-masked]>
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Sat, 25 Jun 2005 13:29:30 -0400
Cc: [email-masked]
Sender: [email-masked]

On Sat June 25 2005 12:20, Russ Allbery wrote:
> 
> Bruce Lilly <[email-masked]> writes:

> > Best current practice seems to be to keep track of local data in a
> > separate database rather than in a non-transmitted addition to the
> > article header.
> 
> No, Xref is very widely used in practice.

Perhaps I was unclear.  I didn't imply that Xref was not used, perhaps
even widely so.  Simply that *best* practice is to put local data in a
local database.

> My guess would also be that 
> some news readers want it in the article proper in order to mark
> crossposts already read appropriatley.

I have no problem with communicating such information (however it is
locally stored) via an appropriate mechanism (e.g. overview data).

> > I therefore propose that the following text (subject to discussion of
> > course) be added to the usefor draft Xref text:
> 
> >   The optional Xref field has been used as a place to store
> >   cross-reference information of use to a local system.  Other
> >   methods of handling such metadata include separate databases.
> 
> s/has been/may be/.  Other than that, I don't particularly mind this.

I'd prefer not to confuse readers between the several meanings of
"may".  How do you feel about "might"?
 
> > That is to clarify the semantics of the field.  When we get to the usepro
> > document, text similar to the following could appear there:
> 
> >   Because the Xref field is only of use to a particular system, it
> >   SHOULD NOT be transmitted to other systems.
> 
> I do object to this.

Because...?  (N.B. the BCP 14 meaning of SHOULD NOT is a recommendation,
not an imperative).

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Sat, 25 Jun 2005 13:12:36 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 24 2005 23:34, Frank Ellermann wrote:

>   2822 msg-id syntax has to be fixed for News, NNTP, and URLs

The WG charter gives no authorization to modify RFC 2822.

Facts:
o CID and MID URIs (RFC 2392) seem to work fine.
o there are mechanisms other than syntax changes (e.g.
  implementation notes) to address specific issues.
o NNTP and URIs are handled by other groups.  Not our problem.

Wording: "fixed" implies that something is "broken", but no
convincing case has been made that 2822's definition is what
is broken.

[...]
> That won't fly with Bruce and Charles for different reasons,
> Bruce wants the 2822 syntax

I repeat: we have no authority to modify RFC 2822. This has nothing
to do with "wants"; it's a simple matter of proper IETF procedure.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047: Path field delimiters and components
Date: Sat, 25 Jun 2005 12:41:59 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 24 2005 12:44, Charles Lindsey wrote:
> 
> In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:
> 
> >Assigned ticket #1047 to this issue.
> 
> >> Specific change to USEFOR -04 text: remove colon from path-identity
> >> ABNF production.
> 
> I oppose this. It has been raised on the WG several times over the years,
> and it has always been concluded that IPv6 addresses were going to arise
> sooner or later and therefore should be accomodated (though possibly
> discouraged as the proposed USEPRO text already does).

References to all of the discussions (you claimed "always") and
Chair-specified consensus, please.  [Frankly, your continued habit of
making unsupported assertions is very wasteful of limited WG
resources.  Please stop it.]

> Yes, there is a theoretical possibility of a backward incompatibility

There is nothing "theoretical" about it; you know this because it has
been discussed at length, with supporting documentation -- we know for
a fact that B news, C news, and INN treat colon as a delimiter, and
that reassignment will therefore unquestionably break backward
compatibility.  It is disingenuous of you, and frankly insulting to
others, for you to categorize this as "theoretical".

> I have already proposed a wording (in reply to John) for inclusion in
> USEFOR to advise care when choosing <path-identity>s.

That wording is inadequate.
 
> [1] Suppose site A still regards ":" as a <path-delimiter>. It feeds site
> B which has always chosen to call itself "beef". In the meantime,
> post-new-standard, some new site C has chosen to call itself "dead:beef"
> (not a wise choice, but I am not allowed to the the 'i' word :-) ).  Now,
> A will fail to forward to B articles which have previously passed through
> "dead:beef". Note that this is much less likely if use of ":" is confined
> to IPv6 addresses. There is also a less disruptive scenario [2].

Sigh (again).  Charles, do you honestly fail to recognize, after it has
been pointed out and after Frank has also mentioned "cafe" etc., that
"dead" and "beef" are composed entirely of characters representing
hexadecimal digits (and moreover DEADBEEF is a well-known set of such
heaxadecimal digits) and therefore might well appear in an IPv6 address
literal?  Clearly "dead", "beef", and any of (16^^4 = 65536) - 2 other
4-character combinations of hexadecimal digits (not to mention 256
combinations of two hexadecimal digits) which may appear in IPv6 address
literals can conflict with unqualified (in the DNS sense) names.  Do you
really fail to realize that obvious fact, or are you intentionally wasting
WG time?  Really, I want to know if it's the former case so that I know
to explain things in minute detail.

To: [email-masked]
Subject: Re: Feedback on proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sat, 25 Jun 2005 09:39:36 -0700
Sender: [email-masked]

Steve Walker <[email-masked]> writes:

> A good protocol will be flexible to allow any kind of information.

Textual Usenet requires a significant amount of metadata, and handling
that data in an interoperable fashion requires structures that may well be
unnecessary for binary posts.  You're objecting, in part, to the addition
of rules that are *required* in order to do internationalization properly,
and that, as I have already demonstrated, are already used in the majority
of posts to text Usenet (and growing).  In other words, the data that
you're presenting is evidence of exactly my point; the schism between the
needs of the binary Usenet and the text Usenet is deep and growing fast.

See also much of the news.software.nntp discussion over the past two
years, including the discussions of yEnc, multipart binary reassembly, and
the observation that message IDs in their current form really don't make a
lot of sense for binary traffic.

I know that the binary folks don't want to use MIME.  The text folks have
to if we're going to get anywhere with interoperable non-English support.
The standard doesn't require MIME; that's about as much of a concession as
I think you'll ever get out of the people, like me, who care about text.

> It appears to me that what has been written so far is just for the
> minority.

That rather depends on how you measure the minority.  Mostly-text or
only-text is the *vast* majority of Usenet installations; I bet there are
under 100 servers in the entire world that are actually trying to carry a
complete binary feed, and that number similarly is dropping due to the
bandwidth and specialized management requirements.  The significant
majority of Usenet sites at this point (and I'm fairly confident of this
statement -- as a server implementor, I talk to a lot of them) are
carrying either only a handful of binary groups for legacy reasons or no
binary groups at all.

The majority of news readers, both by number and by deployment count, find
dealing with MIME much easier than dealing with yEnc.  For occasional
binary attachments to otherwise text groups (such as for private
hierarchies, mailing list gateways, and then like), MIME is a
significantly better protocol and is considerably more interoperable than
either yEnc or uuencode.

The needs for binary groups are different.

I don't particularly want to *break* binary Usenet, but the problems that
I care about fixing are text problems.  If you want to champion things
that binary Usenet needs, that's great, but you're going to need to not
stomp on the things that text needs in the process.

> My point is, that if it's agreed that mime is a good format then the
> word 'SHOULD' be used and not 'MUST".

As other people have already pointed out, this is the case already.

News *readers* have to support MIME because they have to be able to read
MIME messages, as there will be quite a lot of them.  This means that pure
binary download clients will likely not be compliant *news readers*
according to this standard.  I'm not sure if this is a serious enough
problem to worry about; they're not really trying to be news readers in
the sense that we use in these documents.

> The same thing for yEncoded articles.

I believe yEnc is so poorly designed as to be unstandardizeable by the
IETF in its current form.  I came to that conclusion after spending a
bunch of time trying to prove the opposite.  If you can do better, please,
by all means, go for it.  Just because something is widely used doesn't
mean that it's documented well enough that one can standardize it.  In
practice, there are serious interoperability problems between different
yEnc implementations.

Certainly, given the widespread use of yEnc, it would be worth at least
documenting the existing protocol in an informational RFC.  If you're
willing to do that work, I certainly won't stand in your way.  The
designer appears to have no interest in working with a standards body.

>> Now that I've realized that this thread was started by someone who
>> pretty much cares only about the binary groups, this all makes much
>> more sense.

> That certainly not true.  I care about all of Usenet.

I'm sorry.  That wasn't the right thing for me to say.  I don't know what
you care about, and shouldn't be talking as if I did.  The concerns that
you presented are primarily binary-related concerns.

> Why make mime processing required and not recommended?  Why forbid
> non-encoded binary data?

Because the point of standards is to get different implementations to
interoperate properly with each other, and both of those constraints are
required in order to do so.  Note that not even yEnc is unencoded binary
data.  It *is* a different CTE, and really should be registered as a CTE,
but there's a limit to what a standards body can do if the people
designing protocols refuse to standardize them.

You *have* to encode binary data before putting it on Usenet because the
existing Usenet software is not clean in nul handling and relies on line
termination for finding the end of the article.

> I'd also like to point out that no one person here has commented on the
> suggestion of requiring article numbers to be 64 bit.

That's because you're in the wrong working group.  This is an NNTP issue,
and you're about three years late to raise it (even apart from the fact
that we talked about this and there are reasons that I won't go into for
why it wasn't changed).  This working group doesn't deal with article
numbers.

> Here we have a real problem that is bound to cause problems in the near
> future, yet since it won't effect text users from a long time it's
> ignored.

It wasn't ignored.

> While it may seem that I'm the binary user advocate, it's just because
> from what I've read, It's a group of users that have been completely
> ignored by this proposal.  I just don't want to see further seperation
> between user bases.

I beat that drum for years.  I've become convinced that it's a lost cause.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Sat, 25 Jun 2005 12:22:46 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 24 2005 11:55, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >On Wed June 22 2005 15:18, Charles Lindsey wrote:
> >> 
> >> In RFC 2822 an <id-right> is a string of characters satisfying some ABNF.
> 
> >The grammar production named id-right is an ABNF rule name, yes.
> 
> >> It has no semantics attached to it (except that a `\` in a <quoted-pair>
> >> is "invisible").
> 
> >No, the semantics are "the domain name (or a domain literal IP address)".
> 
> I do not see where RFC 2822 says that.

It says so (in those words) in the part which I quoted from 2822 and which
you have snipped.  John Stanley seems to have identified some problem
related to editing (unless of course you were exaggerating about "an hour
or so" (N.B. there was a "smiley" notation indicating sadness, but none
indicating intended humor or exaggeration)); may I suggest the vi editor,
which John also recommended, and with which you can quickly and efficiently
locate the text by typing (each line ends with a carriage return):

  vi rfc2822.txt
  /domain name (o

You can exit vi by typing the two characters (no carriage return required)
  ZZ

Alternatively, you can find a web interface to RFCs via the RFC-Editor web
site (http://www.rfc-editor.org), and you can use your browser's text
search mechanism to find the text within the RFC 2822 document.

> It is essentially the same as we have just been agreeing for the Path
> header.

We seem not to be agreed.  There is some agreement on "not an IP address",
but you seem to want it both (incompatible) ways, "not an IP address" and
"an IP address".  We are not agreed on "opaque label" or "public name",
which has not been discussed much.  In any event, Path is not specified
in RFC 2822, and is irrelevant to discussion of msg-id and 2822.

> All of <path-identity>, <id-left> and <id-right> are "opaque 
> labels", whether you initially construct them out of domain-names or
> domain literals, or whatever else.

Please cite RFC 2822 text supporting your "opaque labels" assertion.
Please also cite specific RFC 2822 text, as specifically requested
earlier, that supports your contention that the semantics of msg-id
has been changed.
 
> >> No, Mark made his suggestion (in the NNTP list, I believe)
> 
> >Mark's suggestion was made here, and is archived in this WG's mailing list
> >archive at
> >http://www.imc.org/ietf-usefor/2003/Feb/0493.html
> 
> >> in the belief 
> >> that he was describing a temporary problem that would be fixed in future
> >> implementations.
> 
> >That is a false assertion; Mark specifically referred to a transport
> >protocol constraint, as his text clearly shows.
> 
> Mark may well have said that on this list when he was a member of it.

Charles, there is no "may" about it.  I have provided a reference to the
archived mailing list message as well as a quotation from that message
which clearly indicate:
1. the suggestion was made here
2. It was not "describing a temporary problem..."

> But 
> he most certainly raised essentially the same point on the NNTP list

If it was the same point, then it still was not "describing a temporary
problem".  And whether or not the issue was raised on the NNTP list, that
is irrelevant to the statement about what he said on *this* list, which
was quoted (several times, including multiple references) prior to your
assertions about what was said and where it was said.

To: [email-masked]
Subject: Re: NEW: USEFOR -04 3.2.11 Xref vs. transport
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sat, 25 Jun 2005 09:20:04 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:

> RFC 1036 says, regarding the Xref field:

>     This is only of value to the local system, so it should not be
>     transmitted.

> There is no corresponding text in the usefor draft version -04.

That's because that sentence describes behavior that very few people
actually follow.  It's not a bad idea, but patching the Xref header out of
outgoing articles seems to be considered too much pain for not nearly
enough gain (that's certainly my opinion).

> Best current practice seems to be to keep track of local data in a
> separate database rather than in a non-transmitted addition to the
> article header.

No, Xref is very widely used in practice.  My guess would also be that
some news readers want it in the article proper in order to mark
crossposts already read appropriatley.

> I therefore propose that the following text (subject to discussion of
> course) be added to the usefor draft Xref text:

>   The optional Xref field has been used as a place to store
>   cross-reference information of use to a local system.  Other
>   methods of handling such metadata include separate databases.

s/has been/may be/.  Other than that, I don't particularly mind this.

> That is to clarify the semantics of the field.  When we get to the usepro
> document, text similar to the following could appear there:

>   Because the Xref field is only of use to a particular system, it
>   SHOULD NOT be transmitted to other systems.

I do object to this.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Date: Sat, 25 Jun 2005 11:56:26 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 24 2005 11:43, Charles Lindsey wrote:

> Yes, it seems we have been discussing something tht RFC 2822 had already
> resolved.
> 
> As regards the wording you just proposed above, I would still prefer it to
> be like:
> 
>     Software MAY accept dates with unknown timezones and SHOULD treat ...
> 
> Of course, whether some particular timezone is "unknown" is still
> dependent upon whether or not you chose to accept all of the <obs-zone>.
> 
> As for the syntax and the rest of the text, I am happy with what Harald
> proposed at the start of this thread, namely removing UT and just leaving
> GMT as the only MUST accept / MUST NOT generate item.

Your comments seem to have been focused on use in the Injection-Date
field rather than on the legacy fields Date and Expires, which UAs
will need to be able to recognize, including the still-used alphabetic
abbreviations, which of course also appear in archives.

A ticket has been opened so that you can make a specific suggestion that
would address your concerns w.r.t. that specific new field without
adversely affecting the legacy fields.  Feel free to take advantage of
that opportunity.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1043 "header": Suggested resolution
Date: Sat, 25 Jun 2005 11:52:39 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 24 2005 11:34, Harald Tveit Alvestrand wrote:

> --On fredag, juni 24, 2005 09:25:38 -0400 Bruce Lilly <[email-masked]> 
> wrote:
> 
> >> NEW:
> >>
> >>   NOTE: Older News specifications used the term "header" as a synonym
> >>      for what [RFC2822] calls "header field". This document follows
> >>      the terminology of RFC 2822.
> >
> > I suggest s/[RFC2822] calls/[RFC2822] and MIME call/ and I suggest
> > adding "and MIME [RFC2045], [RFC2046], [RFC2047], [RFC2049], [RFC2231]"
> > to the last sentence.  I'd also like to see an informative reference to
> > FYI 18 added, but for the sake of conciseness of the document, I can
> > live without it.
> 
> I'd consider such a change to be at best useless and at worst harmful.
> 
> If there was ever a conflict discovered between MIME and 2822 about what a 
> "header field" was, this would make our "note" ambiguous.
> And if there is no conflict, the change adds no information.

The information added relates to MIME-part headers and the fields contained
therein, as defined in RFCs 2045 and 2046 and used in other MIME
specifications, which as noted in 2822 itself (sect. 1.1) "are outside of
the scope of" 2822.  Although there is no conflict, there is additional
information.  [rather than asserting that a specific suggested change,
which can be assumed to have been suggested for a reason, is "useless",
"harmful", "ambiguous", and "adds no information", I suggest that in future
a more professional and productive approach would be to ask a question if
you are unclear about the reason for the suggestion] 

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Sat, 25 Jun 2005 11:44:06 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 24 2005 11:31, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

> >  The purposes listed in -04 3.1.6 impose first and
> >foremost a uniqueness requirement.
> 
> It doesn't explicitly say so, but it is evident that it is necessary.

OK; we're agreed that there is a uniqueness requirement. N.B. requirement
as in "necessary", not "wouldn't it be nice if...".
 
> >  That in turn requires either
> >a hierarchical system (like DNS) or a central authority (like the
> >UUCP mapping project was supposed to be, though it never worked
> >well).
> 
> But that does not follow at all. Usenet has been working quite happily for
> many years now just by admins using their common sense when choosing
> <path-identity>s. I am not aware of serious clashes that have occurred,
> and if they have then they have been sorted out amicably (because it is in
> everybody's interest to do so).

While you might not be aware of problems, they certainly exist. Although
the UUCP mapping project has good intentions and was intended to avoid
clashes, it failed to do so.  I happen to know for a fact (and have
previously mentioned here) that there have been unresolved duplicates;
one that I can recall off the top of my head even after a decade is
multiple sites claiming the name "wolf".  I am not aware that the duplicates
were ever resolved.  Do you have evidence to support your claim that "they
have been sorted out amicably"?

In the general case, would you care to explain in specification language
(as opposed to hand-waving) precisely how a uniqueness requirement can
be met in a distributed system without a hierarchical DNS-like system
or a central authority?

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Sat, 25 Jun 2005 11:35:42 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 24 2005 11:04, Charles Lindsey wrote:
> 
> In <[email-masked]> Russ Allbery <[email-masked]> writes:
> 
> >You have to treat Path identities as opaque labels; anything else gets
> >into way too much work to specify and implement, IMO.
> 
> Absolutely so.
> 
> But for human readers, it is useful to be able to regard them as
> domain-names and/or IP addresses (as the case may be) and to look them up
> in the DNS or in whois databases in order to determine the (alleged) route
> an article took.

We need to specify the semantics of a "path-identity" as one of:
o an opaque label
o a public name
o domain names, IP-addresses, etc.

It is unsatisfactory, because it is self-inconsistent, to give contradictory
semantics.

> Also, in order to detect MISMATCHes, you will likely need to know whether
> the IP address the NNTP packet

NNTP is not the only transport mechanism.

> came from agrees with what the 
> <path-identity> in the header says.

Irrelevant to the discussion at hand, which is interpretation of the
"path-identity" *as* an IP address, not interpretation of an IP address
obtained from another source.

> >....  I'm not seeing a clamor
> >from IPv6 sites wanting to put address literals into the Path header.
> 
> I think that is because usage of IPv6 is still much less than of IPv4. But
> clearly it is going to take off one of these days, probably in the Pacific
> region first, because that is where the supply of IPv4 addresses seems to
> be running out. And then you will start to see them in <path-identity>s;
> but probably not for a couple of years or so.

Russ' complete statement was:

 Yeah, I'd like to see some support for that too. �I'm not seeing a clamor
 from IPv6 sites wanting to put address literals into the Path header.

in response to my call for you to support your assertion/ ad-hominem remark:

 > On Wed June 22 2005 17:26, Charles Lindsey wrote:
 >> So you would rather throw out a much needed facility than put up with a
 >> rather improbable minor inconvenience? Please come and join us in the
 >> Real World for a change.

 > Please support your "much-needed" assertion.

You still have not provided any substantive support for your assertion;
"I think", "probably", "seems to be", etc. are hardly convincing, and
contain no objective data to support the "much needed" assertion.  Nor
does it provide any evidence to support "a clamor from IPv6 sites to put
address literals into the Path".  Would you care to retract your "much
needed" assertion and/or admit that there is no "clamor"?

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: NEW: USEFOR -04 3.2.11 Xref vs. transport
Date: Sat, 25 Jun 2005 11:14:20 -0400
Sender: [email-masked]

RFC 1036 says, regarding the Xref field:

    This is only of value to the local system, so it should not be
    transmitted.

There is no corresponding text in the usefor draft version -04.  Best
current practice seems to be to keep track of local data in a separate
database rather than in a non-transmitted addition to the article
header.  I therefore propose that the following text (subject to discussion
of course) be added to the usefor draft Xref text:

  The optional Xref field has been used as a place to store
  cross-reference information of use to a local system.  Other
  methods of handling such metadata include separate databases.

That is to clarify the semantics of the field.  When we get to the usepro
document, text similar to the following could appear there:

  Because the Xref field is only of use to a particular system, it
  SHOULD NOT be transmitted to other systems.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Steve Walker <[email-masked]>
Subject: Re: Feedback on proposal
Date: Sat, 25 Jun 2005 10:35:24 -0400
Cc: [email-masked]
Sender: [email-masked]

On Sat June 25 2005 09:44, Steve Walker wrote:

> My point is, that if it's agreed that mime is a good format then 
> the word 'SHOULD' be used and not 'MUST".

"MUST" applies to implementations that claim conformance to the
specification.

> The same thing for  
> yEncoded articles.

Unfortunately, there's no stable specification to refer to.  No
IETF community review.  Probably no adequate analysis for security
issues.

> Why make hard absolute rules that can not be  
> followed by a large number of users.

As separately mentioned, packages such as metamail make MIME
available.  And there are no "hard absolute rules"; implementations
not claiming conformance are free to do as they please.  Of course,
they might not interoperate well with fully conforming implementations.
 
> You are writing rules the will certainly break existing 
> practices.

Harmful practices should be deprecated, and in some cases prohibited
(by implementations claiming conformance).

> I have not suggested anything that would harm any  
> group of users.

Frankly, you haven't suggested much that's specific enough to
comment on.

> Why make mime processing required and not  
> recommended?

That's "support", not "processing".  And "required" only for
implementations that wish to claim conformance to the specification.
The reason is to ensure interoperability, e.g. for internationalization
of text in header fields.

> Why forbid non-encoded binary data?

It's not backward compatible with the message format which has been used
since RFC 850.  It's not backward compatible with widely deployed transport
and access mechanisms (NNTP, IMAP) which are compatible with and based on
the existing specifications (RFCs 1036 and 822 among others).

> It seems to me   
> that neither of these things are harming Usenet right now

This WG has carefully considered both issues and determined otherwise.

> and  
> they are not likely to actually change because of a new "standard".

Maybe, maybe not.  There is no enforcement of Internet specifications
other than peer pressure based on interoperability considerations.
Our task is to produce a specification that takes backward compatibility
with standards-conforming deployed implementations and *best* current
practices into account.
 
> I'd also like to point out that no one person here has commented 
> on the suggestion of requiring article numbers to be 64 bit. 
> Here we have a real problem that is bound to cause problems in 
> the near future, yet since it won't effect text users from a long 
> time it's ignored.

I believe that you are mistaken.  Our current priority is the "USEFOR"
specification, and it currently provides for article identifiers (in
the Xref header field) as:

   xref            =  "Xref:" SP [FWS] server-name
                      1*( FWS location ) [FWS] CRLF

   server-name     =  path-identity

   location        =  newsgroup-name ":" article-locator

   article-locator =  1*( %x21-27 / %x29-3A / %x3C-7E )
                      ; US-ASCII printable characters
                      ; except '(' and ';'

      NOTE: The traditional form of an article-locator (as required by
      [NNTP]) is a decimal number, with articles in each newsgroup
      numbered consecutively starting from 1.

RFC 2234 ABNF specifies what the "1*" signifies.  Briefly, it means
that the length of the "article-locator" has *no* upper bound (in
practice there is an upper bound of 996 characters minus the length of
the relevant newsgroup name due to line length limits).  That's
significantly larger that what can be represented in a 64-bit integer.
Implementations are free to use whatever mechanisms they deem appropriate
to meet the specification.  An integer representation limited to merely
64-bit numbers (you neglected to state whether that's signed or unsigned,
BTW) might not be a good design choice...  So now you have a comment on
that particular issue.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: Ned Freed <[email-masked]>
Subject: gzip/deflate compression/encoding
Date: Sat, 25 Jun 2005 09:59:00 -0400
Cc: [email-masked], [email-masked], Dan Kohn <[email-masked]>
Sender: [email-masked]

On Fri June 24 2005 17:36, Ned Freed wrote:

> FWIW, I would be happy to revive the draft if there is any interest.
> The last time I tried this the silence was fairly deafening, however.

It probably makes sense, though as noted last time, there may be some
benefit to decoupling compression and encoding binary->8-bit:
o some binary content, e.g. http://users.erols.com/blilly/mparse/tm35
  body content, won't compress well and may in fact be expanded by gzip
  "compression".
o combining compression and encoding can yield benefits even for 8-bit
  content encoded to a 7-bit transport. E.g. given a 1 MB 8-bit text
  source, compression will typically yield approx. 500 kB binary
  compressed data.  Base64 encoding will expand that by about 26:19
  (4/3 * 78/76) yielding about 684 kB which is obviously smaller than
  what encoding alone would yield.
o similarly, binary content could be compressed and encoded for 7bit
  transport, yielding a size reduction compared to encoding alone.
o Continuing along the same lines, large 7-bit content could be
  compressed and encoded for 7-bit transport to yield an overall size
  reduction, e.g. if one part of the transport path is known to be a
  low-bandwidth or size-constrained bottleneck (typically 1 MB -> 684 kB
  as detailed above).
o compression could be used even where binary transport is available
  (though it's a stretch to call that an "encoding") without the need
  to hide the nature of the original media type by wrapping the compressed
  data in a label identifying the data as "application/x-gzip" or
  "application/octet-stream".

I found out yesterday that 8BITMIME works much more widely than expected,
so this probably should be handled separately from the USEFOR WG (which
is already behind schedule with its work items).  Accordingly, I've
copied the ietf-822 list and set Reply-To to point there.

> I should also mention that Dan and I saw no problem, and indeed some benefit,
> from combining the two drafts.

If you think I can be of assistance, feel free to contact me off-list(s).

Date: Sat, 25 Jun 2005 08:44:43 -0500
From: Steve Walker <[email-masked]>
To: Russ Allbery <[email-masked]>
CC: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Russ Allbery wrote:
> It's probably worth noting that the binary Usenet is essentially an
> extended P2P network at this point and really bears little and decreasing
> relationship to text Usenet or any other application of the protocol.

Usenet was designed to allow people to share information.  It is 
the original p2p network.  What people share isn't important.  It 
doesn't matter if it's a sci-fi story, a letter of protest, or a 
picture of a train.  It's people sharing information.  The 
protocol shouldn't tailored for a small group, but should be 
broad and flexible.

> After a lot of discussions on news.software.nntp about this, it's not
> completely clear to me that the binary Usenet is even well-served by using
> the same article format as everything else.  Their needs and their

A good protocol will be flexible to allow any kind of 
information.  It appears to me that what has been written so far 
is just for the minority.

My point is, that if it's agreed that mime is a good format then 
the word 'SHOULD' be used and not 'MUST".  The same thing for 
yEncoded articles.  Why make hard absolute rules that can not be 
followed by a large number of users.

> Now that I've realized that this thread was started by someone who pretty
> much cares only about the binary groups, this all makes much more sense.

That certainly not true.  I care about all of Usenet.  This 
proposal only seems to only care about a small subset of users. 
You are writing rules the will certainly break existing 
practices. I have not suggested anything that would harm any 
group of users.  Why make mime processing required and not 
recommended?  Why forbid non-encoded binary data?  It seems to me 
that neither of these things are harming Usenet right now and 
they are not likely to actually change because of a new "standard".

I'd also like to point out that no one person here has commented 
on the suggestion of requiring article numbers to be 64 bit. 
Here we have a real problem that is bound to cause problems in 
the near future, yet since it won't effect text users from a long 
time it's ignored.

While it may seem that I'm the binary user advocate, it's just 
because from what I've read, It's a group of users that have been 
completely ignored by this proposal.  I just don't want to see 
further seperation between user bases.

Steve.

To: [email-masked]
Subject: Re: Feedback on proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 24 Jun 2005 21:35:12 -0700
Sender: [email-masked]

Ned Freed <[email-masked]> writes:

> FWIW, I would be happy to revive the draft if there is any interest.
> The last time I tried this the silence was fairly deafening, however.

> I should also mention that Dan and I saw no problem, and indeed some
> benefit, from combining the two drafts.

It's probably worth noting that the binary Usenet is essentially an
extended P2P network at this point and really bears little and decreasing
relationship to text Usenet or any other application of the protocol.
After a lot of discussions on news.software.nntp about this, it's not
completely clear to me that the binary Usenet is even well-served by using
the same article format as everything else.  Their needs and their
standard usage patterns are almost completely different.  (For example,
you could drop threading completely from the binary Usenet and people
would hardly even notice.)

Now that I've realized that this thread was started by someone who pretty
much cares only about the binary groups, this all makes much more sense.
I would expect reactions like this from most people who think that the
binary groups are the primary point of Usenet, even though they're being
carried on a fast-decreasing number of servers and have become somewhat of
a speciality pay market.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date:  Sat, 25 Jun 2005 05:34:47 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 41
Sender: [email-masked]

Bruce Lilly wrote:

>> It has no semantics attached to it (except that a `\` in a
>> <quoted-pair> is "invisible").

> No, the semantics are "the domain name (or a domain literal
> IP address)".
[...]
> a msg-id has had a domain name or domain literal on the RHS
> since RFC 822 (prior to which the RHS was a host name, as
> that was before DNS).  RFC 2822 has not changed the semantics, 
> contrary to your unsupported assertion

Obviously there are strong disagreements about what a msg-id is
both semantically and syntactially:

Bruce + Frank + Sebastian + STD 11 + 1036 + s-o-1036:
  RHS is domain or domain-literal

Charles:
  RHS is what the fixed 2822 syntax allows for <id-right>

Charles + Frank + Alexey + 1036 + s-o-1036 + nntp-27:
  2822 msg-id syntax has to be fixed for News, NNTP, and URLs

Bruce:
  2822 msg-id syntax is good enough

So for these two details the rough consensus is IMHO clear.
Putting it together the consequence _should_ be also clear:

  msg-id = "<" id-local "@" id-domain ">"

That won't fly with Bruce and Charles for different reasons,
Bruce wants the 2822 syntax, Charles wants his own semantics.

Both positions are showstoppers from my POV, i.e. I prefer to
dissolve this WG without any result instead of these positions.

                         Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:21511
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1047: Path field delimiters and components
Date: Fri, 24 Jun 2005 16:44:09 GMT
Lines: 49
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>Assigned ticket #1047 to this issue.

>> Specific change to USEFOR -04 text: remove colon from path-identity
>> ABNF production.

I oppose this. It has been raised on the WG several times over the years,
and it has always been concluded that IPv6 addresses were going to arise
sooner or later and therefore should be accomodated (though possibly
discouraged as the proposed USEPRO text already does).

Yes, there is a theoretical possibility of a backward incompatibility
(i.e. you can construct possible but extremely unlikely scenarios). Even
if a case did occur, it would only affect certain sites, and could easily
be resolved between the sites affected. The worst that could happen is
that a site might find that one of its peers was failing to supply certain
articles[1].

I have already proposed a wording (in reply to John) for inclusion in
USEFOR to advise care when choosing <path-identity>s.

[1] Suppose site A still regards ":" as a <path-delimiter>. It feeds site
B which has always chosen to call itself "beef". In the meantime,
post-new-standard, some new site C has chosen to call itself "dead:beef"
(not a wise choice, but I am not allowed to the the 'i' word :-) ).  Now,
A will fail to forward to B articles which have previously passed through
"dead:beef". Note that this is much less likely if use of ":" is confined
to IPv6 addresses. There is also a less disruptive scenario [2].

[2] Suppose A also feeds site C. It receives an article which has already
passed through C once, with the <path-identity> "dead:beef".  It will not
recognize that <path-identity>, and so it will pass it to C again (which is
wasteful but not harmful). (but you still may well ask how A came to
register C as a peer with the name "dead:beef", given that is not a
valid <path-identity> from the POV of A).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Steve Walker <[email-masked]>
Subject: =?iso-8859-1?q?R=E9=3A_Feedback_on?= proposal
Date: Fri, 24 Jun 2005 17:55:58 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 24 2005 16:23, Steve Walker wrote:

> In the real world, if all clients must support mime 
> how do non-mime articles get posted.

The same way as always. A plain text message in US-ASCII is just
fine as a non-MIME message.

Incidentally, whether you know it or not, your Mozilla Thunderbird
client already supports MIME (modulo bugs):

Message-ID: <[email-masked]>
Date: Fri, 24 Jun 2005 15:23:26 -0500
From: Steve Walker <[email-masked]>
User-Agent: Mozilla Thunderbird 1.0.2-1.3.3 (X11/20050513)
X-Accept-Language: en-us, en
MIME-Version: 1.0

If you read a MIME message, you might not be aware that it is
MIME, unless you're using a non-MIME client.  For example,
you might not realize that the subject of this message is
MIME-encoded (RFC 2047/2231) unless you take special steps
to look at the raw message content.  Likewise for the body text.
MIME is the only current method which provides for standards-
conforming non-ascii content with language tagging for text (see
RFC 2277).  Maybe that doesn't concern you personally, but there
is a world full of people with names such as Claus Andr� F�rber
who would like to have their names displayed properly.
 
> I really don't see software  
> developers who design clients that allow non-mime posting to 
> bother with fully supporting reading mime.

They don't have to.  Nobody forces conformance to any Internet
specification; it's purely voluntary.  And separate packages
such as metamail provide MIME reading capability to old-fashioned
non-MIME clients.

> Why require this in  
> the "standard", why not leave it up to the clients.

Interoperability.

> We'll end up  
> with a much of half hearted attempts at supporting the very 
> complex mime system.

As opposed to the current situation, which is...worse.

> This will greatly complicate the programing  
> needed to generated a standards compliant client.  Simple is better.

Life is complex.

Date: Fri, 24 Jun 2005 23:42:47 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Steve Walker <[email-masked]>, [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Steve,

thanks for reading the proposals before commenting.....

I'm sure you'll appreciate that this group has been around and trying to 
write standards for about 7 years, and has been through a massive amount of 
discussion about just about every topic related to USENET. We're hoping to 
finish the format description document real soon now - as I'm sure you can 
see from the traffic, there are some outstanding issues.

Asking the group at this time to reevaluate early basic decisions such as 
the decision to not depart from the RFC 2822 message format, and to 
recommend MIME tagging of messages, is certainly not exactly optimal; while 
you have only recently discovered the group, most of the participants have 
already spent considerable cycles figuring out what should and should not 
be done.

You've raised an issue - that binary postings common in alt. groups will 
apparently not be conformant with the new format description. If you can 
figure out exactly which part of the USEFOR draft that creates this 
conflict, and can propose language to deal with that, I'm sure we can 
follow up on that.

But please don't come in and say "You're all wrong, you need to start all 
over". We're not at that stage of the group's lifecycle.

                        Harald Alvestrand
                         USEFOR WG chair

--On 23. juni 2005 20:26 -0500 Steve Walker <[email-masked]> wrote:

>
> I just stumbled across your website/mailing list and thought I would
> share my first impression.  First of all, I've developed Usenet software
> and ran commercial news servers for many years, so I understand the
> protocols well.  While I haven't read the proposal 100% yet, what I have
> read very disappointing.  From what I can tell, your work group is going
> to do more damage to Usenet than good.

To: [email-masked]
Date: Fri, 24 Jun 2005 14:36:59 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

> > yEncoding
> > and MIME don't mix well.

> No, see the two proposed drafts mentioned separately.

FWIW, I would be happy to revive the draft if there is any interest.
The last time I tried this the silence was fairly deafening, however.

I should also mention that Dan and I saw no problem, and indeed some benefit,
from combining the two drafts.

				Ned

Date: Fri, 24 Jun 2005 15:37:08 -0500
From: Steve Walker <[email-masked]>
To: [email-masked]
CC: Henry Spencer <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

Bruce Lilly wrote:
> "yEnc" is not a standard.  If you believe otherwise, then (after carefully
> reading RFC 1796) please cite the full Standard RFC in which it is specified.

With 65% of articles using yEnc and less than 4% using mime, I'd 
say it more of a real standard than mime.  Just because the ietf 
doesn't call it a standard doesn't make it an industry standard 
due to widespread use.  If you wrote a "standard" that goes 
against common practice who do you think will win?  It's better 
to write a standard using common practice as the baseline.

What is the point to write a new standard that no one will use it?

Instead of trying to break existing practices, why doesn't anyone 
address any of the real problems facing the Usenet news system?

For example, why doesn't the "standard" specify that 64-bit 
integers are used for article counters?  A real-world problem 
facing NNTP developers is what happens when the 31/32 bit counter 
rolls over?  I highly doubt that most clients can deal with this 
issue.  Some clients are going to roll at 2^31 and some at 2^32.

Changes like this and others would actually provide some benefit 
to making a new standard.

Steve.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Steve Walker <[email-masked]>
Subject: Re: Feedback on proposal
Date: Fri, 24 Jun 2005 16:21:25 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 24 2005 13:55, Steve Walker wrote:

> MIME doesn't deal well with existing Usenet practices.

You might want to have a look at the USEFOR WG charter:
http://www.ietf.org/html.charters/usefor-charter.html

Note that we're working on "best current practice" and "best
proposed practice".  The word "best" is not an accident.  Just
because something has been done (leading in many cases to
interoperability problems) does not mean that it's a good idea.

> yEncoding  
> and MIME don't mix well.

No, see the two proposed drafts mentioned separately.

> Considering a very large percentage of  
> Usenet articles are yEncoded you are creating a situation that 
> has no resolution.

We didn't create the non-standard "yEnc" situation.  Nobody
prevented a proposal for such an encoding scheme within the
existing messaging framework, and at least two people have
separately tried to fix that oversight.

> They will not quit posting just because  
>  >someone< says they >have< to use use mime/base64.

Nobody is saying that.
 
> Usepro wrote:
> NOTE: this corresponds to the range of octets permitted for MIME
>          "8bit data" [RFC 2045].  Thus raw binary data cannot be
>          transmitted in an article body except by the use of a
>          Content-Transfer-Encoding such as base64.

You understand the meaning of "such as", right?

> I started monitor the Usenet stream for 'yEnc' in the Subject 
> header which is required for yEncoding.

"Required" by which Standards Track RFC?  (By the way, you might
want to read RFC 4096.  And definitely read RFC 2822 section 3.6.5
paying special attention to the word "unstructured" (defined in
2822 sect. 2.2.1))

Date: Fri, 24 Jun 2005 15:23:26 -0500
From: Steve Walker <[email-masked]>
To: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Bruce Lilly wrote:
> On Fri June 24 2005 14:09, Steve Walker wrote:
> 
> Nothing in the specification requires all messages to be MIME
> (as noted earlier by Henry).  You may be misinterpreting the
> specification.  I';; send a detailed reply to your longer
> message in a while.

Thanks for pointing this out.  I will go back and read more 
carefully.  In the real world, if all clients must support mime 
how do non-mime articles get posted.  I really don't see software 
developers who design clients that allow non-mime posting to 
bother with fully supporting reading mime.  Why require this in 
the "standard", why not leave it up to the clients.  We'll end up 
with a much of half hearted attempts at supporting the very 
complex mime system. This will greatly complicate the programing 
needed to generated a standards compliant client.  Simple is better.

Regardless, it does prohibit binary data in the message body 
unless it is MIME encoded which will break a large number of 
articles.

Steve.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Steve Walker <[email-masked]>
Subject: Re: Feedback on proposal
Date: Fri, 24 Jun 2005 16:07:46 -0400
Cc: Henry Spencer <[email-masked]>, Usefor Mailing List <[email-masked]>
Sender: [email-masked]

On Fri June 24 2005 15:17, Steve Walker wrote:
> Henry Spencer wrote:

> > I've generally ignored the yEnc business and will have to leave it to
> > others to comment on it.  I will note that there's nothing in our spec
> > that says that news software can't have features beyond what we describe. 
> 
> It says that binary data must be mime/base64 encoded.

It would help if you would cite the specific draft and section so that
others can clearly see what specific text you are referring to.

You also might wish to carefully read surrounding text to make sure it
doesn't mention things like "future extensions", etc.

You are of course welcome to provide *constructive* criticism; you might
want to read the following RFCs and references first, though:
   [BCP9]     Bradner, S., "The Internet Standards Process --
              Revision 3", BCP 9, RFC 2026, October 1996.

   [BCP14]    Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [FYI17]    Harris, S., "The Tao of IETF - A Novice's Guide to
              the Internet Engineering Task Force", RFC 3160,
              August 2001.

   [BCP25]    Bradner, S., "IETF Working Group Guidelines and
              Procedures", BCP 25, RFC 2418, September 1998.

   [BCP83]    Rose, M., "A Practice for Revoking Posting Rights to IETF
              Mailing Lists", BCP 83, RFC 3683, March 2004.

   [BCP94]    Wasserman, M., "Updates to RFC 2418 Regarding the
              Management of IETF Mailing Lists", BCP 94, RFC 3934,
              October 2004.

   [BCP95]    Alvestrand, H., "A Mission Statement for the IETF",
              BCP 95, RFC 3935, October 2004.

   [RFC1796]  Huitema, C., Postel, J., and S. Crocker, "Not All
              RFCs are Standards", RFC 1796, April 1995.

   [RFC1958]  Carpenter, B., "Architectural Principles of the
              Internet", RFC 1958, June 1996.

   [RFC3426]  Floyd, S., "General Architectural and Policy
              Considerations", RFC 3426, November 2002.

   [RFC3439]  Bush, R. and D. Meyer, "Some Internet Architectural
              Guidelines and Philosophy", RFC 3439, December 2002.

   [RFC3444]  Pras, A. and J. Schoenwaelder, "On the Difference
              between Information Models and Data Models",
              RFC 3444, January 2003.

   [Errata]   RFC-Editor errata page,
              http://www.rfc-editor.org/errata.html

and of course the primary normative references for our work, RFCs
2045-2049, 2231, 2234, and 2822.

> That  
> breaks the yEnc standard and 65% of all Usenet posts.

"yEnc" is not a standard.  If you believe otherwise, then (after carefully
reading RFC 1796) please cite the full Standard RFC in which it is specified.

Note that nothing precludes a proper specification of an encoding scheme
similar to "yEnc", and in fact there have been two Internet-Drafts (one
written by Ned Freed, the other by Dan Kohn) which have attempted to do so.
See
http://www.imc.org/ietf-usefor/2003/Feb/0506.html
http://www.imc.org/ietf-usefor/2003/Feb/0575.html
http://www.imc.org/ietf-usefor/2003/Feb/0577.html 

Date: Fri, 24 Jun 2005 12:59:35 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

Henry Spencer <henry@xxxxxxxxxxxxx>:

>I really think reducing our document's readability to its primary
>audience,

Using standard terminology does not reduce the readability of our
document, and certainly not for our primary audience, who are people who
are going to have to read and understand RFC2822 along with our standard.  
It isn't bedtime reading, it's an IETF standard, and it is an IETF
standard intended to be read with RFC2822. Having to translate back and
forth is what reduces USABILITY, which it the goal.

>I can live with it *if* we explicitly apologize for it.

There are other things to apologize for in "our document" than for using
standard terminology. That's one thing we do NOT have to apologize for.

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>BTW, will the change, as proposed, still allow us to speak, informally, of
>the "Date header", etc., or are we suppose to say the "Date (header)
>field" everywhere?

You may speak informally in any way you wish, but it is trivial to convert
the current "Date Header" to "Date Header Field" and be precise in the
technical standard.

>Us poor editors will just have to spend an hour or so with out text
>editors :-( .

Ten minutes in vi did the trick. If it takes you an hour, you need a 
better text editor. What, do you have to find the right spot in the 
punched tape and glue chads back into place to make an edit? Can we get 
past this "it's too much work to be correct" argument yet?

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Steve Walker <[email-masked]>
Subject: Re: Feedback on proposal
Date: Fri, 24 Jun 2005 15:36:12 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 24 2005 14:09, Steve Walker wrote:

> About 96% of Usenet articles do not have the 'mime-version' 
> header.  To require something that less than 4 percent is 
> currently doing is not wise.

Nothing in the specification requires all messages to be MIME
(as noted earlier by Henry).  You may be misinterpreting the
specification.  I';; send a detailed reply to your longer
message in a while.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Steve Walker <[email-masked]>
Subject: Re: Feedback on proposal
Date: Fri, 24 Jun 2005 15:34:10 -0400
Cc: Russ Allbery <[email-masked]>, [email-masked]
Sender: [email-masked]

On Fri June 24 2005 14:01, Steve Walker wrote:

> I'm counting any article with a MIME-Version: header.  Isn't this 
> header required for mime?

Are you checking in a case-insensitive manner? are you including
Microsoft-generated messages that erroneously put the field in a
MIME-part header instead of the message header?

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Henry Spencer <[email-masked]>
Subject: Re: #1043 "header": Suggested resolution
Date: Fri, 24 Jun 2005 15:29:29 -0400
Cc: Usefor Mailing List <[email-masked]>, Harald Tveit Alvestrand <[email-masked]>, Alexey Melnikov <[email-masked]>
Sender: [email-masked]

On Fri June 24 2005 12:44, Henry Spencer wrote:

> I really think reducing our document's readability to its primary
> audience,

As you well know, that is not the intent, and it is frankly offensive
that you are implying that that is the intent.

> solely for the sake of consistency with RFC 2822,

Ditto.

> If we do it, then given that 2822speak is rare in the news world, was not
> used in prior news specs,

RFC 1036:

    A standard USENET message consists of several header lines, followed
...
    continuation header lines (beginning with a blank or tab) is
...

are entirely consistent with RFC 822 (and 2822).  Granted there are also
inconsistencies.

> is not used in the NNTP draft,

Not our bailiwick.

> and has not been  
> seen previously in our work,

False assertion, presumably knowingly made (this has been discussed on
list):

draft-ietf-usefor-usefor-00:

   User agents SHOULD support on receipt and MAY generate MIME extension
   header fields, including but not limited to Content-Disposition
...
   This is the only header field definition that updates [RFC2822].  The
...

> because we thought consistency was more important than readability".

Again, this is absolutely not the case, and the assertion is offensive.

Date: Fri, 24 Jun 2005 14:17:46 -0500
From: Steve Walker <[email-masked]>
To: Henry Spencer <[email-masked]>
CC: Usefor Mailing List <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

Henry Spencer wrote:
>>MIME doesn't deal well with existing Usenet practices.  yEncoding 
>>and MIME don't mix well.  Considering a very large percentage of 
>>Usenet articles are yEncoded...
> 
> 
> How odd that I read hundreds of Usenet articles a day -- and have been
> doing so for 24 years -- and I've never (that I recall) encountered one.

So you want to write a "standard" based only on what you 
personally use it for?  Usenet users are a very large and diverse 
group. One person or a small group of people can't possibly know 
how everyone else uses it.

> You might want to produce some numbers to support that "very large
> percentage" part.  Bear in mind that "Usenet" and "news" are two different
> things; not all news is Usenet. 

Care to provide the RFC that explains that groups that you don't 
read are no longer part of the Usenet news system?

And I did provide numbers.  I'm running the entire Usenet news 
feed through a filter counting yEncoded articles.  It's showing 
about 65 percent of all articles are yEncoded.

> I think you should be saying "news" and not "Usenet"; I would conjecture
> that those articles are heavily concentrated in hierarchies that are not
> part of Usenet proper.  Indeed, it would be interesting to know just where
> all that yEnc traffic is. 

Mainly, alt.* of course.  Are you claiming that this standard 
does not apply to articles posted to alt.* groups?  If so, it 
should state that.

I've been running servers for a long time, and I know that alt.* 
groups are without a doubt the most popular groups.  They get 
more posts and they generate more reads than all other groups 
combined. To ignore them when designing a "standard" is unthinkable.

> I've generally ignored the yEnc business and will have to leave it to
> others to comment on it.  I will note that there's nothing in our spec
> that says that news software can't have features beyond what we describe. 

It says that binary data must be mime/base64 encoded.  That 
breaks the yEnc standard and 65% of all Usenet posts.

Steve.

Date: Fri, 24 Jun 2005 14:37:53 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
cc: Steve Walker <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

On Fri, 24 Jun 2005, Steve Walker wrote:
> > What on Earth are you talking about?  There is nothing in our drafts that
> > makes a non-MIME article invalid...
> 
> Section 2.3 MIME Conformance is mandating that agent MUST support 
> mime.

Correct.  The user-interface *software* must know about MIME, because MIME
articles might appear (which is already true) and the user may want to
read them or prepare followups to them (again already true).  That doesn't
mean the software has to use MIME headers in every article it posts.  They
are mandatory only when things like non-ASCII characters show up in the
articles.

> The spec says on one hand the minimum amount of headers is xxx, 
> but also says that mime is required...

It says that MIME *support* is required in the *software*.  That's quite
different from saying that MIME headers are required in the articles.

> MIME doesn't deal well with existing Usenet practices.  yEncoding 
> and MIME don't mix well.  Considering a very large percentage of 
> Usenet articles are yEncoded...

How odd that I read hundreds of Usenet articles a day -- and have been
doing so for 24 years -- and I've never (that I recall) encountered one. 
You might want to produce some numbers to support that "very large
percentage" part.  Bear in mind that "Usenet" and "news" are two different
things; not all news is Usenet. 

> I started monitor the Usenet stream for 'yEnc' in the Subject 
> header which is required for yEncoding.  I'm seeing (at least) 
> 65% of all Usenet articles are yEnc...

I think you should be saying "news" and not "Usenet"; I would conjecture
that those articles are heavily concentrated in hierarchies that are not
part of Usenet proper.  Indeed, it would be interesting to know just where
all that yEnc traffic is. 

I've generally ignored the yEnc business and will have to leave it to
others to comment on it.  I will note that there's nothing in our spec
that says that news software can't have features beyond what we describe. 

> So the argument is still valid: this proposal is trying to force 
> clients to use mime...

No, this proposal requires that clients *understand* MIME, so they can
handle reading of, and followup to, MIME articles -- which by your own
figures, are already present in modest but significant numbers.  There
is no requirement that they use it in articles they originate.

> ...and it would directly break the majority of all Usenet posts.

No.  See above.

                                                          Henry Spencer
                                                       [email-masked]

Date: Fri, 24 Jun 2005 13:38:02 -0500
From: Steve Walker <[email-masked]>
To: Sebastian Brocks <[email-masked]>
CC: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Sebastian Brocks wrote:
> Steve Walker schrieb:
> 
> 
>>You will not get the other 96% to 
>>switch to mime just because this "standard" is published.
> 
> 
> The vast majority of that 96% is binary traffic, which does not
> care about standards anyway.

So you write something knowing no one will follow and call it the 
standard.  Why don't we write a "standard" that follows standard 
practices.

The software developers can, and do, follow standards.  Otherwise 
they wouldn't be able to read/post at all.  The problem is when 
we get 100 people each doing the same thing 100 different ways. 
This is exactly what a "standard" should solve.  Provide a set of 
guidelines that everyone can use.  To make a bunch of rules, 
knowing that the majority of people can't follow, even if they 
wanted to, is dumb.

If you have plans on ignoring the needs of the single largest 
group of Usenet users, the binary posters and readers, you should 
specify that your rules only apply to the small-x text-only groups.

This should be a "standard" covering "Usenet", all of Usenet, not 
just the tiny portion [insert person here] reads.

Steve.

Date: Fri, 24 Jun 2005 20:11:46 +0200
From: Sebastian Brocks <[email-masked]>
To: Steve Walker <[email-masked]>
CC: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Steve Walker schrieb:

> You will not get the other 96% to 
> switch to mime just because this "standard" is published.

The vast majority of that 96% is binary traffic, which does not
care about standards anyway.

Date: Fri, 24 Jun 2005 14:06:49 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

Henry Spencer wrote:

> On Fri, 24 Jun 2005, Harald Tveit Alvestrand wrote:
> 
>>Since several people weighed in on the subject after my summary, we seem to 
>>be moving towards a rough consensus, and I want to try for a resolution.... 
>>Henry and Charles have continued to argue for the "news" syntax; I'd like 
>>to check that they find it possible to work with the "2822" syntax...
> 
> 
> I really think reducing our document's readability to its primary
> audience, solely for the sake of consistency with RFC 2822, is a mistake. 
> Especially since the usual news terminology is commonly heard even in the
> mail community now. 

2822 is a NORMATIVE reference.  We refer to and require people to
use it.  The ENTIRE audience will need to refer and understand
RFC2822.

If our documents stood by themselves, your argument would have
merit.  But there is no way to understand it without using RFC2822.

I know you advocate that the document should stand alone, but
as written, it requires going back and forth.  Doing that is a
hassle.  Doing it with similar words but different terminology
is a nightmare.

I made this point before.  I do not recall any reply addressing
the point.  Maybe we shouldn't use RFC2822 as a base.

Date: Fri, 24 Jun 2005 13:09:54 -0500
From: Steve Walker <[email-masked]>
To: [email-masked]
CC: Russ Allbery <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

Bruce Lilly wrote:
> I agree with Russ.  Note that any message containing a MIME-Version
> header field (N.B. "mImE-VeRsIoN" is case-insensitive) is by definition
> [RFC 2045] a MIME message; RFC 2045:

Good.  Then my filter is checking in the correct way.  It 
searches for the 'mime-version' header (case-insensitive) and 
counts them vs the number of articles that due not have that header.

About 96% of Usenet articles do not have the 'mime-version' 
header.  To require something that less than 4 percent is 
currently doing is not wise.  You will not get the other 96% to 
switch to mime just because this "standard" is published.  The 
standard is non-mime by GAUP and certainly non-base64.

Steve.

Date: Fri, 24 Jun 2005 13:01:38 -0500
From: Steve Walker <[email-masked]>
To: Russ Allbery <[email-masked]>
CC: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Russ Allbery wrote:
>>I installed a filter to count mime messages.  Of the 75,000 articles
>>tested so far, mime messages account for about 2% of Usenet articles.
> 
> 
> I find this incredibly unlikely.  How are you counting MIME messages?

I'm counting any article with a MIME-Version: header.  Isn't this 
header required for mime?  If you have a better suggestion I'd be 
happy to adjust the filter and provide new numbers.

Steve.

Date: Fri, 24 Jun 2005 12:55:18 -0500
From: Steve Walker <[email-masked]>
To: Henry Spencer <[email-masked]>
CC: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Henry Spencer wrote:

> What on Earth are you talking about?  There is nothing in our drafts that
> makes a non-MIME article invalid.  Use of MIME -- MIME, not base64 -- is
> mandatory only when non-ASCII characters are involved, which is precisely
> when it *should* be mandatory, if only to specify what character set is
> being used.

Usefor:
Section 2.3 MIME Conformance is mandating that agent MUST support 
mime.

So any client that doesn't post mime can no longer be called an 
NNTP client?  You just killed hundreds of news clients.

The spec says on one hand the minimum amount of headers is xxx, 
but also says that mime is required, but not all required mime 
headers are required Usenet headers.  This is a conflict.

MIME doesn't deal well with existing Usenet practices.  yEncoding 
and MIME don't mix well.  Considering a very large percentage of 
Usenet articles are yEncoded you are creating a situation that 
has no resolution.  They will not quit posting just because 
 >someone< says they >have< to use use mime/base64.

Usepro wrote:
NOTE: this corresponds to the range of octets permitted for MIME
         "8bit data" [RFC 2045].  Thus raw binary data cannot be
         transmitted in an article body except by the use of a
         Content-Transfer-Encoding such as base64.

This completely breaks a very large percentage of articles.

I started monitor the Usenet stream for 'yEnc' in the Subject 
header which is required for yEncoding.  I'm seeing (at least) 
65% of all Usenet articles are yEnc.  Some yEnc'ed articles do 
not use the required yEnc in the subject line.  These are not 
being counted in my numbers.

The monitoring for mime is showing slightly higher mime counts 
during the day, it's climbed up to the 3-4 percent mark.

So the argument is still valid: this proposal is trying to force 
clients to use mime regardless of the fact that very few posts 
are currently mime and it would directly break the majority of 
all Usenet posts.

Steve.

To: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
From: Graham Drabble <[email-masked]>
Date: Fri, 24 Jun 2005 18:35:27 +0100
Organization: Home
Sender: [email-masked]

On 24 Jun 2005 Henry Spencer <[email-masked]> wrote in
news:[email-masked]: 

> 
> On Fri, 24 Jun 2005, Harald Tveit Alvestrand wrote:
>> Since several people weighed in on the subject after my summary,
>> we seem to be moving towards a rough consensus, and I want to try
>> for a resolution.... Henry and Charles have continued to argue
>> for the "news" syntax; I'd like to check that they find it
>> possible to work with the "2822" syntax... 
> 
> I really think reducing our document's readability to its primary
> audience, solely for the sake of consistency with RFC 2822, is a
> mistake. Especially since the usual news terminology is commonly
> heard even in the mail community now. 

I would agree here. We could change but I would rate consistency with 
news (past standards and current practice) as more important than 
consistence with RFC2822 and MIME.
-- 
Graham Drabble
http://users.ox.ac.uk/~sjoh1646/

Date: Fri, 24 Jun 2005 18:21:10 +0100
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes

>Usenet has been working quite happily for
>many years now just by admins using their common sense when choosing
><path-identity>s. I am not aware of serious clashes that have occurred,
>and if they have then they have been sorted out amicably (because it is in
>everybody's interest to do so). It ain't broke, so it doesn't need fixing.

It would also be unwise to put any existing servers in a position that
they found themselves feeling that to conform to modern standards they
should change their identity...

Demon Internet (the first UK dialup ISP in the early 1990s) has used
!demon! since before it was called Demon Internet or even before it sent
news over NNTP. When an error was made with a new peering machine and
this tag was omitted a number of customers (with simple single-user
"leaf" systems who only swapped articles with Demon) started fetching
articles and (trying to) re-inject them all. Because servers elsewhere
in the cluster, generally, already had a copy of the article this went
on unnoticed for weeks until occasionally the customer presented the
article earlier than the peering machine.... and when the article was
spam it tripped detectors and we found the problem.

What I'm trying to illustrate is that path identities are not only
important to the few thousand machines that peer news on a high volume
basis, but these identities are nailed into hundred of thousands of end
user systems. We outlaw (or deprecate) existing path identities, whether
dating from UUCP or more recent conventions, at our peril.

and to echo Charles, what was it that was broken ?

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

Date: Fri, 24 Jun 2005 18:11:10 +0100
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1043 USEFOR 2.1 Use of the term "header"
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes
>
>In <[email-masked]> Harald Tveit 
>Alvestrand <[email-masked]> writes:
>
>>[I've created a new ticket number for the "header" issue, because I'm not 
>>sure that's all there is to Bruce's complaint in #1031. I can't tell from 
>>what he wrote.]
>
>>The argument in favour is that this conforms with RFC 2822 and documents 
>>that derive their terminology from that (and coincidentally also allows us 
>>to define "header content" differently than "field body", if we need that - 
>>Charles wants such a term).
>
>There is also the argument that the NNTP draft, which has just passed its
>Last Call, uses "header" as we currently do. That might well be fixed

the discussions on another list suggested that it was likely to be fixed

>BTW, will the change, as proposed, still allow us to speak, informally, of
>the "Date header", etc., or are we suppose to say the "Date (header)
>field" everywhere?

I don't think a standard ever says anything informally!

'Date header field' may feel like a nuisance to type the first time
around, but it tells the implementor that you're making a precise
statement and, where it matters, they can check the references and
remind themselves whether, for example, this does or doesn't include the
field name, the end of line characters or whatever....

I think Henry Spencer is right and you shouldn't just dive in with 2822
language, but indicate clearly (apologise isn't quite the word I'd use
for something that's quite understandable when you read, albeit a very
slight pain to type in correctly) at the start that along with a lot of
formatting detail some jargon has been inherited from 2822 #2.2 and that
having a copy of that to hand is an inescapable necessity

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

Date: Fri, 24 Jun 2005 10:17:17 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>I think my point was that given that the colons in IPv6 literals are known 
>to cause problems, and so far, no prohibition or warning is in the text, 
>the standards writers (that's us) haven't exhibited good judgment.....

While that may have been your intended point, that's not what you said. 
You said:

>anyone who generates something permitted
>by the ABNF even when it's explicitly forbidden by the accompanying text
>is certainly not exhibiting good judgment.

That 'even when' tells me you think it is explicitely forbidden to use ':' 
in path identities, not that we need to make it explicitely forbidden.
We are talking about someone doing something based on USEFOR-04, not
some unknown future version.

>Attempting to return to the problem at hand, rather than the discussion on 
>how to discuss these things - John, can you educate me a little bit about 
>USENET feeds?

I wrote:

>... the same failure that would make the dead:beef admin an
>idiot would make both of these people idiots. 

The point was that if the admin of "dead:beef" is an idiot for using a 
path-identity containing colons (which he is told is legal by USEFOR-04),
then people who use IPV6 address-literal path-identities which can parse 
into other previously valid path-identities using RFC1036 parsing are
just as much idiots, because the problem is the same. Whatever that 
problem is. And I think it was Russ who explained it.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Henry Spencer <[email-masked]>
Subject: Re: Feedback on proposal
Date: Fri, 24 Jun 2005 13:16:06 -0400
Cc: [email-masked], Steve Walker <[email-masked]>
Sender: [email-masked]

On Fri June 24 2005 12:18, Henry Spencer wrote:

> Although (as Bruce noted) this is pretty unlikely:  RFC 1049 was fairly
> obscure and not widely implemented.  It's technically possible for a
> non-MIME message to have Content-Type, but in practice, it's a pretty safe
> bet that the presence of Content-Type means it's really a MIME message but
> produced by broken software.

Yes, and in the unlikely event that such a message is found, the RFC 1049
syntax is sufficiently different from MIME syntax to be able to determine
which is the case. 

Date: Fri, 24 Jun 2005 18:55:39 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Admin: A Chair going on holiday
Sender: [email-masked]

Just FYI....

I'll be on holiday July 2-16.

Date: Fri, 24 Jun 2005 18:54:30 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Admin: Posters this week
Sender: [email-masked]

  1   39  22.54 "Charles Lindsey" <[email-masked]>
  2   26  37.57 Frank Ellermann <[email-masked]>
  3   22  50.29 Harald Tveit Alvestrand <[email-masked]>
  4   22  63.01 Bruce Lilly <[email-masked]>
  5   18  73.41 Henry Spencer <[email-masked]>
  6   17  83.24 Russ Allbery <[email-masked]>
  7   12  90.17 John Stanley <[email-masked]>
  8    3  91.91 "Forrest J. Cavalier III" <[email-masked]>
  9    3  93.64 Seth Breidbart <[email-masked]>
 10    2  94.80 "Ruud H.G. van Tol" <[email-masked]>
 11    2  95.95 Steve Walker <[email-masked]>
 12    2  97.11 [email-masked] (Kai Henningsen)
 13    2  98.27 Eivind Tagseth <[email-masked]>
 14    1  98.84 Ken Murchison <[email-masked]>
 15    1  99.42 Ned Freed <[email-masked]>
 16    1 100.00 "Scott Hollenbeck" <[email-masked]>

Date: Fri, 24 Jun 2005 09:52:56 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1043 USEFOR 2.1 Use of the term "header"
Sender: [email-masked]

 Henry Spencer <henry@xxxxxxxxxxxxx>:

>> Ummm, ok. So what? Who has said it was?

>You did --

Nonsense. I said nothing of the kind. I quoted a statement from Harald 
that we needed to come to a decision and pointed out that there was no 
justification for a decision to be different. 

>When you quote part of a previous message before your
>remarks, the usual assumption is that it's relevant to what you're saying. 

I quoted the entire paragraph, which ended with him saying "we need to 
stop talking and decide." 

Date: Fri, 24 Jun 2005 12:44:28 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

On Fri, 24 Jun 2005, Harald Tveit Alvestrand wrote:
> Since several people weighed in on the subject after my summary, we seem to 
> be moving towards a rough consensus, and I want to try for a resolution.... 
> Henry and Charles have continued to argue for the "news" syntax; I'd like 
> to check that they find it possible to work with the "2822" syntax...

I really think reducing our document's readability to its primary
audience, solely for the sake of consistency with RFC 2822, is a mistake. 
Especially since the usual news terminology is commonly heard even in the
mail community now. 

If we do it, then given that 2822speak is rare in the news world, was not
used in prior news specs, is not used in the NNTP draft, and has not been
seen previously in our work, we need to include an explanatory note saying
"sorry, we're using 2822 terminology, not the usual news terminology,
because we thought consistency was more important than readability".  I'm
serious -- this is a big enough disservice to our community that it calls
for at least an apology. 

I can live with it *if* we explicitly apologize for it.

                                                          Henry Spencer
                                                       [email-masked]

Date: Fri, 24 Jun 2005 12:23:55 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

On Fri, 24 Jun 2005, Ruud H.G. van Tol wrote:
> >> Most Unix
> >> systems will treat 123.045.67.89 as equivalent to 123.37.67.89.  :)
> > No
> 
> Yes. Try 'ping' for just one example of such treatment.

If you read what I actually wrote -- which was more than "No" -- you'll
see that you've misunderstood.  I didn't say "Unix doesn't do that"; I
said "Unix does that but shouldn't".

                                                          Henry Spencer
                                                       [email-masked]

Date: Fri, 24 Jun 2005 12:18:42 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: [email-masked]
cc: Steve Walker <[email-masked]>
Subject: Re: Feedback on proposal
Sender: [email-masked]

On Fri, 24 Jun 2005, Bruce Lilly wrote:
>   ...the possible exception of Content-Type (in the absence of a
>   MIME-Version message header field, a message with a [RFC 1049]
>   Content-Type field and no other Content- fields is a non-MIME
>   message).

Although (as Bruce noted) this is pretty unlikely:  RFC 1049 was fairly
obscure and not widely implemented.  It's technically possible for a
non-MIME message to have Content-Type, but in practice, it's a pretty safe
bet that the presence of Content-Type means it's really a MIME message but
produced by broken software.

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
Xref: clerew local.usefor:21493
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Fri, 24 Jun 2005 13:55:00 GMT
Lines: 59
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Wed June 22 2005 15:25, Charles Lindsey wrote:
>> 
>> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>There is no such thing as "too long" for a field.  What was observed was
>the result of inappropriate folding at 990 octets to stay within the
>per-line limit (which does exist).  Broken software is broken software,
>and we should not impose limits which will be ineffective (a single-line
>References field of 995 octets would still be broken inappropriately at
>990 octets by the software in question), which are counter to agreed
>upon consensus (at least without reopening discussion first), and which
>run counter to established practice per RFCs 822, 850, 1036, and 2822.

It is clear that "established practice" is not working (as we have both
seen). Even if you carefully fold your References header, the next person
to followup to it is quite likely to unfold it again. Just what went wrong
in the particular case is not clear; what is clear is that it would not
have happened if some pruning had taken place (though whether my suggested
limit of 998 would have stopped it is not so clear).

>> Nobody has queried what I wrote so far, but if you 
>> wish it to be discussed, then please ask Alexey to raise a USEPRO ticket
>> for it, and we will do so.

>The procedure should be discuss first, and change the document (which is
>supposed to reflect WG consensus) only after consensus is declared by
>the Chair.  Since you are the one who apparently wishes to make a change,
>you should request a discussion ticket.

The text in question was written before the ticket system came into being.

Frank asked for something stronger, so I wrote a text and published it here.

It was discussed. People suggested wording changes, and they were made.
Nobody opposed the general idea of a MUST limit on the overall length
until you did so.

So if you want to pursue it, then you should ask for it to be ticketed.

It is clear that threading agents can manage perfectly well with only half
a dozen references (the minimum proposal of 3 is tight, but still
adequate).

So the issues are:
1. Do we propose a definite limit, and if so what number?
2. Is that limit a MUST prune or a SHOULD prune?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21498
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Fri, 24 Jun 2005 15:04:51 GMT
Lines: 65
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>You have to treat Path identities as opaque labels; anything else gets
>into way too much work to specify and implement, IMO.

Absolutely so.

But for human readers, it is useful to be able to regard them as
domain-names and/or IP addresses (as the case may be) and to look them up
in the DNS or in whois databases in order to determine the (alleged) route
an article took.

Also, in order to detect MISMATCHes, you will likely need to know whether
the IP address the NNTP packet came from agrees with what the
<path-identity> in the header says. But that does not mean you start doing
DNS lookups and the like for every article. You construct a database of the
IP addresses your peers normally use and the <path-identity>s they
normally use with them. For sure, you may have submitted domain-names and
IP addresses as seen in Path headers to various bits of DNS-related
software for the purpose of constructing your database, but that is an
offline operation so far as your main server software is concerned.

And if your peer insists on using different forms of the same IP address
in different articles, then you should seriously consider depeering him
:-) .

>> No. Frank is on recent record as being opposed to IP addresses in path.
>> You seem to have agreed "Absolutely" to treatment of path components
>> "purely as public names, not as IP address literals".  Russ is also on
>> record as being opposed to IP addresses.  Now where is this supposed
>> "benefit"?

>I'm not sure that I've really gone on the record being *opposed* to them;
>I just think they're a bad idea (which isn't quite the same thing as
>saying that they shouldn't be allowed).  IP addresses are used somewhat
>currently.

The text which en/dis-courages various forms of <path-identity> is
currently in USEPRO rather than USEFOR (because that's where Alexey asked
me to put it); I posted it here a while back.

Currently, it puts IP addresses (whether IPv4 or IPv6) as the least
desirable alternative, which seems to agree with opinions expressed here.
But they _do_ happen, and I see no reason to ban them altogether.

>....  I'm not seeing a clamor
>from IPv6 sites wanting to put address literals into the Path header.

I think that is because usage of IPv6 is still much less than of IPv4. But
clearly it is going to take off one of these days, probably in the Pacific
region first, because that is where the supply of IPv4 addresses seems to
be running out. And then you will start to see them in <path-identity>s;
but probably not for a couple of years or so.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21495
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1043 USEFOR 2.1 Use of the term "header"
Date: Fri, 24 Jun 2005 14:12:02 GMT
Lines: 44
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>[I've created a new ticket number for the "header" issue, because I'm not 
>sure that's all there is to Bruce's complaint in #1031. I can't tell from 
>what he wrote.]

>The argument in favour is that this conforms with RFC 2822 and documents 
>that derive their terminology from that (and coincidentally also allows us 
>to define "header content" differently than "field body", if we need that - 
>Charles wants such a term).

There is also the argument that the NNTP draft, which has just passed its
Last Call, uses "header" as we currently do. That might well be fixed
after some negotiation with the RFC Editor, but i think we should decide
here what _we_ want, and then leave it to the NNTP group to come into line
if they wish (there is at least a month of leeway before that needs to be
decided).

I think the "content" issue is orthogonal to the issue behind this ticket.

If we leave the terminology as it currently stands in USEFOR, then USEFOR
will define and use the term "header body" (if it needs it - Harald has
already pointed out inconsistencies in the one place it is used). USEPRO
will define and use the term "(semantic) header content".

If we change the terminology to follow RFC 2822, then those two terms will
simply become "header field body" and "(semantic) header field content".
Us poor editors will just have to spend an hour or so with out text
editors :-( .

BTW, will the change, as proposed, still allow us to speak, informally, of
the "Date header", etc., or are we suppose to say the "Date (header)
field" everywhere?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21503
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Fri, 24 Jun 2005 15:55:48 GMT
Lines: 54
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Wed June 22 2005 15:18, Charles Lindsey wrote:
>> 
>> In RFC 2822 an <id-right> is a string of characters satisfying some ABNF.

>The grammar production named id-right is an ABNF rule name, yes.

>> It has no semantics attached to it (except that a `\` in a <quoted-pair>
>> is "invisible").

>No, the semantics are "the domain name (or a domain literal IP address)".

I do not see where RFC 2822 says that.

It is essentially the same as we have just been agreeing for the Path
header. All of <path-identity>, <id-left> and <id-right> are "opaque
labels", whether you initially construct them out of domain-names or
domain literals, or whatever else.

>> No, Mark made his suggestion (in the NNTP list, I believe)

>Mark's suggestion was made here, and is archived in this WG's mailing list
>archive at
>http://www.imc.org/ietf-usefor/2003/Feb/0493.html

>> in the belief 
>> that he was describing a temporary problem that would be fixed in future
>> implementations.

>That is a false assertion; Mark specifically referred to a transport
>protocol constraint, as his text clearly shows.

Mark may well have said that on this list when he was a member of it. But
he most certainly raised essentially the same point on the NNTP list
within the last few months (as you may check from their website), and he
signally failed to convince that WG.

>> He was rapidly disabused of that notion. 

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21501
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Fri, 24 Jun 2005 15:31:03 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Thu June 23 2005 14:43, Frank Ellermann wrote:

>> But that's also not necessary for your idea, we can get this
>> effect by saying that a <path-identity> is compared octet by
>> octet (BTW, same idea as for a msg-id), and so it should be
>> always created in the same way.

>That's neither necessary nor sufficient for the purposes the Path
>field has.  The purposes listed in -04 3.1.6 impose first and
>foremost a uniqueness requirement.

It doesn't explicitly say so, but it is evident that it is necessary.

>  That in turn requires either
>a hierarchical system (like DNS) or a central authority (like the
>UUCP mapping project was supposed to be, though it never worked
>well).

But that does not follow at all. Usenet has been working quite happily for
many years now just by admins using their common sense when choosing
<path-identity>s. I am not aware of serious clashes that have occurred,
and if they have then they have been sorted out amicably (because it is in
everybody's interest to do so). It ain't broke, so it doesn't need fixing.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21502
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Date: Fri, 24 Jun 2005 15:43:02 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>Software that accepts dates with unknown timezones SHOULD treat such
> timezones as equivalent to "-0000", as specified in RFC 2822 section 4.3.

Yes, it seems we have been discussing something tht RFC 2822 had already
resolved.

As regards the wording you just proposed above, I would still prefer it to
be like:

    Software MAY accept dates with unknown timezones and SHOULD treat ...

Of course, whether some particular timezone is "unknown" is still
dependent upon whether or not you chose to accept all of the <obs-zone>.

As for the syntax and the rest of the text, I am happy with what Harald
proposed at the start of this thread, namely removing UT and just leaving
GMT as the only MUST accept / MUST NOT generate item.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21497
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Fri, 24 Jun 2005 14:22:46 GMT
Lines: 29
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>--On onsdag, juni 22, 2005 17:23:48 -0700 John Stanley <[email-masked]> 
>wrote:

>While name-calling is not a good idea even when the person one is insulting 
>cannot be identified, anyone who generates something permitted by the ABNF 
>even when it's explicitly forbidden by the accompanying text is certainly 
>not exhibiting good judgment.

>On the other hand, if the standards writers intended to forbid it, and 
>standard does not specifically say so, the standards writers are not 
>showing good judgment.....

Indeed. I proposed a text in response to John Stanley that could be
incorporated into the Message-ID section in order to point out the
unwisdom, at the present time, of using a ":" other than for representing
an <IPv6address>.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21500
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Fri, 24 Jun 2005 15:13:57 GMT
Lines: 28
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>Henry Spencer wrote:
>> 
>> A bit of background you might not be aware of:  in some earlier
>> discussions (on other topics), it was surprisingly difficult to convince
>> some participants that such accompanying text is actually part of the
>> standard, not just helpful commentary or advice to implementers. 
>> 

>In a discussion on the References field, Charles wrote this on 6/13/05
>in <[email-masked]> indicating that he considers notes
>to have no normative effect.  I don't think that Charles would
>consider the main text not normative.

Indeed. For sure I don't consider NOTEs normative, but accompanying text
outside of NOTEs most certainly is.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21496
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1043 USEFOR 2.1 Use of the term "header"
Date: Fri, 24 Jun 2005 14:16:28 GMT
Lines: 26
Sender: [email-masked]

In <[email-masked]> "Scott Hollenbeck" <[email-masked]> writes:

>Area advisor opinion: after re-reading the working group charter I can
>understand why this has been contentious.  The charter says:

>...  I'm thus
>inclined to believe that usefor should use the same terminology used in 2822
>where appropriate, or it should explicitly explain why different terminology
>is being used and precisely where the deviations exist.

For the record, our present draft does explain precisely the nature of the
deviation, but it is short on explaining why. However, our earlier draft
_did_ explain why, and that text could easily be copied into, or adapted
for, the current draft.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21499
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Our web site
Date: Fri, 24 Jun 2005 15:11:15 GMT
Lines: 26
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Unfortunately the online WG mailing list archive is incomplete.

It's all there in mbox format, if you are desperate.

I eventually got around to updating my Perl script to grab them from the
Landfield site, and even managed to download a couple of months' worth.

But next day I found the site down, and I have not been able to catch it
up since. But I shall keep trying.

And I was going to put the USEFOR-04 draft up at the same time, which is
why that has never happened as of yet.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21494
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Fri, 24 Jun 2005 13:57:39 GMT
Lines: 21
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>While RFC2822 may have a 998 per line limit, USEFOR-04 has a 998 per 
>"header" limit. In section 2.2:

>   o  Compliant software MUST NOT generate (but MAY accept) headers of
>      more than 998 octets. 

Yes that has already been pointed out, and I think we all agree that it is
a Bug in USEFOR, and I hope Ken is in process of fixing it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Russ Allbery <[email-masked]>
Subject: Re: Feedback on proposal
Date: Fri, 24 Jun 2005 11:51:44 -0400
Cc: [email-masked], Steve Walker <[email-masked]>
Sender: [email-masked]

On Fri June 24 2005 09:09, Russ Allbery wrote:
> 
> Steve Walker <[email-masked]> writes:

> > I installed a filter to count mime messages.  Of the 75,000 articles
> > tested so far, mime messages account for about 2% of Usenet articles.
> 
> I find this incredibly unlikely.  How are you counting MIME messages?

I agree with Russ.  Note that any message containing a MIME-Version
header field (N.B. "mImE-VeRsIoN" is case-insensitive) is by definition
[RFC 2045] a MIME message; RFC 2045:

    (1)   A MIME-Version header field, which uses a version
          number to declare a message to be conformant with MIME
          and allows mail processing agents to distinguish
          between such messages and those generated by older or
          non-conformant software, which are presumed to lack
          such a field.

Any message containing any message header or MIME-part header field
beginning with "Content-" (case-insensitive) is also apparently intended
to be a MIME message, with one possible but highly unlikely exception
(details below); again quoting RFC 2045:

  MIME-extension-field := <Any RFC 822 header field which
                           begins with the string
                           "Content-">

  MIME-message-headers := entity-headers
                          fields
                          version CRLF
                          ; The ordering of the header
                          ; fields implied by this BNF
                          ; definition should be ignored.

  MIME-part-headers := entity-headers
                       [fields]
                       ; Any field not beginning with
                       ; "content-" can have no defined
                       ; meaning and may be ignored.
                       ; The ordering of the header
                       ; fields implied by this BNF
                       ; definition should be ignored.

Note that:
1. Any message containing any MIME extension field (i.e. beginning
   with "content") is supposed to contain a MIME-Version field in
   the message header
2. A MIME-Version field may not appear in a MIME-part header (it
   may of course appear in the message header of an encapsulated
  "message" composite media type).

One or both of those rules are sometimes broken by non-conforming
software.  One particular large vendor of broken software is notorious
for breaking the second rule.

Messages can be categorized as:
o MIME -- contains a MIME-Version field in the message header
o broken -- contains any Content- field but lacks a MIME-Version
  message header field (but see below re. Content-Type), or
  contains a MIME-Version field in a MIME-part header
o non-MIME -- no MIME-Version field (anywhere) and no fields
  (anywhere) beginning with case-insensitive "Content-", with
  the possible exception of Content-Type (in the absence of a
  MIME-Version message header field, a message with a [RFC 1049]
  Content-Type field and no other Content- fields is a non-MIME
  message).

Date: Fri, 24 Jun 2005 17:34:17 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: #1043 "header": Suggested resolution
Sender: [email-masked]

--On fredag, juni 24, 2005 09:25:38 -0400 Bruce Lilly <[email-masked]> 
wrote:

>> NEW:
>>
>>   NOTE: Older News specifications used the term "header" as a synonym
>>      for what [RFC2822] calls "header field". This document follows
>>      the terminology of RFC 2822.
>
> I suggest s/[RFC2822] calls/[RFC2822] and MIME call/ and I suggest
> adding "and MIME [RFC2045], [RFC2046], [RFC2047], [RFC2049], [RFC2231]"
> to the last sentence.  I'd also like to see an informative reference to
> FYI 18 added, but for the sake of conciseness of the document, I can
> live without it.

I'd consider such a change to be at best useless and at worst harmful.

If there was ever a conflict discovered between MIME and 2822 about what a 
"header field" was, this would make our "note" ambiguous.
And if there is no conflict, the change adds no information.

Date: Fri, 24 Jun 2005 11:11:59 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Steve Walker <[email-masked]>
cc: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

On Fri, 24 Jun 2005, Steve Walker wrote:
> ...I'm fully aware of the fact that you can have a mime message 
> without using base64.  I'm also aware of the fact that most 
> base64 encoded messages are also mime...

However, the reverse is not true:  most MIME messages do *not* use base64
encoding.  (Hint:  every message in this mailing list is a MIME message.)

> ...mime messages account for about 2% of Usenet articles.
> This proposal is trying to declare the 98% of usenet articles are 
> invalid.

What on Earth are you talking about?  There is nothing in our drafts that
makes a non-MIME article invalid.  Use of MIME -- MIME, not base64 -- is
mandatory only when non-ASCII characters are involved, which is precisely
when it *should* be mandatory, if only to specify what character set is
being used.

                                                          Henry Spencer
                                                       [email-masked]

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1043 "header": Suggested resolution
Date: Fri, 24 Jun 2005 09:25:38 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 24 2005 05:30, Harald Tveit Alvestrand wrote:

> NEW:
> 
>   NOTE: Older News specifications used the term "header" as a synonym
>      for what [RFC2822] calls "header field". This document follows
>      the terminology of RFC 2822.

I suggest s/[RFC2822] calls/[RFC2822] and MIME call/ and I suggest
adding "and MIME [RFC2045], [RFC2046], [RFC2047], [RFC2049], [RFC2231]"
to the last sentence.  I'd also like to see an informative reference to
FYI 18 added, but for the sake of conciseness of the document, I can
live without it.
 
> I'd also like to add two more definitions, but those are less important:
> 
>   This specification also uses the following terms:
> 
>   Header line - a line as defined in [RFC2822] section 2.1 that forms
>          part of a header field.

I'd change "header field" to "message header [RFC2822] or MIME-part-header
[RFC2046]".  ("field" in this case is unnecessary because fields are
delimited by line endings (not followed by whitespace))
 
>   Header field content - the field body of a header field, not including
>          leading or trailing whitespace, or comments.
> 
> I'm not sure whether the last one is what Charles wanted for his "semantic 
> content" construct - but I'm sure he'll tell us if it's not.

I'd lean more towards "Structured field body semantic content" for the
term, if we actually need such a term.  I'd also change "or comments" to
"with comments and their enclosing parentheses and any adjacent line
folding and/or whitespace replaced by a single space character" [see RFC
2822 section 3.2.3].

To: [email-masked]
Cc: Steve Walker <[email-masked]>
Subject: Re: Feedback on proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 24 Jun 2005 06:09:11 -0700
Sender: [email-masked]

Steve Walker <[email-masked]> writes:

> While I do not pretend to be a mime expert, I do know what mime is.  I'm
> fully aware of the fact that you can have a mime message without using
> base64.  I'm also aware of the fact that most base64 encoded messages
> are also mime and they are not welcome on Usenet.

Sorry, this is simply not true as universally as you're claiming.

> I installed a filter to count mime messages.  Of the 75,000 articles
> tested so far, mime messages account for about 2% of Usenet articles.

I find this incredibly unlikely.  How are you counting MIME messages?

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Subject: Re:  Meta (was: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues)
Date: Fri, 24 Jun 2005 13:24:58 +0200
Organization: Chaos rules.
Sender: [email-masked]

Frank Ellermann:
> Bruce Lilly:

>> <[email-masked]> posted to this WG's
>> mailing list and dated 27 Jun 1999 04:05:54 -0700.
> 
>> Unfortunately the online WG mailing list archive is
>> incomplete.
> 
> Try <http://mid.gmane.org/[email-masked]>
> for old articles.  In the resulting URL ..../number add /raw if
> you need the original header.  Or for ...:number add :raw.  Or
> use your news reader:
> <news://news.gmane.org/[email-masked]>

And for the whole enchilada: 
  news://news.gmane.org/gmane.ietf.usenet.format
(23000+ messages)

-- 
Grtz, Ruud

From: "Ruud H.G. van Tol" <[email-masked]>
To: "Usefor Mailing List" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Fri, 24 Jun 2005 12:51:20 +0200
Organization: Chaos rules.
Sender: [email-masked]

Henry Spencer:
> Russ Allbery:

>> Most Unix
>> systems will treat 123.045.67.89 as equivalent to 123.37.67.89.  :)
> 
> No

Yes. Try 'ping' for just one example of such treatment.

-- 
Grtz, Ruud

Date: Fri, 24 Jun 2005 11:30:19 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1043 "header": Suggested resolution
Sender: [email-masked]

Since several people weighed in on the subject after my summary, we seem to 
be moving towards a rough consensus, and I want to try for a resolution.... 
Henry and Charles have continued to argue for the "news" syntax; I'd like 
to check that they find it possible to work with the "2822" syntax....

I suggest the following text change to section 2.1, with the necessary 
changes to the places where the terms occur in the rest of the text:

OLD:

 This specification uses the terms "header", "header name", and
 "header content" which are synonymous with the [RFC2822] terms
 "header field", "field name", and "field body" respectively.

NEW:

  NOTE: Older News specifications used the term "header" as a synonym
     for what [RFC2822] calls "header field". This document follows
     the terminology of RFC 2822.

I'd also like to add two more definitions, but those are less important:

  This specification also uses the following terms:

  Header line - a line as defined in [RFC2822] section 2.1 that forms
         part of a header field.

  Header field content - the field body of a header field, not including
         leading or trailing whitespace, or comments.

I'm not sure whether the last one is what Charles wanted for his "semantic 
content" construct - but I'm sure he'll tell us if it's not.

Comments?

                     Harald

Date: Fri, 24 Jun 2005 09:35:36 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1048 Injection-Date syntax
Sender: [email-masked]

I have created ticket #1048 for this issue, with a "Discussion:" field of 
"Proposed no change", since that's what I ended up parsing Bruce's comment 
as.

This may be uncontroversial......

                 Harald

--On 23. juni 2005 13:19 -0400 Bruce Lilly <[email-masked]> wrote:

>
> On Thu June 23 2005 12:10, Charles Lindsey wrote:
>
>> As USEFOR stands, the software MAY choose not to accept things in
>> <obs-zone> (except for GMT) or it MAY choose to accept (some or all) of
>> them.
>>
>> 1. If it has chosen to accept EST from the <obs-zone>, then it will
>> interpret it as the offset given in RFC 2822 and, if it is a relaying or
>> serving agent, pass it on and/or store it as received (for an injecting
>> agent, I accept also the possibility that it might try to fix it).
>>
>> 2. The case that the sentence under dicussion has to cover is where it
>> has chosen not to "accept <obs-zone>". So what does it do when "EST" (or
>> even "XYZ") turns up?
>>
>> 2a. Clearly, dropping it on the floor immediately is one possibility, in
>> which case that is the end of the matter.
>>
>> 2b. So we are down to the case where the software is written to be as
>> liberal as possible. In that case, it will try to pass it on/store it
>> (leaving it to the next agent in line to make its own mind up about
>> accepting/rejecting it). But first it wants to do a staleness check, and
>> for that purpose the porposed sentence says it MAY interpret either EST
>> or XYZ as -0000.
>
> All of those things are protocol issues.  I propose that we defer detailed
> discussion of them to discussion of USEPRO.
>
> date-time appears in precisely three fields: Date, Expires, and
> Injection-Date.
>
> Date and Expires are legacy fields, and will need to be interpreted per
> long-standing rules.  RFC 2822 rules are perfectly suited to that.  RFC
> 2822 generation rules mean that those agents still using alphabetic
> abbreviations won't be conforming to the specification.  Good reason
> for them to be updated.  Nothing breaks whether or not they're updated.
> Of course, only conforming implementations can truthfully be claimed as
> conforming, but there's no enforcement...
>
> Injection-Date is a new field; rules could be anything we want -- there
> are few backward compatibility issues (the only one I can think of is
> really an implementation issue (conversion of software which currently
> uses the Date field)).  We could:
> 1. simply use the same RFC 2822 date-time syntax which is already
>    specified for the field in -04
> 2. devise something different
>
> As separately mentioned, we already discussed the syntax issue (earlier
> syntax was specified in human-unfriendly, somewhat vague form).
>
> We do need to decide on Injection-Date syntax for the USEFOR document.
> Charles appears to be proposing something somehow different from 2822
> date-time, although it's not clear exactly what.
>
> I propose that we use RFC 2822 date-time as it stands, for the following
> reasons:
> 1. compatible with existing parsers (same reason for the earlier syntax
>    discussion for Injection-Date)
> 2. compatible with the legacy Expires field (practical server expiration
>    and history will likely be a combination of local policy, article
>    Expires field, and receipt date, possibly Injection-Date field).
> 3. Human-readable (for tracing purposes)
>
> Discussion details re. injection and transport can be deferred to USEPRO,
> however for purposes of making a syntax decision for Injection-Date, I
> note that:
> o Injection-Date is a new field.  If we use RFC 2822 as-is, that means
> that   conforming implementations will only generate signed 4DIGIT numeric
>   offsets and (for that field and conforming implementations) the issue of
>   zone abbreviations arises only for the case on non-conforming
> generators.   I now yield further discussion of the treatment of such
> non-articles to   John Stanley.
> o Given the new nature of that specific field, a simple note that the
>   date-time-related obs- constructs do not apply to it precisely because
>   it is not a legacy field could appear in some document (but I think
>   that would be redundant).
> o We could, in the USEPRO document, give special leave to agents to use
>   a simplified parser for that field, one which does not handle the
>   alphabetic zone abbreviations (because it is not a legacy field).
>   Frankly I doubt that anybody will go through the trouble to write a
>   special-purpose parser given the ready availability of efficient and
>   tested date-time parsers.
>
> Regarding specific text changes to Injection-Date draft text, I propose
> no change (other than what might be required to the sentence/paragraph
> beginning "see the" due to other issues).  I do not know specifically
> what Charles proposes.
>
>
>

Date: Fri, 24 Jun 2005 09:04:27 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: John Stanley <[email-masked]>, [email-masked]
Subject: #1047 Path field delimiters - feed behaviour?
Sender: [email-masked]

Attempting to return to the problem at hand, rather than the discussion on 
how to discuss these things - John, can you educate me a little bit about 
USENET feeds?

--On 23. juni 2005 17:05 -0700 John Stanley <[email-masked]> wrote:

> It would seem to be unwise to use it in IPV6 addresses as well, since
> ::FFFF:129.144.52.38 and ::FFFF:129.145.23.23 are clearly two different
> addresses but the same failure that would make the dead:beef admin an
> idiot would make both of these people idiots. (I.e., non-conforming
> servers would match the "path-identity" of FFFF for both systems and
> feeds  would be disrupted.)

If, in a message arriving at the old system "here", there is a Path: header 
that says

Path: there!::FFFF:129.144.52.38!not-for-mail

and "here" does not know about any system named "FFFF", will any change in 
handling happen?

Conversely, if a message arrives with a Path: header that says

Path: there!::FFFF:129.144.52.38!::FFFF:129.145.23.23!not-for-mail

will the two occurences of "FFFF" in the Path: cause anything bad to happen?

Or is there some third circumstance that I haven't guessed at yet that will 
cause something bad to happen?

I'd like to understand what you mean when you say "feeds will be 
disrupted".....

                   Harald

Date: Fri, 24 Jun 2005 08:57:31 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: John Stanley <[email-masked]>, [email-masked]
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

--On 23. juni 2005 10:25 -0700 John Stanley <[email-masked]> wrote:

> Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:
>
>> While name-calling is not a good idea even when the person one is
>> insulting cannot be identified, anyone who generates something permitted
>> by the ABNF even when it's explicitly forbidden by the accompanying text
>> is certainly not exhibiting good judgment.
>
> I've read section 3.1.6 (Path) three times now, and I find no prohibition
> of any kind against a path-identity that contains a colon. Since 3.1.6 is
> the "accompanying text" to the ABNF I quoted, I would expect any
> prohibition to appear therein, and it does not. What "explicit"
> prohibition do you see in 3.1.6, or are you reading something other than
> draft-ietf-usefor-usefor-04.txt as found on the IETF website?

I think my point was that given that the colons in IPv6 literals are known 
to cause problems, and so far, no prohibition or warning is in the text, 
the standards writers (that's us) haven't exhibited good judgment.....

Date: Fri, 24 Jun 2005 08:53:42 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: John Stanley <[email-masked]>, [email-masked]
Subject: Re: Non-standard language justification (was Re: #1008 USEFOR 1.5 References: Final text on "followup")
Sender: [email-masked]

--On 23. juni 2005 10:39 -0700 John Stanley <[email-masked]> wrote:

> Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:
>
>> Version numbers are cheap (outside of the IETF pre-meeting blackouts),
>> so  we might as well just ship official versions (my opinion). I hope we
>> have  enough issues settled to make it worthwhile to ask our editor to
>> emit  usefor-05 before the Paris blackout.
>
> You do not understand. The issue is not "emitting" version numbers. The
> issue is that drafts that the editors send to IETF as our work product do
> NOT show up on the IETF-USEFOR website as the current draft. People who
> join this discussion today are going to go to the IETF-USEFOR website to
> look for the "latest complete draft", and they will find USEFOR-03.

That's another problem, and I'm trying to get that fixed (I got the 
password for the website yesterday).

My point is that I don't see a benefit from having two publication 
processes for WG drafts ("unofficial" and "official" drafts) when we have 
one process that works reasonably well, and is visible to the whole IETF 
community.

                        Harald

             Harald

Date: Fri, 24 Jun 2005 00:59:00 -0500
From: Steve Walker <[email-masked]>
To: Russ Allbery <[email-masked]>
CC: [email-masked]
Subject: Re: Feedback on proposal
Sender: [email-masked]

Russ Allbery wrote:
> Steve Walker <[email-masked]> writes:
> 
> 
>>Another example, requiring the use of MIME/base64 on Usenet articles is
>>against decades of practice.  As a matter, of fact we filter any base64
>>encoded article that arrives.  Base64 is considered the encoding method
>>of clueless and spamers on Usenet.
> 
> 
> I stopped reading at this point.  I used to spend lots of time educating
> people who didn't understand the difference between MIME and base64, but
> this is no longer a new concept.  I've stopped.  At this point,
> particularly if someone is going to claim to be some authority on how
> Usenet works, they should already understand this.

While I do not pretend to be a mime expert, I do know what mime 
is.  I'm fully aware of the fact that you can have a mime message 
without using base64.  I'm also aware of the fact that most 
base64 encoded messages are also mime and they are not welcome on 
Usenet.  Remember the problems when AOL released their mime 
enabled, base64 encoding, users onto Usenet several years back.

> If you had investigated a random newsgroup on your server before posting,
> you would discover that MIME messages are already the majority.  I picked
> rec.arts.poems at random, and out of 7,132 messages, 4,317 use MIME (61%).
> In other words, they're already existing best practice, and increasingly
> so.

I installed a filter to count mime messages.  Of the 75,000 
articles tested so far, mime messages account for about 2% of 
Usenet articles.

This proposal is trying to declare the 98% of usenet articles are 
invalid. I can't even imagine why anyone would even consider 
trying this.  If you really like mime, suggest that people use 
it, but don't attempt to require the impossible.

> Please check the rest of your objections against reality; you will find

I'm bringing up real world problems.  You can't claim to be the 
"standard" when 98% of the world is against it.

Steve Walker
News Admin
NNTPServer.com

Date: Thu, 23 Jun 2005 23:27:43 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

Charles Lindsey wrote:

> He should have known that ":" was a new feature of this standard, and that
> it used to be a delimiter (not that anybody ever used it as such). 

The drafts are intended to OBSOLETE old standards, not make them
normative references.

 > Just
 > because a thing is allowed in the format does not mean that it is wise to
 > use it.

If something is allowed in the format but it is not wise to use it,
there had better be something in USEFOR or USEPRO explaining it.  This
isn't "idiot-proofing". It is necessary, otherwise there is not a
complete and unambiguous specification.

To: Steve Walker <[email-masked]>
Cc: [email-masked]
Subject: Re: Feedback on proposal
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 23 Jun 2005 19:49:18 -0700
Sender: [email-masked]

Steve Walker <[email-masked]> writes:

> Another example, requiring the use of MIME/base64 on Usenet articles is
> against decades of practice.  As a matter, of fact we filter any base64
> encoded article that arrives.  Base64 is considered the encoding method
> of clueless and spamers on Usenet.

I stopped reading at this point.  I used to spend lots of time educating
people who didn't understand the difference between MIME and base64, but
this is no longer a new concept.  I've stopped.  At this point,
particularly if someone is going to claim to be some authority on how
Usenet works, they should already understand this.

If you had investigated a random newsgroup on your server before posting,
you would discover that MIME messages are already the majority.  I picked
rec.arts.poems at random, and out of 7,132 messages, 4,317 use MIME (61%).
In other words, they're already existing best practice, and increasingly
so.

Please check the rest of your objections against reality; you will find
that many of them are similarly mistaken or based on misunderstandings of
how existing software works.

Date: Thu, 23 Jun 2005 20:26:06 -0500
From: Steve Walker <[email-masked]>
To: [email-masked]
Subject: Feedback on proposal
Sender: [email-masked]

I just stumbled across your website/mailing list and thought I 
would share my first impression.  First of all, I've developed 
Usenet software and ran commercial news servers for many years, 
so I understand the protocols well.  While I haven't read the 
proposal 100% yet, what I have read very disappointing.  From 
what I can tell, your work group is going to do more damage to 
Usenet than good.  What is being proposed isn't Usenet at all. 
It's an entirely new protocol and to call it Usenet will just 
confuse people.  At least call it UseneXT so people won't try to 
use a real Usenet client/server with it.

I agree the old RFC's needed some modernization, but instead of 
working with the good and fixing the bad you are proposing a 
whole new, incompatible, protocol.  Usenet was designed around 
the ideas of using the minimal amount of unnecessary controls. 
It is this limited chaos which has allowed it to grow and adapt 
into what it is today.

Writing books of unnecessary and incompatible rules is going to 
fragment the software base.  On one hand you will have the people 
that read RFC's like holy bibles and on the other you will have 
people that say, I've been doing it like this for 15 years and 
not going to change.

Making rules that correspond with Generally Accepted Usenet 
Practices (GAUP) is a good thing.  Trying to force rules that 
break GAUP will divide the Usenet community and software 
developers.

For example, the GAUP on Message-ID length is less than 250 
bytes.  Formalizing this is RFC's is good.  The software 
developers can then use a fixed size buffers and know that if the 
message-id doesn't fit they should drop it anyway.

Another example, requiring the use of MIME/base64 on Usenet 
articles is against decades of practice.  As a matter, of fact we 
filter any base64 encoded article that arrives.  Base64 is 
considered the encoding method of clueless and spamers on Usenet.

Requiring a strict format of the Path header is pointless and 
places extra CPU load on the servers.  The same thing can be said 
by the dozens of nit-picking rules I saw.

Usenet has always been about requiring the very minimum and 
accepting everything.  You should be making rules that simplify 
software, not cause overly complected, buggy software to be 
developed.

As a news server operator I already have to deal with hundreds of 
news clients.  They can't get the existing protocol right, let 
alone follow the hundreds of new, pointless, rules in this 
proposal.  In the real world, the client creators will not 
support this proposal as it's too complex. It just won't happen 
and there is no need for them to.  The existing protocol works 
well, it just needs some minor fixes that simply life, not 
complicate it.

As for the servers I run, I have no plans on making any efforts 
to make changes as specified in these proposals.  That is, unless 
it becomes GAUP.  If you are writing this to see your name on 
electronic paper, then you will succeed.  If you are writing this 
to make a positive difference, I don't see that happening.  Take 
a step back and ask yourself, "when this is all done and all the 
server owners have spent hundreds of millions of dollars 
upgrading software, what has really changed?"  We still have a 
discussion system that grows faster than hardware capability.  We 
still have a million articles of spam per day.  We still have 
problems with article number synchronization on distributed 
systems. We still have stupid kids dumping 100 DVD images a day 
into a misspelled newsgroup that no one will read.

Why don't you use your efforts to solve real problems, instead of 
creating new ones.  If you are going to design an incompatible 
protocol at least fix something major in the original.

Steve Walker
News Admin
NNTPServer.com

Date: Thu, 23 Jun 2005 20:19:54 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1043 USEFOR 2.1 Use of the term "header" 
Sender: [email-masked]

On Thu, 23 Jun 2005, John Stanley wrote:
> >"There's no difference on the wire" is not the same as "there's no
> >justification".
> 
> Ummm, ok. So what? Who has said it was?

You did -- you quoted Harald (I think it was) saying "no difference on the
wire", and then *followed that up* by saying "since there's no
justification...".  When you quote part of a previous message before your
remarks, the usual assumption is that it's relevant to what you're saying. 

                                                          Henry Spencer
                                                       [email-masked]

Date: Thu, 23 Jun 2005 17:05:28 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>He should have known that ":" was a new feature of this standard,

It should not matter if ":" is a new or old feature. If the standard tells 
him that he can use ":" in a path-identity and he does so, he ought not to 
be called an idiot. If using ":" in a path-identity makes him an idiot, 
then the people who told him he could use it in a path identity are just 
as much idiots as he is. And given the current USEFOR-04, we are telling 
him he can use it.

>However, if you want to make the document idiot-proof, you could say:

What I want is for YOU to stop calling people who do things that our
standard tells them they can do "idiots". If Bruce is publicly flogged for 
saying something is "nonsense" (not a person, just their argument), then
calling people idiots is certainly not acceptable.

       Since older implementations may still treat
    it as a <path-delimiter>, it would be unwise to use it otherwise than
    as a part of an <IPv6address>.

It would seem to be unwise to use it in IPV6 addresses as well, since 
::FFFF:129.144.52.38 and ::FFFF:129.145.23.23 are clearly two different
addresses but the same failure that would make the dead:beef admin an 
idiot would make both of these people idiots. (I.e., non-conforming 
servers would match the "path-identity" of FFFF for both systems and feeds 
would be disrupted.) And yet, we seem to be bending over backwards so they 
can use these literals, but we won't call them idiots because, well, I 
don't know why.

Date: Thu, 23 Jun 2005 16:48:13 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1043 USEFOR 2.1 Use of the term "header" 
Sender: [email-masked]

Henry Spencer <henry@xxxxxxxxxxxxx>:

>"There's no difference on the wire" is not the same as "there's no
>justification".

Ummm, ok. So what? Who has said it was?

Ohhh, I get it. You've ignored the part of the discussion where I pointed
out that the differences from RFC2822 aren't fully documented in USEFOR-04
and the justifications for any differences are completely absent, and
you think I'm saying "there's no justification" not because there is not a
single word of justification but because of some irrelevant matter like
"same on the wire" or something.

You win. They aren't the same. Next non-sequitor?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Definition of "private" (Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts)
Date:  Fri, 24 Jun 2005 00:32:13 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 30
Sender: [email-masked]

Charles Lindsey wrote:

> Has the alt group reached article number 2600 yet?

Found in <news:[email-masked]> :
Xref: mephistopheles.news.clara.net rec.arts.poems:468350 alt:152780

Looking closer I found not one (out of five, then I had
enough) article restricted to alt.  X-Posts as a side
effect of typos, comma instead of dot in an alt.all NG.

> As to whether the group "general" remains a part of
> Usenet is debateable.

Maye, but not in an IETF standard, unless you have some
technical reasons.  Catching comma instead of dot typos
isn't very convincing.

| There are nearly 500 different newsgroups, with subjects
| ranging from UNIX internals to politics to hobbies
[...]
| There may be also groups found only at your site.  One
| group, called 'general', is fairly likely to exist.
[...]
| Newsgroup names have one or more parts separated by periods.

Quoth Tim O'Reilly 1992, chapter 7, Introduction to Usenet.

                         Bye, Frank

Date: Thu, 23 Jun 2005 18:12:52 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: Non-standard language justification
Sender: [email-masked]

On Thu, 23 Jun 2005, John Stanley wrote:
> >No, that part isn't a problem.
> 
> Yes, it is. You may not care that things you've never seen are presented 
> to the world as if you agreed with them, but I do.

And if I've seen them, and disagreed with them, and argued against
them, and voted against them, they're still presented exactly the same
way.

Seth

Date: Thu, 23 Jun 2005 17:56:35 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Non-standard language justification 
Sender: [email-masked]

On Thu, 23 Jun 2005, John Stanley wrote:
> >No, that part isn't a problem.
> 
> Yes, it is. You may not care that things you've never seen are presented 
> to the world as if you agreed with them, but I do.

There is nothing in a *draft* -- note that word -- which says that the
entire WG agrees with it.  (Actually, even a finished RFC doesn't imply
unanimous consent, since that is not the IETF definition of "consensus".)

To quote what I said about this a year ago, the last time you brought
this up:

  That's the way IETF drafts work -- they are documents for discussion 
  and review, not official products that the WG is deemed to stand 
  behind.  They get sent to IETF as drafts, as a way of making them 
  available, not because they are "our product".

  Is the word "draft" so hard to understand?

  Why have two levels of draft process when one suffices?  There's no 
  need to have "tentative" drafts and "real" ones -- a draft is 
  tentative by definition.

                                                          Henry Spencer
                                                       [email-masked]

Date: Thu, 23 Jun 2005 14:15:44 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Non-standard language justification 
Sender: [email-masked]

Henry Spencer <henry@xxxxxxxxxxxxx>:

>No, that part isn't a problem.

Yes, it is. You may not care that things you've never seen are presented 
to the world as if you agreed with them, but I do.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Thu, 23 Jun 2005 17:08:02 -0400
Cc: [email-masked]
Sender: [email-masked]

On Thu June 23 2005 14:43, Frank Ellermann wrote:

> > I've suggested avoiding the problem(s) (performance as well
> > as the IPv6 issue) by explicitly treating the path identities
> > as not IP addresses.
> 
> There are _many_ <path-identity> in usefor-04, usepro-03, and
> already "approved" modifications of these texts.  Changing its
> semantics now is a dubious idea.

Let's stick to USEFOR for the moment.

> But that's also not necessary for your idea, we can get this
> effect by saying that a <path-identity> is compared octet by
> octet (BTW, same idea as for a msg-id), and so it should be
> always created in the same way.

That's neither necessary nor sufficient for the purposes the Path
field has.  The purposes listed in -04 3.1.6 impose first and
foremost a uniqueness requirement.  That in turn requires either
a hierarchical system (like DNS) or a central authority (like the
UUCP mapping project was supposed to be, though it never worked
well).  As mentioned about a year ago, I'd be happy to have the
draft say "MUST be a fully-qualified domain name" [*], thereby avoiding
the problems with IP address literals and UUCP node names, and
satisfying the uniqueness requirement.  As a nice side-effect,
it precludes conflict with the newfangled "POSTED" and "MISMATCH"
keywords (as those aren't FQDNs).

> It's IMO unnecessary to invent any "canonical representation"
> for this simple purpose.  Add a general rule discouraging
> domain literals, because that's the only case where it gets
> really tricky, and it should be good enough.

Merely "discouraging" or even "STRONGLY DISCOURAGED" 127.0.0.1
won't satisfy the uniqueness requirement.

> The added colon for IPv6 has to be mentioned in a "differences
> from 1036" section.

Not if the difference (and associated backward compatibility
issues) are removed.

> And at the moment "dead", "beef", "cafe", "affe", etc. are no
> FQDNs.

Neither is "sn-us", but that doesn't prevent its use.

> If some poor old UUCP host in the last world map 1999 
> had one of these names it's tough but we can't change it.  We
> could document it if you think that it's a real problem today,
> not only with some scripts trying to analyze historical paths.

All things considered, I think requiring FQDN (for conforming
implementations) is the best approach.
 
> Inventing our own IPv6 path-id convention based on s/:/_/ is
> too horrible to consider it.

It might not be so bad it the use were restricted to literals used
to document validation failures.  Indeed, since underscore is not
valid in DNS labels, it would avoid confusion between such literals
and FQDNs.  As IPv4 addresses can be mapped to IPv6 (RFC 3513 sect.
2.5.5), that can also be used for IPv4 literals for documenting
validation failures, e.g. 129.144.52.38 -> ::FFFF:129.144.52.38 ->
__FFFF_129.144.52.38 which is no FQDN.

---------
* probably with an implementation note indicating that unqualified
names may exist, that there is no established mechanism to avoid
clashes, etc.

To: [email-masked]
Xref: clerew local.usefor:21439
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Definition of "private" (Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts)
Date: Thu, 23 Jun 2005 16:19:02 GMT
Lines: 41
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> <newsgroup-name>s of the following forms SHOULD NOT appear
>> in Newsgroups and Followup-To headers in articles intended
>> for propagation throughout the public Usenet.
>[...]
>> any <newsgroup-name> having only one <component>

>As stated I disagree.  Except from some random junk like "fi"
>or "free" there are apparently valid NGs "alt" and "general"
>on news.clara.net (checked today, only for non-empty groups).

Has the alt group reached article number 2600 yet?

Yes for sure servers do contain "random junk" and I doubt clara.net is the
worst by any means.

As to whether the group "general" remains a part of Usenet is debateable.
If there is content in it, do the Paths indicate wide propagation? If so,
it would be interesting to look at some other servers and see if they have
essentially the same set of articles.

But, "general" apart, I think current practice on Usenet is that
single-component groups are not supposed to be there, and my wording was
intended to reflect that. But, as I said, my wording is up for debate.

And perhaps clara.net has a "mutual agreement" with a limited set of peers
to carry some "unusual" groups :-) .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21440
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Thu, 23 Jun 2005 16:30:19 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>If the sites "dead" and
>>"beef" find that some idiot has invented a site "dead:beef",

>I'm curious. Is this "idiot" an idiot because they are doing something 
>explicitely allowed in the USEFOR standard, or for some other unspecified 
>reason?

He should have known that ":" was a new feature of this standard, and that
it used to be a delimiter (not that anybody ever used it as such). Just
because a thing is allowed in the format does not mean that it is wise to
use it.

However, if you want to make the document idiot-proof, you could say:

    NOTE: ":" has been included in <path-identity> so as to permit an
    <IPv6address> to be used. Since older implementations may still treat
    it as a <path-delimiter>, it would be unwise to use it otherwise than
    as a part of an <IPv6address>.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date:  Thu, 23 Jun 2005 20:43:55 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 45
Sender: [email-masked]

Bruce Lilly wrote:

[I've reset the subject because we're back to the old topic]

> The problem is that if we try to say something resembling
> "IPv4 is supported, IPv6 is not"

Oh, yes, that won't fly.  It's much worse than omitting an
empty IANA considerations, claiming that CRAM-MD5 is secure,
and getting the IPR boilerplate wrong put together.

> I've suggested avoiding the problem(s) (performance as well
> as the IPv6 issue) by explicitly treating the path identities
> as not IP addresses.

There are _many_ <path-identity> in usefor-04, usepro-03, and
already "approved" modifications of these texts.  Changing its
semantics now is a dubious idea.

But that's also not necessary for your idea, we can get this
effect by saying that a <path-identity> is compared octet by
octet (BTW, same idea as for a msg-id), and so it should be
always created in the same way.

It's IMO unnecessary to invent any "canonical representation"
for this simple purpose.  Add a general rule discouraging
domain literals, because that's the only case where it gets
really tricky, and it should be good enough.

The added colon for IPv6 has to be mentioned in a "differences
from 1036" section.  That's no showstopper, we have these IPv6
domain literals everywhere in the syntax.

And at the moment "dead", "beef", "cafe", "affe", etc. are no
FQDNs.  If some poor old UUCP host in the last world map 1999
had one of these names it's tough but we can't change it.  We
could document it if you think that it's a real problem today,
not only with some scripts trying to analyze historical paths.

Inventing our own IPv6 path-id convention based on s/:/_/ is
too horrible to consider it.  A "MUST NOT domain literal" in
the path might be worse, I still like "STRONGLY DISCOURAGED".

                         Bye, Frank

Date: Thu, 23 Jun 2005 11:04:13 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

Bruce Lilly <blilly@xxxxxxxxx>:

>> There is, 998 is the precise limit (unfolded without CrLf) for
>> the References and at least some news servers.

>The RFC 2822 998 limit is per line, not unfolded.
 
While RFC2822 may have a 998 per line limit, USEFOR-04 has a 998 per 
"header" limit. In section 2.2:

   o  Compliant software MUST NOT generate (but MAY accept) headers of
      more than 998 octets. 

But just earlier:

   This specification uses the terms "header", "header name", and
   "header content" which are synonymous with the [RFC2822] terms
   "header field", "field name", and "field body" respectively.

So the USEFOR limit is on the "header field" (as used in RFC2822), not the
individual line in that field, which is what RFC2822 puts its limit on. In
other words, USEFOR has a stricter limit on this than RFC2822.

And that isn't just for References and a few others, it applies to all.

>As mentioned earlier, we have a rather long-standing agreement that
>there is no field length limit.  Reintroducing one *is* controversial.

I don't know about "reintroduced". I see it in USEFOR-03, so it's been
around for awhile. I don't recall an agreement, but then, I don't
particularly care if there is a limit or not, just that we don't look
stupid by saying there is and then is not a limit in the same section.

Which comes from the direct contradiction in a NOTE that follows:

         NOTE: There is NO restriction on the number of lines into which
         a header may be split, and hence there is NO restriction on the
         total length of a header (in particular it may, by suitable
         folding, be made to exceed the 998 octets restriction
         pertaining to a single header line).
                                                                                
You cannot "suitably fold" something to create something that you MUST NOT 
create in the first place. 

But I mentioned this yesterday.

Cc: [email-masked]
To: Russ Allbery <[email-masked]>
Date: Thu, 23 Jun 2005 11:02:24 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: #1043 USEFOR 2.1 Use of the term "header"
Sender: [email-masked]

> > - Use the RFC 2822 terms "header field", "field name" and "field body"
> > for the terms currently defined in USEFOR 2.1.

> > The argument in favour is that this conforms with RFC 2822 and documents
> > that derive their terminology from that (and coincidentally also allows
> > us to define "header content" differently than "field body", if we need
> > that - Charles wants such a term).

> > Seen in this "camp": Forrest, Bruce, Richard and John Stanley

> I think we should use the RFC 2822 terminology as well.

Agreed. The RFC 2822 terms are actually pretty good.

				Ned

Date: Thu, 23 Jun 2005 14:01:32 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Non-standard language justification (was Re: #1008 USEFOR 1.5 References: Final text on "followup")
Sender: [email-masked]

On Thu, 23 Jun 2005, John Stanley wrote:
> Except, of course, that nobody here will have seen it before it goes, 
> other than the editor, so there is still a problem.

No, that part isn't a problem.  There is no reason to have a pre-draft
review:  the purpose of drafts is to make documents available for review!

                                                          Henry Spencer
                                                       [email-masked]

Date: Thu, 23 Jun 2005 13:57:44 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

On Thu, 23 Jun 2005, Frank Ellermann wrote:
> Yes, I know this, it's no 2822 or MIME limit.  Maybe Russ or
> Henry have an idea.  S-o-1036 mumbles something about a "length
> of header logical lines", maybe that's the source of this limit
> 1000 for unfolded References on some news servers (?)

Most likely.  In hindsight, the "if you really must" part of that
discussion in son-of-1036 isn't useful:  an injecting agent wants to know
what it can depend on, so if the standard grants even reluctant permission
to impose a limit, in practice that's a hard limit. 

> All I want is that this SHOULD not be the problem of the user.
> Either UAs get it right, or servers stop to reject articles
> with longer References.  MAY (or maybe not) isn't good enough.

Concur.  The standard needs to state what servers *must* accept, and
consequently what injecting agents can *always* generate.

                                                          Henry Spencer
                                                       [email-masked]

Date: Thu, 23 Jun 2005 10:39:36 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Non-standard language justification (was Re: #1008 USEFOR 1.5 References: Final text on "followup")
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>Version numbers are cheap (outside of the IETF pre-meeting blackouts), so 
>we might as well just ship official versions (my opinion). I hope we have 
>enough issues settled to make it worthwhile to ask our editor to emit 
>usefor-05 before the Paris blackout.

You do not understand. The issue is not "emitting" version numbers. The
issue is that drafts that the editors send to IETF as our work product do 
NOT show up on the IETF-USEFOR website as the current draft. People who 
join this discussion today are going to go to the IETF-USEFOR website to 
look for the "latest complete draft", and they will find USEFOR-03. 

And after they've taken the time to read and digest that document, if they
comment on anything it says, they will be told that "that's not the
current document" and that they ought to "keep up" with the discussion. 

This is a historical problem on this list. We'd get a "current version" on
the website, we'd go through a month or three of discussion where the
editor would announce "I've changed the text to say X", and then perhaps
"I've changed the text to say Y" (referring to the same text), or even
just "I've made some minor editorial changes", and then someone who hasn't
read every article and marked up his own local copy of the "current draft"
would comment on that text and be chastised for not "keeping up". This
problem was insolvable because the editor keeps the text in "several
files" and it would require updating the website at five minute intervals
(or so we were told.) Once a day was simply out of the question.

And now we cannot update the website within three weeks of a draft (which 
we never got to see prior to it being sent off, by the way) being sent to 
the IETF as our official latest draft.

THAT'S the problem. Emitting USEFOR-05 prior to France isn't the problem.
Except, of course, that nobody here will have seen it before it goes, 
other than the editor, so there is still a problem.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: Meta (was: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues)
Date: Thu, 23 Jun 2005 13:32:21 -0400
Cc: [email-masked]
Sender: [email-masked]

> > I suspect that would run afoul of IETF provisions requiring
> > equal support for IP versions.
> 
> It's also an odd idea, some problems for IPv4 like dynamic IPs
> or private IPs behind NAT should not exist for most IPv6 IPs.

Off-topic: some people aren't convinced that IPv6 means the end of
NAT.
 
> > If supporting IPv6 address literals is a problem, supporting
> > IPv4 address literals (as address literals) is a problem.
> 
> Sorry, I don't get this, yes, there are problems, that's why I
> want the STD 11 "STRONGLY DISCOURAGED".  For the fun effect of
> screaming in a stadard maybe literally.

The problem is that if we try to say something resembling "IPv4
is supported, IPv6 is not"  it is virtually guaranteed that during
IETF Last Call and/or IESG review somebody will point out the
requirement for treating IPv6 the same as IPv4 and will go to the
trouble of looking up the exact reference.  It is likely to be a
contentious issue.

I've suggested avoiding the problem(s) (performance as well as the
IPv6 issue) by explicitly treating the path identities as not IP
addresses.  And I've seen some support for that (we can debate
"public name" vs. "opaque string" etc. if and when we reach consensus
on "not IP address literals").

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date:  Thu, 23 Jun 2005 19:28:55 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 28
Sender: [email-masked]

Bruce Lilly wrote:
 
> that's not a ticket item nor a declaration of consensus.

2005-04-27, Harald introduced this ticket monster later.  If
you want it to get a ticket number you're supposed to use a
Subject: NEW: USEPRO 7.6 (4) ...

Maybe a bit premature, usepro-03 still says "MAY", the new
text exists only as proposal.  But I know that it's "SHOULD".

Then Alexey is supposed to find any Subject: NEW: USEPRO and
create a usepro-ticket if it's a real issue.

> The RFC 2822 998 limit is per line, not unfolded.

Yes, I know this, it's no 2822 or MIME limit.  Maybe Russ or
Henry have an idea.  S-o-1036 mumbles something about a "length
of header logical lines", maybe that's the source of this limit
1000 for unfolded References on some news servers (?)

All I want is that this SHOULD not be the problem of the user.

Either UAs get it right, or servers stop to reject articles
with longer References.  MAY (or maybe not) isn't good enough.

                         Bye, Frank

Date: Thu, 23 Jun 2005 10:25:29 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>While name-calling is not a good idea even when the person one is 
>insulting cannot be identified, anyone who generates something permitted 
>by the ABNF even when it's explicitly forbidden by the accompanying text 
>is certainly not exhibiting good judgment.

I've read section 3.1.6 (Path) three times now, and I find no prohibition
of any kind against a path-identity that contains a colon. Since 3.1.6 is
the "accompanying text" to the ABNF I quoted, I would expect any
prohibition to appear therein, and it does not. What "explicit"  
prohibition do you see in 3.1.6, or are you reading something other than 
draft-ietf-usefor-usefor-04.txt as found on the IETF website?

Now, RFC1036 tells us that currently a ':' is a punctuation character and 
will be treated as a separator, and thus while it is not prohibited in the 
path-identity it will result in incorrect parsing of the path header 
field. 

I also see that UUCP does not prohibit colons in node names, although I 
found none in the last copy of the UUCP maps I have on hand. My archaic 
HDB UUCP system seems quite happy to know about a site called "ded:bef", 
and will spool items up for that site, but perhaps the failure will come 
when an actual transfer is attempted. 

So, calling someone an "idiot" for doing something explicitely allowed in 
the standard is patently wrong and offensive to every person who would try 
to conform to our standard. After all, "conform and be an idiot" is a very 
negative message to send.

Henry Spencer <henry@xxxxxxxxxxxxx> jumps in:

>A bit of background you might not be aware of:  in some earlier
>discussions (on other topics), it was surprisingly difficult to convince
>some participants that such accompanying text is actually part of the
>standard, not just helpful commentary or advice to implementers. 

No, what was hard to accomplish was to justify the ABNF and the text
saying contradictory things. Your snide comment could be viewed as 
insulting, were this a group where insults weren't allowed.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: NEW: Injection-Date syntax (Was Re: #1028 Usefor 3.1.2 Timezones: Suggested text)
Date: Thu, 23 Jun 2005 13:19:50 -0400
Cc: [email-masked]
Sender: [email-masked]

On Thu June 23 2005 12:10, Charles Lindsey wrote:

> As USEFOR stands, the software MAY choose not to accept things in
> <obs-zone> (except for GMT) or it MAY choose to accept (some or all) of
> them.
> 
> 1. If it has chosen to accept EST from the <obs-zone>, then it will interpret
> it as the offset given in RFC 2822 and, if it is a relaying or serving
> agent, pass it on and/or store it as received (for an injecting agent, I
> accept also the possibility that it might try to fix it).
> 
> 2. The case that the sentence under dicussion has to cover is where it has
> chosen not to "accept <obs-zone>". So what does it do when "EST" (or even
> "XYZ") turns up?
> 
> 2a. Clearly, dropping it on the floor immediately is one possibility, in which
> case that is the end of the matter.
> 
> 2b. So we are down to the case where the software is written to be as liberal
> as possible. In that case, it will try to pass it on/store it (leaving it
> to the next agent in line to make its own mind up about
> accepting/rejecting it). But first it wants to do a staleness check, and
> for that purpose the porposed sentence says it MAY interpret either EST or
> XYZ as -0000.

All of those things are protocol issues.  I propose that we defer detailed
discussion of them to discussion of USEPRO.

date-time appears in precisely three fields: Date, Expires, and
Injection-Date.

Date and Expires are legacy fields, and will need to be interpreted per
long-standing rules.  RFC 2822 rules are perfectly suited to that.  RFC
2822 generation rules mean that those agents still using alphabetic
abbreviations won't be conforming to the specification.  Good reason
for them to be updated.  Nothing breaks whether or not they're updated.
Of course, only conforming implementations can truthfully be claimed as
conforming, but there's no enforcement...

Injection-Date is a new field; rules could be anything we want -- there
are few backward compatibility issues (the only one I can think of is
really an implementation issue (conversion of software which currently
uses the Date field)).  We could:
1. simply use the same RFC 2822 date-time syntax which is already
   specified for the field in -04
2. devise something different

As separately mentioned, we already discussed the syntax issue (earlier
syntax was specified in human-unfriendly, somewhat vague form).

We do need to decide on Injection-Date syntax for the USEFOR document.
Charles appears to be proposing something somehow different from 2822
date-time, although it's not clear exactly what.

I propose that we use RFC 2822 date-time as it stands, for the following
reasons:
1. compatible with existing parsers (same reason for the earlier syntax
   discussion for Injection-Date)
2. compatible with the legacy Expires field (practical server expiration
   and history will likely be a combination of local policy, article
   Expires field, and receipt date, possibly Injection-Date field).
3. Human-readable (for tracing purposes)

Discussion details re. injection and transport can be deferred to USEPRO,
however for purposes of making a syntax decision for Injection-Date, I
note that:
o Injection-Date is a new field.  If we use RFC 2822 as-is, that means that
  conforming implementations will only generate signed 4DIGIT numeric
  offsets and (for that field and conforming implementations) the issue of
  zone abbreviations arises only for the case on non-conforming generators.
  I now yield further discussion of the treatment of such non-articles to
  John Stanley.
o Given the new nature of that specific field, a simple note that the
  date-time-related obs- constructs do not apply to it precisely because
  it is not a legacy field could appear in some document (but I think
  that would be redundant).
o We could, in the USEPRO document, give special leave to agents to use
  a simplified parser for that field, one which does not handle the
  alphabetic zone abbreviations (because it is not a legacy field).
  Frankly I doubt that anybody will go through the trouble to write a
  special-purpose parser given the ready availability of efficient and
  tested date-time parsers.

Regarding specific text changes to Injection-Date draft text, I propose
no change (other than what might be required to the sentence/paragraph
beginning "see the" due to other issues).  I do not know specifically
what Charles proposes.

Date: Thu, 23 Jun 2005 13:08:06 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1043 USEFOR 2.1 Use of the term "header"
Sender: [email-masked]

On Thu, 23 Jun 2005, John Stanley wrote:
> >As I said - this won't change the protocol on the wire one bit. But we 
> >need to stop talking and decide.
> 
> Since there is no justification for the difference, why is there a 
> question about there being a difference?

"There's no difference on the wire" is not the same as "there's no
justification".  All questions of terminology affect only human
understanding of the specs; none of them alter what's on the wire. 

                                                          Henry Spencer
                                                       [email-masked]

Date: Thu, 23 Jun 2005 09:56:41 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: #1043 USEFOR 2.1 Use of the term "header"
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>As I said - this won't change the protocol on the wire one bit. But we 
>need to stop talking and decide.

Since there is no justification for the difference, why is there a 
question about there being a difference?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Meta (was: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues)
Date:  Thu, 23 Jun 2005 18:44:53 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 42
Sender: [email-masked]

Bruce Lilly wrote:

> <[email-masked]> posted to this WG's
> mailing list and dated 27 Jun 1999 04:05:54 -0700.

> Unfortunately the online WG mailing list archive is
> incomplete.

Try <http://mid.gmane.org/[email-masked]>
for old articles.  In the resulting URL ..../number add /raw if
you need the original header.  Or for ...:number add :raw.  Or
use your news reader:

<news://news.gmane.org/[email-masked]>

>> There's something to be said for supporting IPv4 for legacy
>> reasons but telling people using IPv6 that, sorry, they have
>> to come up with a regular name.
 
> I suspect that would run afoul of IETF provisions requiring
> equal support for IP versions.

It's also an odd idea, some problems for IPv4 like dynamic IPs
or private IPs behind NAT should not exist for most IPv6 IPs.

> If supporting IPv6 address literals is a problem, supporting
> IPv4 address literals (as address literals) is a problem.

Sorry, I don't get this, yes, there are problems, that's why I
want the STD 11 "STRONGLY DISCOURAGED".  For the fun effect of
screaming in a stadard maybe literally.

But the problems for IPv4 and IPv6 are completely different.

BTW, the old "is a leading zero octal" question made it into an
I-D (or even RfC), and STD 66 offers a nice unambiguous syntax.

For a laugh see the "octal"-part in:
<http://mid.gmane.org/[email-masked]>

                    Bye, Frank

Date: Thu, 23 Jun 2005 12:44:24 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Sender: [email-masked]

On Thu, 23 Jun 2005, Harald Tveit Alvestrand wrote:
> So maybe we can change [to]:
> Software that accepts dates with unknown timezones SHOULD treat such
>  timezones as equivalent to "-0000", as specified in RFC 2822 section 4.3.

That seems sensible.

> It may have some value pointing out that we're not changing anything.... 
> shortening it to just "SHOULD be handled as 2822 4.3 says" makes the 
> document harder to read.

Agreed.  It may be more elegant in some sense never to say anything that
2822 already says, but in practice, it's terribly bad document design to
force people to flip back and forth (and search for the relevant sentence
of 2822; 4.3 is a page and a half long) even for small points. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
Xref: clerew local.usefor:21438
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Date: Thu, 23 Jun 2005 16:10:06 GMT
Lines: 48
Sender: [email-masked]

In <[email-masked]> Seth Breidbart <[email-masked]> writes:

>"Charles Lindsey" <[email-masked]> wrote:

>> Software MAY interpret an unknown timezone as "-0000" for the purpose of
>> comparing one <date-time> with another.

>How else is it allowed to treat it?  If it's not going to just drop it
>on the floor for having an unknown zone, it's supposed to do a
>comparison.  Are we specifying that software is allowed to treat EST
>as +4600 if it wants?

As USEFOR stands, the software MAY choose not to accept things in
<obs-zone> (except for GMT) or it MAY choose to accept (some or all) of
them.

1. If it has chosen to accept EST from the <obs-zone>, then it will interpret
it as the offset given in RFC 2822 and, if it is a relaying or serving
agent, pass it on and/or store it as received (for an injecting agent, I
accept also the possibility that it might try to fix it).

2. The case that the sentence under dicussion has to cover is where it has
chosen not to "accept <obs-zone>". So what does it do when "EST" (or even
"XYZ") turns up?

2a. Clearly, dropping it on the floor immediately is one possibility, in which
case that is the end of the matter.

2b. So we are down to the case where the software is written to be as liberal
as possible. In that case, it will try to pass it on/store it (leaving it
to the next agent in line to make its own mind up about
accepting/rejecting it). But first it wants to do a staleness check, and
for that purpose the porposed sentence says it MAY interpret either EST or
XYZ as -0000.

I think #1, #2a and #2b cover all the scenarios we want to envisage. If
you can think of a different scenario, then let's hear it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Thu, 23 Jun 2005 12:26:03 -0400
Cc: [email-masked]
Sender: [email-masked]

> Bruce Lilly wrote:
[quoting C. Lindsey} 
> >> I have put the requirement for trimming in there (it was
> >> optional before, though recommended in USEAGE) because Frank
> >> asked for something stronger (and nobody has disputed that).
> 
> > I recall no ticket item for the issue, no discussion, and no
> > declaration of consensus.  References?
> 
> <http://mid.gmane.org/[email-masked]> and many
> others, scan the list for "trim".

Thanks for the reference, Frank, but that's not a ticket item nor
a declaration of consensus.
 
> > There is no such thing as "too long" for a field.
> 
> There is, 998 is the precise limit (unfolded without CrLf) for
> the References and at least some news servers.

The RFC 2822 998 limit is per line, not unfolded.
 
> > The procedure should be discuss first, and change the
> > document (which is supposed to reflect WG consensus) only
> > after consensus is declared
> 
> That's the procedure for controversial issues.

As mentioned earlier, we have a rather long-standing agreement that
there is no field length limit.  Reintroducing one *is* controversial.

> In that case 
> there was no controversy:  Injecting agents rejecting articles
> with too long References are bad for the users, therefore the
> followup agent SHOULD get it right.  Not only MAY.  That was
> the only change.

Charles' text (quoted a few messages back in the (untrimmed)
References field of this message) says "MUST".  Presumably we
have tickets so there can be organized discussion, a clear decision,
and text reflecting that decision.  Not something completely
different.  

To: [email-masked]
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 23 Jun 2005 09:09:04 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:
> On Thu June 23 2005 10:30, Russ Allbery wrote:

>> That depends on your implementation of address conversion.  Most Unix
>> systems will treat 123.045.67.89 as equivalent to 123.37.67.89.  :)
>> (Which just goes to illustrate your point.)

> Actually, my point was that *IF* one allows treatment of path components
> as IP address literals, *THEN* a simple textual comparison is
> insufficient *AND THEREFORE* there will be a performance penalty.  I
> don't know of any Unix (or other) systems that would treat
> "123.45.67.89" and "123.045.67.89" as equivalent as text.

I agree with this.

>> I'm not sure that I've really gone on the record being *opposed* to
>> them; I just think they're a bad idea (which isn't quite the same thing
>> as saying that they shouldn't be allowed).  IP addresses are used
>> somewhat currently.

> *As* IP address literals (with eliding leading zeroes, reverse lookup to
> check against known names, etc.)?  Or as textual labels?

As textual labels.

> Reference:
>  "What are you talking about? �I've been opposed to using IP addresses in
>  the Path except for entries that fail validation since the beginning, and
>  I've never been worried about long Path headers."
> from message <[email-masked]> posted to this WG's
> mailing list and dated 27 Jun 1999 04:05:54 -0700.

> Unfortunately the online WG mailing list archive is incomplete.

> Apologies if I have unintentionally misinterpreted your position, Russ.

You haven't, really; there are just various definitions of "opposed"
happening.  I don't think IP addresses belong in the Path header; IP
addresses are, among other things, too transitory and too difficult to
trace compared to good textual names.  *However*, there are some
situations where using the IP address as a Path identity might be a
reasonable choice, or there might be some reason why one really wants an
IP address not as a Path identity for optimization but rather for trace
information (such as, as mentioned, including additional trace information
for Path mismatches -- IPv6 *is* a problem for that, actually).

Although, that being said, it's worth noting that nearly all of my
opinions on how best to write these documents have changed since 1999; I
was naive and far too optimistic about some things six years ago, and in
many cases have changed my opinion completely.

> I suspect that would run afoul of IETF provisions requiring equal
> support for IP versions.  If supporting IPv6 address literals is a
> problem, supporting IPv4 address literals (as address literals) is a
> problem.

Provisions for legacy support may well be exempt from those sorts of
rules, if that's the only reason why IP addresses are supported.  (But
that might require some other solution to the tracing problem.)

Date: Thu, 23 Jun 2005 11:51:58 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

Henry Spencer wrote:
> On Thu, 23 Jun 2005, Harald Tveit Alvestrand wrote:
> 
>>...anyone who generates something permitted by the ABNF 
>>even when it's explicitly forbidden by the accompanying text is certainly 
>>not exhibiting good judgment.
> 
> 
> A bit of background you might not be aware of:  in some earlier
> discussions (on other topics), it was surprisingly difficult to convince
> some participants that such accompanying text is actually part of the
> standard, not just helpful commentary or advice to implementers. 
> 

In a discussion on the References field, Charles wrote this on 6/13/05
in <[email-masked]> indicating that he considers notes
to have no normative effect.  I don't think that Charles would
consider the main text not normative.

 > If all References headers are constructed according to the preceding
 > paragraphs, then circular references cannot arise. If, by some accident,
 > one does arise, then "garbage in, garbage out" applies as usual.
 >
 > But that paragpraph is just a NOTE, so it has no normative effect (and I
 > would prefer it to have been said in USEFOR - see my reply to Forrest).
 >

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1043 USEFOR 2.1 Use of the term "header"
Date:  Thu, 23 Jun 2005 17:59:19 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
 
> I'm tempted to invoke the decision making procedure of
> RFC 3929 section 4.4 and flip a coin...

It only irritates me if somebody insists on adding "field"
in a perfectly clear context like "Date header".  Otherwise
Bruce brainwashed me enough to use the 2822-terminology, so
it's more like "moo" than "yes" or "no" in my case.

Removing me and adding Russ & Scott you get a 2:6 for Bruce
and his "hacker slang".
                       Bye, Frank

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Thu, 23 Jun 2005 11:59:54 -0400
Cc: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

On Thu June 23 2005 10:17, Bruce Lilly wrote:

> colon delimiter) is interpreted as a host name, and an upstream neighbor
> of a site with such a host name will not receive such articles *at all*,
> nor will sites downstream of the named host who are dependent of that host
> for a feed of the affected newsgroups and/or distributions.

Correction:  the upstream neighbor won't send, and the named site (and its
downstream dependents) won't receive the articles.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date:  Thu, 23 Jun 2005 17:41:47 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 33
Sender: [email-masked]

Bruce Lilly wrote:

>> I have put the requirement for trimming in there (it was
>> optional before, though recommended in USEAGE) because Frank
>> asked for something stronger (and nobody has disputed that).

> I recall no ticket item for the issue, no discussion, and no
> declaration of consensus.  References?

<http://mid.gmane.org/[email-masked]> and many
others, scan the list for "trim".

> There is no such thing as "too long" for a field.

There is, 998 is the precise limit (unfolded without CrLf) for
the References and at least some news servers.

We have some text around this magic number, e.g. the 1+2=3
msg-ids in the References were chosen to be below this limit
in the worst case, 764 = 3*250+2+12, 12=length('References: ').
It also works for 974 = 3*(64+1+255)+14, discussed months ago.

> The procedure should be discuss first, and change the
> document (which is supposed to reflect WG consensus) only
> after consensus is declared

That's the procedure for controversial issues.  In that case
there was no controversy:  Injecting agents rejecting articles
with too long References are bad for the users, therefore the
followup agent SHOULD get it right.  Not only MAY.  That was
the only change.
                          Bye, Frank

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Russ Allbery <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Thu, 23 Jun 2005 11:26:24 -0400
Cc: [email-masked]
Sender: [email-masked]

On Thu June 23 2005 10:30, Russ Allbery wrote:
> 
> Bruce Lilly <[email-masked]> writes:
> 
> > An IP address literal (any version) in path would have to be treated
> > specially for the widely-implemented purpose of determining if an
> > article should be propagated to neighbors. 123.45.67.89 and
> > 123.045.67.89 are different bits of text, but equivalent IP address
> > literals.
> 
> That depends on your implementation of address conversion.  Most Unix
> systems will treat 123.045.67.89 as equivalent to 123.37.67.89.  :)
> (Which just goes to illustrate your point.)

Actually, my point was that *IF* one allows treatment of path components
as IP address literals, *THEN* a simple textual comparison is insufficient
*AND THEREFORE* there will be a performance penalty.  I don't know of any
Unix (or other) systems that would treat "123.45.67.89" and "123.045.67.89"
as equivalent as text.

> You have to treat Path identities as opaque labels; anything else gets
> into way too much work to specify and implement, IMO.

It appears that we're agreed that treatment as IP address literals is
"way too much work".  We might differ regarding "opaque labels" vs.
"public names".
 
> I'm not sure that I've really gone on the record being *opposed* to them;
> I just think they're a bad idea (which isn't quite the same thing as
> saying that they shouldn't be allowed).  IP addresses are used somewhat
> currently.

*As* IP address literals (with eliding leading zeroes, reverse lookup to
check against known names, etc.)?  Or as textual labels?

Reference:
 "What are you talking about? �I've been opposed to using IP addresses in
 the Path except for entries that fail validation since the beginning, and
 I've never been worried about long Path headers."
from message <[email-masked]> posted to this WG's
mailing list and dated 27 Jun 1999 04:05:54 -0700.

Unfortunately the online WG mailing list archive is incomplete.

Apologies if I have unintentionally misinterpreted your position, Russ.

> There's something to be said for supporting IPv4 for legacy reasons but
> telling people using IPv6 that, sorry, they have to come up with a regular
> name.

I suspect that would run afoul of IETF provisions requiring equal support
for IP versions.  If supporting IPv6 address literals is a problem,
supporting IPv4 address literals (as address literals) is a problem.

To: Usefor Mailing List <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 23 Jun 2005 08:18:36 -0700
Sender: [email-masked]

Henry Spencer <[email-masked]> writes:
> On Fri, 17 Jun 2005, Russ Allbery wrote:

>> "Just send eight" is common in world-wide newsgroups just as much as it
>> is in more limited hierarchies.  That issue is not related to
>> cooperating subnets and the concept of cooperating subnets is not
>> particularly helpful in resolving or working around it.

> Sure it is.  A cooperating subnet can establish a standard convention on
> just which 8-bit character set is in use.  No such convention is
> possible in a worldwide standard; some form of character-set identifier
> in the article is essential.

Establishing such a convention is exactly how just-send-eight is being
handled in world-wide newsgroups right now, and it works about as well in
practice as I'd expect it to work with a cooperating subnet.  I really
don't see the difference.  In both cases, you encounter exactly the same
problems with having to custom-configure otherwise general software.

See the character set guessing code in Gnus, which maps unknown character
sets to different local character sets based on the newsgroup hierarchy.
This is exactly the same as what would be needed for a cooperating subnet;
the benefits and problems are pretty much exactly identical.

Date: Thu, 23 Jun 2005 17:12:44 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Cc: Henry Spencer <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Sender: [email-masked]

--On torsdag, juni 23, 2005 10:22:37 -0400 Bruce Lilly <[email-masked]> 
wrote:

> 2822, in discussion of obs-zone, says:
>
>    Other multi-character (usually between 3 and 5) alphabetic time zones
>    have been used in Internet messages.  Any such time zone whose
>    meaning is not known SHOULD be considered equivalent to "-0000"
>    unless there is out-of-band information confirming their meaning.
>
> Aside from "+" vs. "-", do you believe that there is some difference in
> the meaning of the text from the above 2822 quote?

I didn't remember that particular piece of 2822. Now that I read it - no, I 
don't think there is a difference.

Since Dan Bernstein's "-0000" got accepted to the degree that it is used 
here, I think pointing to this section (2822 section 4.3) is a perfectly 
reasonable thing to do.

So maybe we can change:

> Software that accepts dates with unknown timezones SHOULD treat such
>  timezones as equivalent to "+0000".

To:

Software that accepts dates with unknown timezones SHOULD treat such
 timezones as equivalent to "-0000", as specified in RFC 2822 section 4.3.

It may have some value pointing out that we're not changing anything.... 
shortening it to just "SHOULD be handled as 2822 4.3 says" makes the 
document harder to read.

Date: Thu, 23 Jun 2005 11:02:48 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Sender: [email-masked]

On Fri, 17 Jun 2005, Russ Allbery wrote:
> > Except that it *can't* be, because practices which we *cannot* endorse
> > as the worldwide standard are common in some of those cooperating
> > subnets -- for example, the "just send eight" approach...
> 
> "Just send eight" is common in world-wide newsgroups just as much as it is
> in more limited hierarchies.  That issue is not related to cooperating
> subnets and the concept of cooperating subnets is not particularly helpful
> in resolving or working around it.

Sure it is.  A cooperating subnet can establish a standard convention on
just which 8-bit character set is in use.  No such convention is possible
in a worldwide standard; some form of character-set identifier in the
article is essential.

                                                          Henry Spencer
                                                       [email-masked]

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1043 USEFOR 2.1 Use of the term "header"
Date: Thu, 23 Jun 2005 10:48:55 -0400
Cc: [email-masked]
Sender: [email-masked]

On Thu June 23 2005 09:17, Harald Tveit Alvestrand wrote:

> - Use the RFC 2822 terms "header field", "field name" and "field body" for 
> the terms currently defined in USEFOR 2.1.
> 
> The argument in favour is that this conforms with RFC 2822 and documents 
> that derive their terminology from that
[...]
> Are there others than the 7 individuals above who have a strong opinion one 
> way or the other, or are there people among the 7 individuals who feel that 
> I've mischaracterized their position?

As I have tried to make clear, the issue goes beyond "RFC 2822 and documents
that derive their terminology from that":
o The suggestion is consistent with the Internet user's glossary (FYI 18)
o It facilitates understanding of the normative references which must
  themselves be understood by implementers (since that is the meaning of
  "normative")
o It is terminology used in the MIME documents, which are independent of
  2822 and which are themselves normative references.  See:
  http://www.imc.org/ietf-usefor/2003/Feb/0827.html
o It is strongly advocated by the Internet Architecture RFC (1958) section
  3.13.

> As I said - this won't change the protocol on the wire one bit.

True, but the WG's charter is to produce IETF Standards Track specifications,
and that itself is a point in favor of using consistent standard terminology
as opposed to "colloquial usage".

If the argument in favor were based solely on 2822 it would be a weaker
argument than it is.

Date: Thu, 23 Jun 2005 10:46:42 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

On Thu, 23 Jun 2005, Russ Allbery wrote:
> > ...123.45.67.89 and
> > 123.045.67.89 are different bits of text, but equivalent IP address
> > literals.
> 
> That depends on your implementation of address conversion.  Most Unix
> systems will treat 123.045.67.89 as equivalent to 123.37.67.89.  :)

No, they are unquestionably equivalent -- this notation for IP addresses
is known as "dotted decimal" (see, e.g., RFC 1123), not "dotted decimal,
octal, or hex".  RFC 1166, in particular, explicitly states that
128.009.000.032 and 128.9.0.32 are equivalent.  (I had cause to do some
research on this some years ago on another project.)

The Unix implementation's interpretation of leading zeros is simply wrong.

> You have to treat Path identities as opaque labels; anything else gets
> into way too much work to specify and implement, IMO.

Now this, I concur on.

                                                          Henry Spencer
                                                       [email-masked]

Date: Thu, 23 Jun 2005 10:32:55 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

On Thu, 23 Jun 2005, Harald Tveit Alvestrand wrote:
> ...anyone who generates something permitted by the ABNF 
> even when it's explicitly forbidden by the accompanying text is certainly 
> not exhibiting good judgment.

A bit of background you might not be aware of:  in some earlier
discussions (on other topics), it was surprisingly difficult to convince
some participants that such accompanying text is actually part of the
standard, not just helpful commentary or advice to implementers. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 23 Jun 2005 07:30:08 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:

> An IP address literal (any version) in path would have to be treated
> specially for the widely-implemented purpose of determining if an
> article should be propagated to neighbors. 123.45.67.89 and
> 123.045.67.89 are different bits of text, but equivalent IP address
> literals.

That depends on your implementation of address conversion.  Most Unix
systems will treat 123.045.67.89 as equivalent to 123.37.67.89.  :)
(Which just goes to illustrate your point.)

> IPv6 is much more complex as there are literally dozens of ways to
> represent a given IPv6 literal.  A neighbor might not be known by IP
> address but by name, so treatment if IP address literals would involve a
> DNS lookup in in-addr.arpa.

You have to treat Path identities as opaque labels; anything else gets
into way too much work to specify and implement, IMO.

> No. Frank is on recent record as being opposed to IP addresses in path.
> You seem to have agreed "Absolutely" to treatment of path components
> "purely as public names, not as IP address literals".  Russ is also on
> record as being opposed to IP addresses.  Now where is this supposed
> "benefit"?

I'm not sure that I've really gone on the record being *opposed* to them;
I just think they're a bad idea (which isn't quite the same thing as
saying that they shouldn't be allowed).  IP addresses are used somewhat
currently.

There's something to be said for supporting IPv4 for legacy reasons but
telling people using IPv6 that, sorry, they have to come up with a regular
name.

> On Wed June 22 2005 17:26, Charles Lindsey wrote:
>> So you would rather throw out a much needed facility than put up with a
>> rather improbable minor inconvenience? Please come and join us in the
>> Real World for a change.

> Please support your "much-needed" assertion.

Yeah, I'd like to see some support for that too.  I'm not seeing a clamor
from IPv6 sites wanting to put address literals into the Path header.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Date: Thu, 23 Jun 2005 10:22:37 -0400
Cc: Henry Spencer <[email-masked]>, Usefor Mailing List <[email-masked]>
Sender: [email-masked]

On Thu June 23 2005 07:17, Harald Tveit Alvestrand wrote:

> Note that the following times:
> 
> 12:00:00 +0000
> 12:00:00 -0000
> 08:00:00 -0400
> 
> are all equal according to RFC 2822.

Yes.

> -0000 means "I'm not telling you my  
> local timezone", not "I'm uncertain about the time".

The actual 2822 text is:
   Though "-0000" also indicates Universal Time, it is
   used to indicate that the time was generated on a system that may be
   in a local time zone other than Universal Time and therefore
   indicates that the date-time contains no information about the local
   time zone.

> The text proposed says that the following timezone:
> 
> 12:00:00 NONSENSE
> 
> can also be considered equal to the above, if one chooses to accept it at 
> all.

2822, in discussion of obs-zone, says:

   Other multi-character (usually between 3 and 5) alphabetic time zones
   have been used in Internet messages.  Any such time zone whose
   meaning is not known SHOULD be considered equivalent to "-0000"
   unless there is out-of-band information confirming their meaning.

Aside from "+" vs. "-", do you believe that there is some difference in the
meaning of the text from the above 2822 quote?

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Thu, 23 Jun 2005 10:17:03 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 22 2005 17:26, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >On Fri June 17 2005 14:50, Frank Ellermann wrote:
> 
> >> Charles Lindsey wrote:
> >> 
> >> > Hold on! We never agreed that domain literals were a Bad
> >> > Thing to use in an <id-right>.
> 
> >Frank has a valid point; 127.0.0.1, 192.168.0.1, 10.0.0.1,
> >172.16.0.1, 192.0.0.1, 224.0.0.1, 169.254.0.1, and many similar
> >IP address literals lack the uniqueness characteristic necessary
> >for some uses (including msg-id domain lierals).
> 
> Indeed, there are lots of stupid domain literals that should not be used

Precisely what is the purpose of the word "stupid" in your remarks?

[colon in path components (currently (1036) a delimiter)]
> >> It's bad in the path-identifier where it breaks Bruce's B-News
> >> monster in the case of IPv6 colons.
> 
> >It breaks C news and INN as well.  Moreover, as recently reaffirmed
> >by Russ, INN has no special treatment of numeric-plus-dots names in
> >the Path field due to severe performance issues associated with
> >such special treatment.
> 
> I do not recall that there was any performance issue. They are just a
> string of characters like any other <path-identity>.

If, and only if, there is an explicit statement that the components are
names, not IP address literals.  As noted elsewhere, the -04 USEFOR
draft text seems to be acceptable, but might benefit from clarification.
If treated as address literals, there are several severe performance
penalties...

> >....  Russ has noted that DNS lookups can take quite
> >a long time.  The implications for path field components is clear:
> >they have to be treated purely as public names, not as IP address
> >literals.
> 
> Absolutely. But I don't expect DNS lookups for <path-identity>s on a
> routine basis

An IP address literal (any version) in path would have to be treated
specially for the widely-implemented purpose of determining if an
article should be propagated to neighbors. 123.45.67.89 and 123.045.67.89
are different bits of text, but equivalent IP address literals.  IPv6
is much more complex as there are literally dozens of ways to represent
a given IPv6 literal.  A neighbor might not be known by IP address but
by name, so treatment if IP address literals would involve a DNS lookup
in in-addr.arpa.

> Though when it comes to implementing our new arrangements for catching
> MISMATCHes, there will be a little extra work to do - probably involving
> a cache of known path-identities for each peering site.

> >  That obviates any argument about backwards compatibility
> >breaking reassignment of colon (ostensibly for IPv6 literals), and
> >it means that I don;t have to remind Charles that his claim that
> >handling of e.g. dead:beef would affect only one site and not sites
> >named "dead" or "beef" or their neighbors and downstream sites has
> >been twice debunked before he recently repeated it.
> 
> I think it has always been well understood that the benefit of being able
> to handle IPv6 addresses outweighed some possible inconvenience to sites
> that used some rather unusual <path-identity>s.

No. Frank is on recent record as being opposed to IP addresses in path.
You seem to have agreed "Absolutely" to treatment of path components "purely
as public names, not as IP address literals".  Russ is also on record as
being opposed to IP addresses.   Now where is this supposed "benefit"?

> If the sites "dead" and 
> "beef" find that some idiot has invented a site "dead:beef",

What purpose is served by use of the derogatory term "idiot"?

> and if they 
> find that they are receiving too many articles twice over in consequence,
> then they will have to deal with it as best they can.

Sigh.  No Charles, as has been explained to you twice before, the issue is
neither "only affects that site" nor "receiving too many articles twice
over", it is that if some combination of characters (hexadecimal digits)
appears next to a colon in Path, the sequence of characters (excluding the
colon delimiter) is interpreted as a host name, and an upstream neighbor
of a site with such a host name will not receive such articles *at all*,
nor will sites downstream of the named host who are dependent of that host
for a feed of the affected newsgroups and/or distributions.  I.e. it breaks
the flooding algorithm by changing the meaning of a long-standing delimiter.
It breaks backwards compatibility.
 
> It is already 
> well-known that there are idiot sites out there, and Usenet has always
> managed to work its way around them.

Does your continued use of offensive language serve any purpose?
 
> >Specific change to USEFOR -04 text: remove colon from path-identity
> >ABNF production.
> 
> So you would rather throw out a much needed facility than put up with a
> rather improbable minor inconvenience? Please come and join us in the Real
> World for a change.

Please support your "much-needed" assertion.

Your "minor inconvenience" claim is a false assertion, as has been explained
to you in detail at least twice before, and now again.

Your "join us in the real world for a change" snide remark is highly
offensive and totally uncalled for.
 
> >The second note claims that component names "are traditionally in
> >lower case", but I've seen mixed case and upper-case names.  I'd
> >like to see that text elided. possibly replaced with a statement
> >that a choice of name of "POSTED", "MISMATCH", or even "PoStEd"
> >would be a really bad idea.  That of course assumes that we want
> >to continue pursuing introduction of those keywords.
> 
> Again, there is a limit as to how far one should go to deal with idiotic
> sysadmins.

Is that supposed to be a substantive comment of some sort?  If so I
fail to see the substance.   It sounds like yet another uncalled for
insult.

Date: Thu, 23 Jun 2005 16:15:18 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: John Stanley <[email-masked]>, [email-masked]
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

--On onsdag, juni 22, 2005 17:23:48 -0700 John Stanley <[email-masked]> 
wrote:

>
>
> "Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:
>
>> If the sites "dead" and
>> "beef" find that some idiot has invented a site "dead:beef",
>
> I'm curious. Is this "idiot" an idiot because they are doing something
> explicitely allowed in the USEFOR standard, or for some other unspecified
> reason?
>
> I mean, if:
>
>    path-identity   =  ( ALPHA / DIGIT )
>                       *( ALPHA / DIGIT / "-" / "." / ":" / "_" )
>
>  and someone actually follows that rule to create a path-identify for his
> site that causes problems for existing sites, why is HE the idiot?

While name-calling is not a good idea even when the person one is insulting 
cannot be identified, anyone who generates something permitted by the ABNF 
even when it's explicitly forbidden by the accompanying text is certainly 
not exhibiting good judgment.

On the other hand, if the standards writers intended to forbid it, and 
standard does not specifically say so, the standards writers are not 
showing good judgment.....

Date: Thu, 23 Jun 2005 10:11:59 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Sender: [email-masked]

On Thu, 23 Jun 2005, Harald Tveit Alvestrand wrote:
> > I go along with the earlier suggestion that this should be "-0000", which
> > is RFC2822ese for "treat as GMT but actual zone uncertain".  In reader
> > agents, in particular, the distinction might be useful.
> 
> 12:00:00 +0000
> 12:00:00 -0000
> 08:00:00 -0400
> are all equal according to RFC 2822.

In the sense that they represent the same time, yes.  But there is extra
information there, *beyond* just the time, and in that respect they are
three different values.  To a reader agent, the first means "this guy is
on GMT", the third means "this guy is on EDT", and the second means "the
time is expressed in GMT but that might not be his real time zone". 

> -0000 means "I'm not telling you my 
> local timezone", not "I'm uncertain about the time".

I was speaking of the meaning to the *reader*, which is the relevant one
here, since we're talking about how to handle unknown zone indications. 
To a reader, -0000 means simply "sender's time zone is uncertain". 

Even a sender doesn't *necessarily* use -0000 to mean "I know but I'm not
telling".  It's conceivable for a system with users from multiple zones to
put that in because it isn't sure of the *user's* correct zone. 

                                                          Henry Spencer
                                                       [email-masked]

From: "Scott Hollenbeck" <[email-masked]>
To: "'Harald Tveit Alvestrand'" <[email-masked]>, [email-masked]
Subject: RE: #1043 USEFOR 2.1 Use of the term "header"
Date: Thu, 23 Jun 2005 10:00:06 -0400
Sender: [email-masked]

> -----Original Message-----
> From: Harald Tveit Alvestrand [mailto:[email-masked]] 
> Sent: Thursday, June 23, 2005 9:18 AM
> To: [email-masked]
> Subject: #1043 USEFOR 2.1 Use of the term "header"

[snip]

> Are there others than the 7 individuals above who have a 
> strong opinion one 
> way or the other, or are there people among the 7 individuals 
> who feel that 
> I've mischaracterized their position?

Area advisor opinion: after re-reading the working group charter I can
understand why this has been contentious.  The charter says:

"The Goal of this working group is to publish a standards-track successor to
RFC 1036 that with particular attention to backward compatibility,
formalizes best current practice and best proposed practice."

Ignoring the "that with" error, that could be read to imply that there's a
requirement to use whatever terms have found their way into "best current
practice".  However, 2822 is a normative reference in usefor.  I'm thus
inclined to believe that usefor should use the same terminology used in 2822
where appropriate, or it should explicitly explain why different terminology
is being used and precisely where the deviations exist.

Not a position, just advice.

-Scott-

Date: Thu, 23 Jun 2005 15:59:19 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked], Frank Ellermann <[email-masked]>
Subject: #1047: Path field delimiters and components
Sender: [email-masked]

Assigned ticket #1047 to this issue.

--On tirsdag, juni 21, 2005 21:33:08 -0400 Bruce Lilly <[email-masked]> 
wrote:

>> It's bad in the path-identifier where it breaks Bruce's B-News
>> monster in the case of IPv6 colons.
>
> It breaks C news and INN as well.  Moreover, as recently reaffirmed
> by Russ, INN has no special treatment of numeric-plus-dots names in
> the Path field due to severe performance issues associated with
> such special treatment.  Note that the Path field as currently
> defined has no provision for comments; the justification given for
> that in earlier drafts was that the processing to treat comments
> as whitespace (i.e. essentially to ignore them) would be an onerous
> performance problem.  Russ has noted that DNS lookups can take quite
> a long time.  The implications for path field components is clear:
> they have to be treated purely as public names, not as IP address
> literals.  That obviates any argument about backwards compatibility
> breaking reassignment of colon (ostensibly for IPv6 literals), and
> it means that I don;t have to remind Charles that his claim that
> handling of e.g. dead:beef would affect only one site and not sites
> named "dead" or "beef" or their neighbors and downstream sites has
> been twice debunked before he recently repeated it.
>
> Specific change to USEFOR -04 text: remove colon from path-identity
> ABNF production.
>
> As the text currently doe not mention IP address literals and does
> use "name" (albeit only in a note), no other change is necessary
> (semantic clarification that components are public names would
> however be welcome).
>
> The second note claims that component names "are traditionally in
> lower case", but I've seen mixed case and upper-case names.  I'd
> like to see that text elided. possibly replaced with a statement
> that a choice of name of "POSTED", "MISMATCH", or even "PoStEd"
> would be a really bad idea.  That of course assumes that we want
> to continue pursuing introduction of those keywords.

Date: Thu, 23 Jun 2005 15:56:30 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: #1046: USEFOR 5. MIME boundary security considerations
Sender: [email-masked]

Ticket #1046 assigned.

--On tirsdag, juni 21, 2005 19:45:55 +0200 Frank Ellermann 
<[email-masked]> wrote:

>
> Harald Tveit Alvestrand wrote:
>
>> If you want to raise a new issue, and want to make sure it
>> gets a ticket, please use a subject line saying something
>> like
>> NEW: USEPRO 3.14.159 Section numbering is irrational
>
> Last proposed "security considerations" addition was in
> <http://mid.gmane.org/[email-masked]>
>
>  MIME security considerations are discussed in [RFC2046].  Note that
>  applying some [RFC2231] extensions for parameters like multi-line
>  paramters on a boundary parameter as defined in [RFC2046] might be
>  abused to bypass naive methods of handling parameters.  Two examples:
>
>  Content-Type:
>   multipart/mixed (; boundary=foo ; xyz=");bOuNdArY*=''next%20part(")
>  Content-Type: multipart/digest;
>   boundary (not=me) = ("yes ;-) simple (foo;bar") ; x-foo = xyzzy
>
>
>

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Date: Thu, 23 Jun 2005 09:50:15 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 22 2005 16:08, Harald Tveit Alvestrand wrote:
> 
> Bruce,
> 
> --On 21. juni 2005 22:20 -0400 Bruce Lilly <[email-masked]> wrote:
> 
> > Such attempts violate several principles which we've agreed on:
> > 1. stay as close to 2822+MIME as possible.
> >    In this case, there's no need for a deviation; 2822 requires
> >    accepting "UT" and the standard 3-letter abbreviations.
> 
> Section 2.1 of usefor-04 is very clear:
> 
>    News articles MUST conform to the syntax specified in Section 3 of
>    [RFC2822].  Netnews agents MAY also accept the obsolete syntax
>    specified in Section 4 of [RFC2822], but they MUST NOT generate
>    productions of such syntax.
> 
> This is a deviation from RFC 2822, which says "MUST accept the obsolete 
> syntax". Since this apparently had a previous consensus of the WG, I don't 
> see that you can say that we have agreement on the principle.

I'm not convinced that there was any consensus on such a sweeping
statement.  With the exception of unquoted dot in obs-phrase, the
2822 obs-syntax covers deprecated constructs which were legal in 822.
In particular, "Netnews" user "agents" are going to need to be able
to handle the legal zone abbreviations which appear in archived
articles and which are still in current use.  It might be reasonable
to make specific allowances for *transport* agents...

> > 2. where we have good cause to deviate, the article format is more
> >    restrictive.
> >    Allowing generation of something which is prohibited for generation
> >    by 2822 is clearly not more restrictive
> 
> Makes sense to me; others may have other opinions.
> Are you suggesting (specific text change) that the doc says that systems 
> MUST NOT generate GMT?

I'm suggesting that for the purpose of the syntax & semantics document
(USEFOR) we simply use RFC 2822 date-time by reference.  Protocol-related
(e.g. transport) restrictions on use of format provisions in transport-
related fields can be specified in USEPRO, subject of course to discussion
and WG consensus.

> > 3. backwards compatibility (see our charter).  Differences cause problems
> >    not only for UAs and gateways, but for messaging protocols (e.g. IMAP).
> 
> Are you referring to this charter text?
> 
> >The Goal of this working group is to publish a standards-track
> >successor to RFC 1036 that with particular attention to backward
> >compatibility, formalizes best current practice and best proposed practice.

Yes.  N.B. "with particular attention to backward compatibility".
 
> > 4. RFC 2822, MIME, IMAP, etc. are outside of our bailiwick; we have no
> >    authority to change them.
> >    orig-date and zone are part of 2822.
> 
> Are you proposing that we delete the ABNF description?

For orig-date and zone (both defined in RFC 2822), yes.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Thu, 23 Jun 2005 09:34:17 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 22 2005 15:18, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >On Tue June 14 2005 09:05, Frank Ellermann wrote:
> >> 
> >> There's a dispute about the meaning of <id-right>.  I say it's
> >> in fact an <id-domain>, a proper name space with an owner.
> >> 
> >> TTBOMK Charles says, whatever it is, in a msg-id it's case
> >> sensitive.
> 
> >RFC 2822 is crystal clear on the semantics and the nature of the
> >syntax differences from 822 and 2822 angle-addr:
> 
> >   Since the msg-id has
> >   a similar syntax to angle-addr (identical except that comments and
> >   folding white space are not allowed), a good method is to put the
> >   domain name (or a domain literal IP address) of the host on which the
> >   message identifier was created on the right hand side of the "@", and
> >   put a combination of the current absolute date and time along with
> >   some other currently unique (perhaps sequential) identifier available
> >   on the system (for example, a process id number) on the left hand
> >   side.
> 
> In RFC 2822 an <id-right> is a string of characters satisfying some ABNF.

The grammar production named id-right is an ABNF rule name, yes.

> It has no semantics attached to it (except that a `\` in a <quoted-pair>
> is "invisible").

No, the semantics are "the domain name (or a domain literal IP address)".

> If you generate such an <id-right> by "putting" a <domain> 
> in there, it is still an <id-right> (even if it happens to look like a
> <domain>) and it still has no further semantics, and hence no requirement
> to be considered case insensitively at any stage.

References to specific RFC 2822 text supporting your claims, please.
 
> In our draft, the same is true (except that we have fixed the
> <quoted-pair> problem). But, for the removal of all doubt, we have also
> excluded your bizarre interpretation by stating the intent that an
> octet-by-octet comparison will always suffice.

Kindly refrain from ad-hominem attacks.  There is no "bizarre
interpretation", a msg-id has had a domain name or domain literal
on the RHS since RFC 822 (prior to which the RHS was a host name,
as that was before DNS).  RFC 2822 has not changed the semantics,
contrary to your unsupported assertion (RFC 2822 Appendix B contains
a list of changes, and semantics of msg-id is not among them).  2822
dis change a very specific part of the syntax, which as noted is
"identical except" for those specific syntax changes.

> >In RFC 822, the syntax was in fact identical; because of input specifically
> >related to Usenet the syntax was changed to exclude CFWS.   The semantics
> >hasn't changed,
> 
> Yes, it has changed (whether intended so or not).

Repeating an unsupported assertion does not add to the discussion.  If
you cannot support your assertion, please retract it so we can move on.

> >Given the stated methodology and the rationale for it (especially
> >"otherwise result in subtle differences and interoperability challenges"),
> >as well as the basis of our document supposed to be 2822, and the
> >suggestion made years ago by Mark Crispin, viz:
> 
> >  NNTP Transport Requirements
> 
> >  � �The BLURDYBLOOP command in NNTP uses the message-id as an argument.
> >�   �The syntax for this command does not provide any way for spaces to
> >�   �appear in the message-id. �Therefore, message-ids in the Message-ID,
> >�   �In-Reply-To, and References headers of a message MUST NOT contain
> >�   �the space character.
> 
> >I can see no reason for an incompatible redefinition in our document.
> 
> The differences in the <msg-id> in our document are essential for
> technical reasons that have been explained so many times that I will not
> repeat them here.

IOW, more unsupported assertions, not even repeated in this case.
 
> Those concessions turned out to be insufficient for the essential
> requirement of Netnews that an octet-by-octet comparison of <msg-id>s
> would always suffice.

Kindly support your assertions.
 
> >My recommendation is to leave msg-id as defined in our primary normative
> >reference RFC 2822 and to add such notes along the lines suggested by
> >Mark as may be required,
> 
> No, Mark made his suggestion (in the NNTP list, I believe)

Mark's suggestion was made here, and is archived in this WG's mailing list
archive at
http://www.imc.org/ietf-usefor/2003/Feb/0493.html

> in the belief 
> that he was describing a temporary problem that would be fixed in future
> implementations.

That is a false assertion; Mark specifically referred to a transport
protocol constraint, as his text clearly shows.

> He was rapidly disabused of that notion. 

References, please.

To: [email-masked]
Subject: Re: #1043 USEFOR 2.1 Use of the term "header"
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 23 Jun 2005 06:33:54 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> - Use the RFC 2822 terms "header field", "field name" and "field body"
> for the terms currently defined in USEFOR 2.1.

> The argument in favour is that this conforms with RFC 2822 and documents
> that derive their terminology from that (and coincidentally also allows
> us to define "header content" differently than "field body", if we need
> that - Charles wants such a term).

> Seen in this "camp": Forrest, Bruce, Richard and John Stanley

I think we should use the RFC 2822 terminology as well.

Date: Thu, 23 Jun 2005 15:17:52 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1043 USEFOR 2.1 Use of the term "header"
Sender: [email-masked]

[I've created a new ticket number for the "header" issue, because I'm not 
sure that's all there is to Bruce's complaint in #1031. I can't tell from 
what he wrote.]

This is the kind of issue that is most irritating of all, because the group 
is split down the middle, there are reasonable arguments on both sides, the 
issue has to be decided.... and whatever we do, the decision won't change 
one bit on the wire, so there are no technical issues involved that could 
be used to argue strongly for one side or another.

The two positions I've heard:

- Use the terms "header", "header name" and "header content" the way they 
are currently defined in USEFOR

The argument in favour is that this conforms most closely with previous 
News documents and colloquial usage. There are also some mail standards 
that use "header" this way.

Seen in this "camp": Henry, Frank and Charles

- Use the RFC 2822 terms "header field", "field name" and "field body" for 
the terms currently defined in USEFOR 2.1.

The argument in favour is that this conforms with RFC 2822 and documents 
that derive their terminology from that (and coincidentally also allows us 
to define "header content" differently than "field body", if we need that - 
Charles wants such a term).

Seen in this "camp": Forrest, Bruce, Richard and John Stanley

I'm tempted to invoke the decision making procedure of RFC 3929 section 4.4 
and flip a coin... but that might be considered a too flippant way of 
making the call.

Are there others than the 7 individuals above who have a strong opinion one 
way or the other, or are there people among the 7 individuals who feel that 
I've mischaracterized their position?

As I said - this won't change the protocol on the wire one bit. But we need 
to stop talking and decide.

                          Harald

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Thu, 23 Jun 2005 09:09:17 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 22 2005 15:25, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

> >The result of which was to affirm no such limits, and we moved on from
> >there.  Now you're trying to introduce a per-field limit. Why -- what
> >has changed recently to justify reintroducing something that was long
> >ago rejected?
> 
> It is indeed rather unusual to impose any limit (other than an advisory
> one) on the overall length of a header.
> 
> I have put the requirement for trimming in there (it was optional before,
> though recommended in USEAGE) because Frank asked for something stronger
> (and nobody has disputed that).

I recall no ticket item for the issue, no discussion, and no declaration
of consensus.  References?

> I put that total length limit in there because (as we have both witnessed)
> some existing agents do some most unpleasant things when References
> headers get too long.

There is no such thing as "too long" for a field.  What was observed was
the result of inappropriate folding at 990 octets to stay within the
per-line limit (which does exist).  Broken software is broken software,
and we should not impose limits which will be ineffective (a single-line
References field of 995 octets would still be broken inappropriately at
990 octets by the software in question), which are counter to agreed
upon consensus (at least without reopening discussion first), and which
run counter to established practice per RFCs 822, 850, 1036, and 2822.

> Nobody has queried what I wrote so far, but if you 
> wish it to be discussed, then please ask Alexey to raise a USEPRO ticket
> for it, and we will do so.

The procedure should be discuss first, and change the document (which is
supposed to reflect WG consensus) only after consensus is declared by
the Chair.  Since you are the one who apparently wishes to make a change,
you should request a discussion ticket.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Date: Thu, 23 Jun 2005 08:58:44 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 22 2005 14:50, Charles Lindsey wrote:

> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

> >1. stay as close to 2822+MIME as possible.
> >   In this case, there's no need for a deviation; 2822 requires
> >   accepting "UT" and the standard 3-letter abbreviations.
> 
> We have agreed that all the 3-letter abbreviations are a Bad Thing and
> should disappear from active use ASAP.

I do not recall any such agreement on such a wide-sweeping statement.
Can you supply a reference to support your assertion?

> To this end, we forbid their 
> generation within News, we do NOT REQUIRE acceptance except for GMT, but
> you MAY accept them if you wish (as with all obs-syntax).

A UA which does not handle the standard "UT" and 3-letter abbreviations
is going to be unable to handle Date fields in archived articles; such
abbreviations were legal under RFCs 850 and 1036, and as recently noted
continue to be in use.

Now if you want to re-introduce separate syntax for Injection-Date (note
that we had that discussion years ago and decided to use date-time
*because* parsers for it already exist in news software), (to use your
own words) "then please ask Alexey to raise a USEPRO ticket for it".
 
> >3. backwards compatibility (see our charter).  Differences cause problems
> >   not only for UAs and gateways, but for messaging protocols (e.g. IMAP).
> 
> I see nothing in the proposal that will cause problems for other such
> agents.

The Date field is used in all applications using the Internet Message
Format (mail, news, voice messaging, EDI, fax) and date-time is used in
a number of fields other than Date.  Changing the definitions of "zone"
and "orig-date" causes problems for combined news/mail UAs as well as
protocols that handle messages largely independently of specific
applications.

Again, if you want to make a special case for Injection-Date which is
proposed for news use only, that's a separate matter.
 
> >4. RFC 2822, MIME, IMAP, etc. are outside of our bailiwick; we have no
> >   authority to change them.
> >   orig-date and zone are part of 2822.
> 
> >> Software that accepts dates with unknown timezones SHOULD treat such
> >> timezones as equivalent to "+0000".
> 
> >Aside from the issue of - rather than +, the more important issue is
> >that acceptance of broken date-time (illegal or missing zones) violates
> >all of the principles listed above.
> 
> True if you intend to propagate it further, but not if you are just trying
> to decide whether to reject an article as "stale".

The author's or authors' origination date has nothing to do with "staleness",
which in any event is a protocol issue, not a syntax issue.
 
> So if some particular injecting agent decides that "EST" is "unknown",

Clarification is needed (unless we simply use RFC 2822 by reference). "EST"
has always been a standard "known" zone abbreviation with a specific
equivalent offset (-0500).

> then it either swallows its pride and passes it on, or else it physically
> replaces it by -0000 (I could write that into USEPRO if people wish).

This person certainly does not wish so.  I believe that we have a principle
that message content should not be modified (except of course for Path and
similar trace fields).  Feel free to suggest a new ticket, but kindly do
not make arbitrary changes w/o WG discussion and Chair-declared consensus.

Date: Thu, 23 Jun 2005 13:17:03 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Henry Spencer <[email-masked]>, Usefor Mailing List <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Sender: [email-masked]

--On tirsdag, juni 21, 2005 13:53:08 -0400 Henry Spencer 
<[email-masked]> wrote:

>> Software that accepts dates with unknown timezones SHOULD treat such
>> timezones as equivalent to "+0000".
>
> I go along with the earlier suggestion that this should be "-0000", which
> is RFC2822ese for "treat as GMT but actual timezone uncertain".  In reader
> agents, in particular, the distinction might be useful.

In the interest of civil list behaviour, I'll just say that my opinion on 
the usefulness of -0000 hasn't changed since this was proposed by Dan 
Bernstein in DRUMS.

Note that the following times:

12:00:00 +0000
12:00:00 -0000
08:00:00 -0400

are all equal according to RFC 2822. -0000 means "I'm not telling you my 
local timezone", not "I'm uncertain about the time".
The text proposed says that the following timezone:

12:00:00 NONSENSE

can also be considered equal to the above, if one chooses to accept it at 
all.

                           Harald

Date: Wed, 22 Jun 2005 22:13:25 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: John Stanley <[email-masked]>, [email-masked]
Subject: Re: Non-standard language justification (was Re: #1008 USEFOR 1.5 References: Final text on "followup")
Sender: [email-masked]

--On 21. juni 2005 17:26 -0700 John Stanley <[email-masked]> wrote:

> And since the point was skipped in the last response, I'll make it again.
> Is there a justification for the IETF-USEFOR working group web page not to
> have the latest complete draft? I know, I've asked for up to date
> DISCUSSION versions to be kept on the web page so we can all discuss the
> same thing and Charles nixed that because it was too hard and would take
> updating the web every five minutes, but now we don't even keep an up to
> date draft as sent to the IETF from three weeks ago?

The chairs will attempt to get this fixed.
Version numbers are cheap (outside of the IETF pre-meeting blackouts), so 
we might as well just ship official versions (my opinion). I hope we have 
enough issues settled to make it worthwhile to ask our editor to emit 
usefor-05 before the Paris blackout.

                    Harlad

Date: Wed, 22 Jun 2005 22:08:55 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Sender: [email-masked]

Bruce,

--On 21. juni 2005 22:20 -0400 Bruce Lilly <[email-masked]> wrote:

> Such attempts violate several principles which we've agreed on:
> 1. stay as close to 2822+MIME as possible.
>    In this case, there's no need for a deviation; 2822 requires
>    accepting "UT" and the standard 3-letter abbreviations.

Section 2.1 of usefor-04 is very clear:

   News articles MUST conform to the syntax specified in Section 3 of
   [RFC2822].  Netnews agents MAY also accept the obsolete syntax
   specified in Section 4 of [RFC2822], but they MUST NOT generate
   productions of such syntax.

This is a deviation from RFC 2822, which says "MUST accept the obsolete 
syntax". Since this apparently had a previous consensus of the WG, I don't 
see that you can say that we have agreement on the principle.

> 2. where we have good cause to deviate, the article format is more
>    restrictive.
>    Allowing generation of something which is prohibited for generation
>    by 2822 is clearly not more restrictive

Makes sense to me; others may have other opinions.
Are you suggesting (specific text change) that the doc says that systems 
MUST NOT generate GMT?

> 3. backwards compatibility (see our charter).  Differences cause problems
>    not only for UAs and gateways, but for messaging protocols (e.g. IMAP).

Are you referring to this charter text?

>The Goal of this working group is to publish a standards-track
>successor to RFC 1036 that with particular attention to backward
>compatibility, formalizes best current practice and best proposed practice.

> 4. RFC 2822, MIME, IMAP, etc. are outside of our bailiwick; we have no
>    authority to change them.
>    orig-date and zone are part of 2822.

Are you proposing that we delete the ABNF description?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: NEW: USEFOR 5. MIME boundary security considerations
Date:  Thu, 23 Jun 2005 02:33:42 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 46
Sender: [email-masked]

Bruce Lilly wrote:
 
> While I see no problem with addressing the issue per se, nor
> with the specific examples, it seems that the gist of the
> proposed wording is "avoid MIME"

It's more like "be careful with insane boundary constructs".

Near the top of the USEFOR memo (2.3) there's already a "MUST":

| User agents MUST meet the definition of MIME-conformance in
| [RFC2049] and MUST also support [RFC2231].

In the "security considerations" (5.) we only need to address
the most critical consequences of this statement.

> If you feel strongly that it should be posed as a security
> consideration

Yes, obviously, because 2046 doesn't discuss comments in the
boundary, let alone any 2231 constructs.

> I suggest "A naive rather than fully compliant implementation
> may make such a naive implementation vulnerable to security
> breaches.

One "naive" too many for my taste, and I very much doubt that
2231 constructs in a boundary are what the authors intended -
in fact they say "don't do this if it's unnecesary" as general
rule.  For this reason it's also better to avoid a statement
about "fully compliant".  Maybe a 2046bis or 2231bis clearly
disallows such boundary-constructs, good riddance.

> Specifications have been analyzed for security implications
> as documented in the relevant RFCs

Both 2046 and 2231 fail miserably for the boundary parameter,
that's why we have to mention it explicitly at all.

> deviations from those specifications may defeat security
> measures built into the specifications, and the resulting
> vulnerabilities might not be obvious to a non-expert

Yes, that's the point.  It's also not obvious for about every
UA claiming to be MIME-compatible, or is it ?  Bye, Frank

Date: Wed, 22 Jun 2005 17:23:48 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>If the sites "dead" and
>"beef" find that some idiot has invented a site "dead:beef",

I'm curious. Is this "idiot" an idiot because they are doing something 
explicitely allowed in the USEFOR standard, or for some other unspecified 
reason?

I mean, if:

   path-identity   =  ( ALPHA / DIGIT )
                      *( ALPHA / DIGIT / "-" / "." / ":" / "_" )
                                                                                
and someone actually follows that rule to create a path-identify for his
site that causes problems for existing sites, why is HE the idiot?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Definition of "private" (Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts)
Date:  Thu, 23 Jun 2005 02:02:19 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

Charles Lindsey wrote:

> <newsgroup-name>s of the following forms SHOULD NOT appear
> in Newsgroups and Followup-To headers in articles intended
> for propagation throughout the public Usenet.
[...]
> any <newsgroup-name> having only one <component>

As stated I disagree.  Except from some random junk like "fi"
or "free" there are apparently valid NGs "alt" and "general"
on news.clara.net (checked today, only for non-empty groups).

                   Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date:  Thu, 23 Jun 2005 00:43:43 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 65
Sender: [email-masked]

Bruce Lilly wrote:

> A dynamic IP address might be usable if the LHS has a
> timestamp as suggested in RFC 2822.

Please note the *if* here, timestamps are not the only
way to create some form of <id-local> - especially not
for devices so desperate that they use a domain literal
as <id-domain> instead of a proper FQDN.

While it's a bad idea some algorithms might try hashes
and / or counters.  Even if they use timestamps the way
how they encode them in a <id-local> varies.

Different local times on different machines are also a
source for insufficient <id-local> concepts.  All these
unclear conditions covered by a very simple STD 11 rule:

| THE USE OF DOMAIN-LITERALS IS STRONGLY DISCOURAGED.

> A static, Internet-routable IP address literal is OK
> for a msg-id RHS.

Yes.

> A note pointing out the limitations of dynamic IP
> addresses and the unsuitability of unroutable, example,
> loopback, DHCP negotiation, and multicast IP addresses
> should suffice.

E.g. the STD 11 rule translated to 2119 SHOULD NOT style.

> nor a protocol issue per se; it could go in USEAGE.

USEAGE is not on the same level as STD 11, and IMHO it's a
protocol issue in many cases.

As far as USEFOR and msg-id are concerned there's already
a note in chapter 5 (security considerations).

When it comes to domain literals my mental picture is an
independent Web cam without FQDN and clock connected to
the public Internet.  Maybe it tries to post pictures.

Yes, ridiculous, but the HTTP/1.1 spec. manages to allow
for such "Web server without clock" cases - IIRC it says
SHOULD instead of MUST at a critical place about "Date:".

 [path]
> the text currently does not mention IP address literals

Somewhere you'd find an "approved" IPv6 path-example for
the next USEPRO.

>> replace the bogus 2822 notation <id-right> by <id-domain>

> Here we disagree, as I have noted separately.

You probably want "my" semantics with the old 2822-syntax.
AFAIK this "my" is also "your" or the "true" semantics of
a msg-id.  I'd be very surprised if you say that the RHS
can be "anything" as long as it's syntactically correct.

                          Bye, Frank

Date: Wed, 22 Jun 2005 18:26:25 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Sender: [email-masked]

"Charles Lindsey" <[email-masked]> wrote:
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
>>On Tue June 21 2005 13:10, Ken Murchison wrote:

>>> Software that accepts dates with unknown timezones SHOULD treat such
>>> timezones as equivalent to "+0000".
>
>>Aside from the issue of - rather than +, the more important issue is
>>that acceptance of broken date-time (illegal or missing zones) violates
>>all of the principles listed above.
>
> True if you intend to propagate it further, but not if you are just trying
> to decide whether to reject an article as "stale".
>
> If some particular relaying agent decides that "EST" is "unknown", (and
> does not reject it as stale), then it either swallows its pride and passes
> it on, or else it drops it on the floor.
>
> So here is a suggested replacement:
>
> Software MAY interpret an unknown timezone as "-0000" for the purpose of
> comparing one <date-time> with another.

How else is it allowed to treat it?  If it's not going to just drop it
on the floor for having an unknown zone, it's supposed to do a
comparison.  Are we specifying that software is allowed to treat EST
as +4600 if it wants?

Seth

To: [email-masked]
Xref: clerew local.usefor:21430
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Wed, 22 Jun 2005 21:26:52 GMT
Lines: 102
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Fri June 17 2005 14:50, Frank Ellermann wrote:

>> Charles Lindsey wrote:
>> 
>> > Hold on! We never agreed that domain literals were a Bad
>> > Thing to use in an <id-right>.

>Frank has a valid point; 127.0.0.1, 192.168.0.1, 10.0.0.1,
>172.16.0.1, 192.0.0.1, 224.0.0.1, 169.254.0.1, and many similar
>IP address literals lack the uniqueness characteristic necessary
>for some uses (including msg-id domain lierals).

Indeed, there are lots of stupid domain literals that should not be used
as an <id-right>, but RFC 2822 already has that problem (and it is not our
job to fix RFC 2822). I would have thought the point obvious enough that
we do not need to mention it explicitly.

>A dynamic IP address might be usable if the LHS has a timestamp
>as suggested in RFC 2822.

Exactly. In fact, and IP address over which you have sole control at that
exact moment in time is OK (or you have some agreement with the person who
_does_ have such sole control).

>....  This isn't a syntax or
>semantics issue, nor a protocol issue per se; it could go in USEAGE.

Possibly.

>------ new topic -------
> 
>> It's bad in the path-identifier where it breaks Bruce's B-News
>> monster in the case of IPv6 colons.

>It breaks C news and INN as well.  Moreover, as recently reaffirmed
>by Russ, INN has no special treatment of numeric-plus-dots names in
>the Path field due to severe performance issues associated with
>such special treatment.

I do not recall that there was any performance issue. They are just a
string of characters like any other <path-identity>.

>....  Russ has noted that DNS lookups can take quite
>a long time.  The implications for path field components is clear:
>they have to be treated purely as public names, not as IP address
>literals.

Absolutely. But I don't expect DNS lookups for <path-identity>s on a
routine basis (indeed, our text used to say that, but it was one of Brad
Templeton's excesses that I got rid ot).

Though when it comes to implementing our new arrangements for catching
MISMATCHes, there will be a little extra work to do - probably involving
a cache of known path-identities for each peering site.

>  That obviates any argument about backwards compatibility
>breaking reassignment of colon (ostensibly for IPv6 literals), and
>it means that I don;t have to remind Charles that his claim that
>handling of e.g. dead:beef would affect only one site and not sites
>named "dead" or "beef" or their neighbors and downstream sites has
>been twice debunked before he recently repeated it.

I think it has always been well understood that the benefit of being able
to handle IPv6 addresses outweighed some possible inconvenience to sites
that used some rather unusual <path-identity>s. If the sites "dead" and
"beef" find that some idiot has invented a site "dead:beef", and if they
find that they are receiving too many articles twice over in consequence,
then they will have to deal with it as best they can. It is already
well-known that there are idiot sites out there, and Usenet has always
managed to work its way around them.

>Specific change to USEFOR -04 text: remove colon from path-identity
>ABNF production.

So you would rather throw out a much needed facility than put up with a
rather improbable minor inconvenience? Please come and join us in the Real
World for a change.

>The second note claims that component names "are traditionally in
>lower case", but I've seen mixed case and upper-case names.  I'd
>like to see that text elided. possibly replaced with a statement
>that a choice of name of "POSTED", "MISMATCH", or even "PoStEd"
>would be a really bad idea.  That of course assumes that we want
>to continue pursuing introduction of those keywords.

Again, there is a limit as to how far one should go to deal with idiotic
sysadmins.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21426
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Wed, 22 Jun 2005 19:18:24 GMT
Lines: 87
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Tue June 14 2005 09:05, Frank Ellermann wrote:
>> 
>> There's a dispute about the meaning of <id-right>.  I say it's
>> in fact an <id-domain>, a proper name space with an owner.
>> 
>> TTBOMK Charles says, whatever it is, in a msg-id it's case
>> sensitive.

>RFC 2822 is crystal clear on the semantics and the nature of the
>syntax differences from 822 and 2822 angle-addr:

>   Since the msg-id has
>   a similar syntax to angle-addr (identical except that comments and
>   folding white space are not allowed), a good method is to put the
>   domain name (or a domain literal IP address) of the host on which the
>   message identifier was created on the right hand side of the "@", and
>   put a combination of the current absolute date and time along with
>   some other currently unique (perhaps sequential) identifier available
>   on the system (for example, a process id number) on the left hand
>   side.

In RFC 2822 an <id-right> is a string of characters satisfying some ABNF.
It has no semantics attached to it (except that a `\` in a <quoted-pair>
is "invisible"). If you generate such an <id-right> by "putting" a <domain>
in there, it is still an <id-right> (even if it happens to look like a
<domain>) and it still has no further semantics, and hence no requirement
to be considered case insensitively at any stage.

In our draft, the same is true (except that we have fixed the
<quoted-pair> problem). But, for the removal of all doubt, we have also
excluded your bizarre interpretation by stating the intent that an
octet-by-octet comparison will always suffice.

>In RFC 822, the syntax was in fact identical; because of input specifically
>related to Usenet the syntax was changed to exclude CFWS.   The semantics
>hasn't changed,

Yes, it has changed (whether intended so or not).

>Given the stated methodology and the rationale for it (especially
>"otherwise result in subtle differences and interoperability challenges"),
>as well as the basis of our document supposed to be 2822, and the
>suggestion made years ago by Mark Crispin, viz:

>  NNTP Transport Requirements

>  � �The BLURDYBLOOP command in NNTP uses the message-id as an argument.
>�   �The syntax for this command does not provide any way for spaces to
>�   �appear in the message-id. �Therefore, message-ids in the Message-ID,
>�   �In-Reply-To, and References headers of a message MUST NOT contain
>�   �the space character.

>I can see no reason for an incompatible redefinition in our document.

The differences in the <msg-id> in our document are essential for
technical reasons that have been explained so many times that I will not
repeat them here.

>The sole purpose of the incompatible redefinition of msg-id, already
>defined in RFC 2822 making concessions to specific Usenet issues,
>seems to be to:

Those concessions turned out to be insufficient for the essential
requirement of Netnews that an octet-by-octet comparison of <msg-id>s
would always suffice.

>My recommendation is to leave msg-id as defined in our primary normative
>reference RFC 2822 and to add such notes along the lines suggested by
>Mark as may be required,

No, Mark made his suggestion (in the NNTP list, I believe) in the belief
that he was describing a temporary problem that would be fixed in future
implementations. He was rapidly disabused of that notion.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21429
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: OT: 78
Date: Wed, 22 Jun 2005 21:01:26 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> What is this tool that hits you, and exactly what is the
>> nature of the breakage that you see?

>It's a "norton commander clone" with among many other features
>a built-in text viewer.  That beast folds lines if they reach
>column 79.  I have many "magic" 78 and 79 in my REXX scripts.

And it does that even if the window is wide enough to accomodate 80
characters? Weird!

>> the reasons I have seen in attempting to justify a 78 or
>> 79 limit have nothing to do with the number of characters
>> needed to represent an EOL

>Is there any other (and preferably less ridiculous) reason...

Yes, lots of viewers do what your norton commander clone does, but not
until they reach column 80.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21424
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Date: Wed, 22 Jun 2005 18:50:28 GMT
Lines: 86
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Tue June 21 2005 13:10, Ken Murchison wrote:

>> The Date header is the same as that specified in Sections 3.3 and
>> 3.6.1 of [RFC2822] with the added restrictions detailed in
>> Section 2.2. However, the use of "GMT" as a time zone (part of <obs-
>> zone>), although deprecated, is widespread in news articles today.
>> Therefore, agents MUST accept <date-time> constructs which include
>> the updated <zone> construct below.

>The "MUST accept" is no change from 2822.

>> orig-date = "Date:" SP date-time CRLF
>> 
>> zone = (( "+" / "-" ) 4DIGIT) / "GMT"
>> 
>> Note that agents SHOULD NOT generate <date-time> constructs which
>> include "GMT" and MUST NOT generate <date-time> constructs which
>> include any other zone names defined by <obs-zone>.

>I really wish people would stop trying to impose such things on ABNF
>grammar which is defined in documents outside of the scope of this WG.
>Such attempts violate several principles which we've agreed on:
>1. stay as close to 2822+MIME as possible.
>   In this case, there's no need for a deviation; 2822 requires
>   accepting "UT" and the standard 3-letter abbreviations.

We have agreed that all the 3-letter abbreviations are a Bad Thing and
should disappear from active use ASAP. To this end, we forbid their
generation within News, we do NOT REQUIRE acceptance except for GMT, but
you MAY accept them if you wish (as with all obs-syntax).

>2. where we have good cause to deviate, the article format is more
>   restrictive.
>   Allowing generation of something which is prohibited for generation
>   by 2822 is clearly not more restrictive

I agree. s/SHOULD NOT generate/MUST NOT generate/ in Ken's text (and
simplify wording further).

>3. backwards compatibility (see our charter).  Differences cause problems
>   not only for UAs and gateways, but for messaging protocols (e.g. IMAP).

I see nothing in the proposal that will cause problems for other such
agents.

>4. RFC 2822, MIME, IMAP, etc. are outside of our bailiwick; we have no
>   authority to change them.
>   orig-date and zone are part of 2822.

>> Software that accepts dates with unknown timezones SHOULD treat such
>> timezones as equivalent to "+0000".

>Aside from the issue of - rather than +, the more important issue is
>that acceptance of broken date-time (illegal or missing zones) violates
>all of the principles listed above.

True if you intend to propagate it further, but not if you are just trying
to decide whether to reject an article as "stale".

So if some particular injecting agent decides that "EST" is "unknown",
then it either swallows its pride and passes it on, or else it physically
replaces it by -0000 (I could write that into USEPRO if people wish).

If some particular relaying agent decides that "EST" is "unknown", (and
does not reject it as stale), then it either swallows its pride and passes
it on, or else it drops it on the floor.

Likewise a serving agent either stores it or doesn't.

So here is a suggested replacement:

Software MAY interpret an unknown timezone as "-0000" for the purpose of
comparing one <date-time> with another.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21428
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Definition of "private" (Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts)
Date: Wed, 22 Jun 2005 20:58:06 GMT
Lines: 113
Sender: [email-masked]

In <78BBACC38DC374725A4EDD07@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>--On 20. juni 2005 10:23 +0000 Charles Lindsey <[email-masked]> wrote:

>> In <[email-masked]> Russ Allbery <[email-masked]>
>> writes:
>>
>>> You keep using this word "private."  I do not think this word means what
>>> you think it means.
>>
>> I think it was Harald's term. If there is doubt about its meaning, then a
>> definition would be needed. There is a clear enough definition of
>> "cooperating subnet" in son-of-1036 and in our earlier drafts.

>RFC 2434, guidelines for writing IANA sections:

>      Private Use - For private or local use only, with the type and
>           purpose defined by the local site. No attempt is made to
>           prevent multiple sites from using the same value in different
>           (and incompatible) ways. There is no need for IANA to review
>           such assignments and assignments are not generally useful for
>           interoperability.

Hmmm! I think that sort of definition is too restrictive for the kind of
useage we envisage, where mutual agreement between several sites may be
involved.

Indeed, I think "mutual agreement" to do something different out of the
ordinary is the term that we want.

Anyway, here is a proposed text, based on what Harald originally wrote,
and some of the comments that have been made since:

Let the ABNF be "rather liberal". In fact use the existing ABNF.
Let the text explain the various restrictions.

And restrict them like this:
 
  A newsgroup <component> SHOULD NOT consist of digits only (because of
  bad interactions with a widely deployed storage technique), and SHOULD
  NOT contain uppercase letters (because some existing agents treat them
  case insensitievly). Such <component>s MAY be used to refer to existing
  groups created prior to this standard.

   <component>s beginning with underline ("_") are reserved for use by future
   versions of this standard and MUST NOT be generated by posting agents, nor
   accepted by injecting agents.  However, such names MUST be accepted by
   other agents.

   <newsgroup-name>s of the following forms MUST NOT be used:
   o any <newsgroup-name> containing the <component> "ctl" (because it
     formerly caused such articles to be interpreted as control
     messages);
   o any <newsgroup-name> containing the <component> "all" (because it
     is used as a wildcard in some implementations);
   o any <newsgroup-name> starting with the <component> "to" (because of
     its special meaning to route certain control messages on a
     point-to-point basis [USEPRO], for which specific purpose it MAY
     continue to be used);
   o the <newsgroup-name> "poster" (because of its special meaning in the
     Followup-To header).

   <newsgroup-name>s of the following forms SHOULD NOT appear in
   Newsgroups and Followup-To headers in articles intended for
   propagation throughout the public Usenet. They MAY appear in Xref
   headers and they MAY appear in articles passed between agents where
   there is some mutual agreement to do so.
   o any <newsgroup-name> having only one <component>;
   o any <newsgroup-name> starting with the <component> "control"
     (used by many serving agents for pseudo-newgroups to hold control
     messages);
   o the <newsgroup-name> "junk" (used by many serving agents to store
     invalid articles);
   o any <newsgroup-name> starting with the <component> "example"
    (reserved for examples in this and other standards);
   o any <newsgroup-name> containing <component>s beginning with "+" or
     "-" (reserved for special-purpose newsgroups, e.g. read-only
     groups, not to be propagated except by mutual agreement).

Now I can see several objections to this, so there is still much to
discuss.

1. I have followed the argument, put forward by several people, that we
should have a "one-stop-shop" for all rules restricting newsgroup-names;
hence this all goes in USEFOR.

2. Russ will take issue with my inclusion of "To" alongside "ctl" and
"all".

3. Russ will also take issue with the whole of that last paragraph. So
discussion of exact wording is in order (e.g. is "SHOULD NOT" the right
thing to say).

4. There is room for discussion of whether the various categories are in
the right list, and whether they are described correctly.

5. And when all that is settled. there will need to be some wording in
USEPRO as regards what MUST/SHOULD not appear in control messages and what
MUST/SHOULD be enforced by injecting agents.
 
-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21427
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Wed, 22 Jun 2005 19:25:56 GMT
Lines: 43
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Thu June 16 2005 07:48, Charles Lindsey wrote:

>>    If the resulting References header would, after unfolding, exceed 998
>>    characters in length (including its header name but not the final
>>    CRLF), it MUST be trimmed (and otherwise it MAY be trimmed). Trimming
>>    involves removing any number of message identifiers from its content,
>>    except that the first message identifier and the last two MUST NOT be
>>    removed.

>The idea of per-field length limits (which did not exist in 822, 850,
>1036 and do not exist in 2822) was discussed near the beginning of this
>WG.  For example:

>The result of which was to affirm no such limits, and we moved on from
>there.  Now you're trying to introduce a per-field limit. Why -- what
>has changed recently to justify reintroducing something that was long
>ago rejected?

It is indeed rather unusual to impose any limit (other than an advisory
one) on the overall length of a header.

I have put the requirement for trimming in there (it was optional before,
though recommended in USEAGE) because Frank asked for something stronger
(and nobody has disputed that).

I put that total length limit in there because (as we have both witnessed)
some existing agents do some most unpleasant things when References
headers get too long. Nobody has queried what I wrote so far, but if you
wish it to be discussed, then please ask Alexey to raise a USEPRO ticket
for it, and we will do so.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21425
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Wed, 22 Jun 2005 18:54:47 GMT
Lines: 48
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> Why? If it's good enough for RFC 2822, it's good enough
>> for us.

>RfC 2822 missed the idea of Message-IDs, a globally unique
>identifier forever.  Where collisions and abusing foreign
>name spaces may have all sorts of undesirable side effects.

>It's already clear that we cannot use the old 2822 msg-id
>syntax.  For NetNews our starting point is RfC 1036 anyway,
>that's based on 822, and 822 said:

>| THE USE OF DOMAIN-LITERALS IS STRONGLY DISCOURAGED.

>Everywhere to the right of an "@" or in path identities.

>> Feel free to trawl the archives.

><http://mid.gmane.org/[email-masked]>
><http://mid.gmane.org/[email-masked]>
><http://mid.gmane.org/[email-masked]>

I see nothing in those messages to indicate that anybody but yourself had
accepted the idea that domain-literals (assuming they actually looked like
an Internet address) were to be discouraged.

>> All current UAs should (and AIUI do) send mail correctly
>> to addresses using domain literals.

>Not here. my UA sends mails to its configured smart hosts,
>and these smart hosts reject all mails to domain literals:
>"administrative restriction".

Then those smart hosts are broken.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Cc: [email-masked]
Subject: Re: Admin: Please watch your language
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 21 Jun 2005 19:46:28 -0700
Sender: [email-masked]

Well, as someone who used the utter nonsense phrasing recently, I
apologize to Charles and the other group participants for using language
that really wasn't constructive and was purely a voicing of frustration on
my part.  Harald's right -- it doesn't help us arrive at a conclusion.

I'll try to maintain a more even temper.  The temptation is to raise the
level of rhetoric when one feels like one isn't being listened to, in an
effort to make more impact, but that really isn't the relevant problem at
the moment.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: NEW: USEFOR 5. MIME boundary security considerations
Date: Tue, 21 Jun 2005 22:35:26 -0400
Cc: [email-masked]
Sender: [email-masked]

On Tue June 21 2005 13:45, Frank Ellermann wrote:

> Last proposed "security considerations" addition was in
> <http://mid.gmane.org/[email-masked]>
> 
>  MIME security considerations are discussed in [RFC2046].  Note that
>  applying some [RFC2231] extensions for parameters like multi-line
>  paramters on a boundary parameter as defined in [RFC2046] might be
>  abused to bypass naive methods of handling parameters.  Two examples:
> 
>  Content-Type:
>   multipart/mixed (; boundary=foo ; xyz=");bOuNdArY*=''next%20part(")
>  Content-Type: multipart/digest;
>   boundary (not=me) = ("yes ;-) simple (foo;bar") ; x-foo = xyzzy

While I see no problem with addressing the issue per se, nor with
the specific examples, it seems that the gist of the proposed wording
is "avoid MIME" (2231 is part of the MIME specification).  I would
much prefer something similar to "it requires complete understanding
and implementation of the normative references to be compliant".  Of
course, the USEFOR draft -04 already says that (section 1.6).  If you
feel strongly that it should be posed as a security consideration, then
I suggest "A naive rather than fully compliant implementation may
make such a naive implementation vulnerable to security breaches.
Specifications have been analyzed for security implications as
documented in the relevant RFCs; deviations from those specifications
may defeat security measures built into the specifications, and the
resulting vulnerabilities might not be obvious to a non-expert".

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Ken Murchison <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Date: Tue, 21 Jun 2005 22:20:33 -0400
Cc: [email-masked]
Sender: [email-masked]

On Tue June 21 2005 13:10, Ken Murchison wrote:

> The Date header is the same as that specified in Sections 3.3 and
> 3.6.1 of [RFC2822] with the added restrictions detailed in
> Section 2.2. However, the use of "GMT" as a time zone (part of <obs-
> zone>), although deprecated, is widespread in news articles today.
> Therefore, agents MUST accept <date-time> constructs which include
> the updated <zone> construct below.

The "MUST accept" is no change from 2822.

> orig-date = "Date:" SP date-time CRLF
> 
> zone = (( "+" / "-" ) 4DIGIT) / "GMT"
> 
> Note that agents SHOULD NOT generate <date-time> constructs which
> include "GMT" and MUST NOT generate <date-time> constructs which
> include any other zone names defined by <obs-zone>.

I really wish people would stop trying to impose such things on ABNF
grammar which is defined in documents outside of the scope of this WG.
Such attempts violate several principles which we've agreed on:
1. stay as close to 2822+MIME as possible.
   In this case, there's no need for a deviation; 2822 requires
   accepting "UT" and the standard 3-letter abbreviations.
2. where we have good cause to deviate, the article format is more
   restrictive.
   Allowing generation of something which is prohibited for generation
   by 2822 is clearly not more restrictive
3. backwards compatibility (see our charter).  Differences cause problems
   not only for UAs and gateways, but for messaging protocols (e.g. IMAP).
4. RFC 2822, MIME, IMAP, etc. are outside of our bailiwick; we have no
   authority to change them.
   orig-date and zone are part of 2822.

> Software that accepts dates with unknown timezones SHOULD treat such
> timezones as equivalent to "+0000".

Aside from the issue of - rather than +, the more important issue is
that acceptance of broken date-time (illegal or missing zones) violates
all of the principles listed above.  Standard zones (in archived messages
or generated by 1036/822-compliant legacy software) really should be
treated as the equivalent standard offsets (as per 822/2822).  As the
1-letter zones were long ago botched and effectively convey no
information (RFC 1123), it would be reasonable to treat those as
equivalent to -0000, and indeed that is what 2822 recommends.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: Admin: Please watch your language
Date: Tue, 21 Jun 2005 21:48:53 -0400
Cc: [email-masked], [email-masked]
Sender: [email-masked]

On Tue June 21 2005 19:47, Frank Ellermann wrote:

> Okay, I can confirm this.  BTW, it was about a
> somewhat dubious joke which could be understood
> as an attack against you. 

I can state definitively that nothing I have said, on-list
or off-list, would have precipitated that.  I understand your
little jokes, but I can see how a newcomer or somebody
skimming the archives might not.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: NEW: Path field delimiters and components; also #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Tue, 21 Jun 2005 21:33:08 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 17 2005 14:50, Frank Ellermann wrote:

> Charles Lindsey wrote:
> 
> > 2822 says more than that. A little further down:
> [...]
> >| some domain identifier
> 
> Yes, that's not good enough, it SHOULD be FQDN allowing domain
> literal only for special cases.
> 
> >> RfC 2822 also doesn't discourage domain literals generally
> >> like STD 11.  We have to fix it by a clear SHOULD NOT as in
> >> STD 11.
> 
> > Hold on! We never agreed that domain literals were a Bad
> > Thing to use in an <id-right>.

Frank has a valid point; 127.0.0.1, 192.168.0.1, 10.0.0.1,
172.16.0.1, 192.0.0.1, 224.0.0.1, 169.254.0.1, and many similar
IP address literals lack the uniqueness characteristic necessary
for some uses (including msg-id domain lierals).

A dynamic IP address might be usable if the LHS has a timestamp
as suggested in RFC 2822.

A static, Internet-routable IP address literal is OK for a msg-id
RHS.

A note pointing out the limitations of dynamic IP addresses and
the unsuitability of unroutable, example, loopback, DHCP negotiation,
and multicast IP addresses should suffice.  This isn't a syntax or
semantics issue, nor a protocol issue per se; it could go in USEAGE.

> That's not how I recall it, and a domain-literal is everywhere
> a Bad Thing:

------ new topic -------
 
> It's bad in the path-identifier where it breaks Bruce's B-News
> monster in the case of IPv6 colons.

It breaks C news and INN as well.  Moreover, as recently reaffirmed
by Russ, INN has no special treatment of numeric-plus-dots names in
the Path field due to severe performance issues associated with
such special treatment.  Note that the Path field as currently
defined has no provision for comments; the justification given for
that in earlier drafts was that the processing to treat comments
as whitespace (i.e. essentially to ignore them) would be an onerous
performance problem.  Russ has noted that DNS lookups can take quite
a long time.  The implications for path field components is clear:
they have to be treated purely as public names, not as IP address
literals.  That obviates any argument about backwards compatibility
breaking reassignment of colon (ostensibly for IPv6 literals), and
it means that I don;t have to remind Charles that his claim that
handling of e.g. dead:beef would affect only one site and not sites
named "dead" or "beef" or their neighbors and downstream sites has
been twice debunked before he recently repeated it.

Specific change to USEFOR -04 text: remove colon from path-identity
ABNF production.

As the text currently doe not mention IP address literals and does
use "name" (albeit only in a note), no other change is necessary
(semantic clarification that components are public names would
however be welcome).

The second note claims that component names "are traditionally in
lower case", but I've seen mixed case and upper-case names.  I'd
like to see that text elided. possibly replaced with a statement
that a choice of name of "POSTED", "MISMATCH", or even "PoStEd"
would be a really bad idea.  That of course assumes that we want
to continue pursuing introduction of those keywords.
 
---- returning to msg-id -----

> All these tons of text are so obvious and unnecessary if you'd
> only replace the bogus 2822 notation <id-right> by <id-domain>.

Here we disagree, as I have noted separately.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Tue, 21 Jun 2005 20:36:09 -0400
Cc: [email-masked]
Sender: [email-masked]

On Thu June 16 2005 07:48, Charles Lindsey wrote:

> No, I have just created a References header whose *semantic*content*
> contains all the right msg-ids. Who knows whether it is folded or not? Now
> I want to say that you had better check that it would be less than 998 if
> you were to unfold it (because some poorly designed implementation might
> try to unfold it and process it further in a 998 buffer).
[...]
>    If the resulting References header would, after unfolding, exceed 998
>    characters in length (including its header name but not the final
>    CRLF), it MUST be trimmed (and otherwise it MAY be trimmed). Trimming
>    involves removing any number of message identifiers from its content,
>    except that the first message identifier and the last two MUST NOT be
>    removed.

The idea of per-field length limits (which did not exist in 822, 850,
1036 and do not exist in 2822) was discussed near the beginning of this
WG.  For example:
-------------------------------
Date: Mon, 02 Feb 1998 11:50:48 -0800 (PST)
From: Chris Newman <[email-masked]>
Subject: Header/Message Limits
To: [email-masked]
Message-id: <[email-masked]>
MIME-version: 1.0
Content-type: TEXT/PLAIN;
  charset=US-ASCII
Originator-Info: login-id=chris; server=THOR.INNOSOFT.COM
Resent-Message-ID: <"PFEAz3.0.hd5.yCYrq"@ambient>
Resent-From: [email-masked]
X-Mailing-List: <[email-masked]> archive/latest/2643
X-Loop: [email-masked]
Precedence: list
Resent-Sender: [email-masked]
X-UID: 12518
X-Length: 2117

The current limit for messages in email is specified in RFC 1123, section
5.3.8.  I recommend reading it.

Usenet and email currently have no specified limits on headers, and they
seem to work just fine.  Why fix something which isn't broken?

User interfaces will do what is necessary to present headers to the user,
and if they have to truncate them, they will.

Servers which wish to process arbitrary-length headers efficiently can
simply mmap() the articles (or use a fixed buffer at the max message
size) and do in-place processing with minimal calls to allocation 
functions.

If you do add length limits, find the length limits in deployed software
and document them with the same sort of advice as in the above section.

But please don't add length limits which are smaller than those which
function in deployed software.

		- Chris
-------------------------

The result of which was to affirm no such limits, and we moved on from
there.  Now you're trying to introduce a per-field limit. Why -- what
has changed recently to justify reintroducing something that was long
ago rejected?

Date: Tue, 21 Jun 2005 17:26:30 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Non-standard language justification (was Re: #1008 USEFOR 1.5 References: Final text on "followup")
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>Yes. There was justification in the old article-* drafts, but it has sadly
>got lost in the rush to make USEFOR as brief (and unhelpful) as possible
>:-( .

In other words, USEFOR-04 does not document all of the differences and
contains no justification. (E.g., if "header" means something else in our
document than in RFC2822, what term do we use for the "header" as defined
in RFC2822?)

>I have seen lots of messages from abuse@... sites, but never one asking me
>to send a "full header". Lots of them asking me to send "full headers",
>though.

So, their non-conforming use of terminology in email is an excuse for us 
to propogate the use into news? I think not.

The solution isn't that hard. Let me time this ...

Oh my, a contradiction! 

   o  Compliant software MUST NOT generate (but MAY accept) headers of
      more than 998 octets. 

and:

         NOTE: There is NO restriction on the number of lines into which
         a header may be split, and hence there is NO restriction on the
         total length of a header (in particular it may, by suitable
         folding, be made to exceed the 998 octets restriction
         pertaining to a single header line).

We have a restriction on the total length of a header, and then say there
is no restriction on the total length of a header? (If one MUST NOT
generate something, one cannot then, "by suitable folding", generate said
thing.) And note that the note is patently incorrect when it refers to
"the 998 octets restriction pertaining to a single header line". The 998
octet restriction is on the "header", not the 'header line'. It is RFC2047
that contains a restriction on a "header line", and that is a measly 76
octets.

Subtracting the time spent dealing with the contradiction, a total of 
about 10 minutes converting all "header" to "header field" in the 
appropriate places. It would have been faster to apply sed and clean up 
the problem spots instead of using vi manually.

Not a very good justification for continuing to use different language. 
Was the contradiction harder to spot because we use non-standard language? 
I didn't really spot it until I tried to convert from our gobbly into 
standard gobbly and saw immediately that there was a problem. Perhaps 
there is no better reason to use standard language in a technical 
standard?

And since the point was skipped in the last response, I'll make it again.
Is there a justification for the IETF-USEFOR working group web page not to
have the latest complete draft? I know, I've asked for up to date
DISCUSSION versions to be kept on the web page so we can all discuss the
same thing and Charles nixed that because it was too hard and would take
updating the web every five minutes, but now we don't even keep an up to
date draft as sent to the IETF from three weeks ago?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  OT
Date:  Wed, 22 Jun 2005 02:09:53 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 6
Sender: [email-masked]

So far for Re:mail and "off list" :-(

| When the "Reply-To:" field is present, it indicates the mailbox(es)
| to which the author of the message suggests that replies be sent.

Date: Wed, 22 Jun 2005 01:47:27 +0200
From: Frank Ellermann <[email-masked]>
Organization: <URL:http://purl.net/xyzzy>
To: [email-masked]
CC: [email-masked]
Subject: [off list] Re: Admin: Please watch your language
Sender: [email-masked]

Bruce Lilly wrote:
 
> On list, so others can see what this is about

Off list, because it's really nobody's business.

> I of course have no idea what other private
> warnings might have been issued, and I can only
> hope that all participants are being held to
> the same standard.

Okay, I can confirm this.  BTW, it was about a
somewhat dubious joke which could be understood
as an attack against you.  Are we now ready with
discussing private mails ?
                           Bye, Frank

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: Admin: Please watch your language
Date: Tue, 21 Jun 2005 17:46:30 -0400
Cc: Alexey Melnikov <[email-masked]>, [email-masked]
Sender: [email-masked]

On Fri June 10 2005 09:09, Harald Tveit Alvestrand wrote:
> 
> Bruce,
> 
> this is a warning. On list and for the record. You were already warned 
> privately about insulting participants.

On list, so others can see what this is about, and for the record, the
previous warning was for use of the word "nonsense":

  Calling other people's statements "nonsense" is not going to help us get 
  these documents finished.

I have, before and since, seen others use the exact word, indeed the
phrase "utter nonsense" has been used by several WG participants on
several occasions, before and since.  I of course have no idea what
other private warnings might have been issued, and I can only hope
that all participants are being held to the same standard.

> Your language is insulting to the editors, the chairs and the members of 
> the working group who do not agree with you

I can assure you that there was no such intent; the text quoted below
(retained verbatim) was in direct response to your statement in response
to Forrest's remarks which you quoted.  It was intended specifically to
address the striking differences between the draft under discussion and
other IETF work products, by way of an analogy.

And I would add that I intentionally ignored certain remarks in the
message to which I was responding which:
1. could be construed as an ad-hominem attack
2. would be as true with a number of other names substituted
and
3. added nothing to the substantive matters under discussion.

That message is of course part of the WG archive:
http://www.imc.org/ietf-usefor/mail-archive/msg01052.html

> - as well as being just vague  
> enough that it is not possible to figure out what exact text you are 
> objecting to, and what you want it to say instead.

It was, as the context below clearly shows, a direct response to a
particular statement, not a specific reference to specific text.

> If you continue in this vein, I will remove your posting rights to this 
> list.
> 
>                       Harald
> 
> 
> --On fredag, juni 10, 2005 09:00:52 -0400 Bruce Lilly <[email-masked]> 
> wrote:
> 
> > On Fri June 10 2005 02:30, Harald Tveit Alvestrand wrote:
> >
> >> After 8 years on the IESG reading drafts, I found nothing special about
> >> draft-ietf-usefor-usefor-04 that would warrant the "don't resemble any
> >> other IETF work-product" claim. So it seems you have different criteria
> >> than mine.
> >
> > I can't recall seeing a draft other than the one under discussion and
> > its predecessors that has anything like
> > "When we say 'white' we mean black, and when we say 'black' we mean red,
> > and when we say 'red' we mean green.  We won't tell you what we really
> > mean when we say 'green' -- sometimes it might mean green and sometimes it
> > might mean something else."
> > Which is basically what that draft says (except it's not nearly as honest
> > about the last part).

Your response appeared within minutes of my message.  In more than a week
since, I have seen at least three messages other than this one expressing
surprise and wondering specifically what the perceived problem is:

http://www.imc.org/ietf-usefor/mail-archive/msg01059.html
http://www.imc.org/ietf-usefor/mail-archive/msg01075.html
http://www.imc.org/ietf-usefor/mail-archive/msg01077.html

That makes at least four of us who are unsure what the specific problem is.

More troubling is that the substantive matters in the remaining (unquoted)
part of the article appear, as far as I can tell, to remain unaddressed
more than a week later.  Specifically, those issues include:

1. Have criteria for determining consensus changed, and if so, what are
   the new criteria?  This issue had been raised earlier by other WG
   participants.

2. Issues related to ensuring that WG consensus, once determined, is
   accurately reflected in the documents, permanently (or at least until
   some new issue warrants the Chair reopening a discussion).

3. Ground rules for discussion of specific principles and/or text.

4. A very specific suggestion for a specific "actionable item" as defined
   in the earlier message to which it was a response.

5. One example of earlier WG discussion and seven distinct points in
   response to a specific request to make a case for use of standard
   terminology.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues, also #1030 backward compatibility and handling incompatibilities
Date: Tue, 21 Jun 2005 16:26:47 -0400
Cc: [email-masked]
Sender: [email-masked]

On Tue June 14 2005 09:05, Frank Ellermann wrote:
> 
> Harald Tveit Alvestrand wrote:
> 
> > I'm going to resolve the ticket as "just replace mqtext with
> > *mqtext".
> 
> > OK?
> 
> There's a dispute about the meaning of <id-right>.  I say it's
> in fact an <id-domain>, a proper name space with an owner.
> 
> TTBOMK Charles says, whatever it is, in a msg-id it's case
> sensitive.

RFC 2822 is crystal clear on the semantics and the nature of the
syntax differences from 822 and 2822 angle-addr:

   Since the msg-id has
   a similar syntax to angle-addr (identical except that comments and
   folding white space are not allowed), a good method is to put the
   domain name (or a domain literal IP address) of the host on which the
   message identifier was created on the right hand side of the "@", and
   put a combination of the current absolute date and time along with
   some other currently unique (perhaps sequential) identifier available
   on the system (for example, a process id number) on the left hand
   side.

In RFC 822, the syntax was in fact identical; because of input specifically
related to Usenet the syntax was changed to exclude CFWS.   The semantics
hasn't changed, id-right is still a domain name or domain literal, and
there is nothing in 2822 that even *suggests* case-sensitivity.

> Message-IDs are at the very core of Netnews.  More than three
> opinions about it for a clearer rough consensus might help.
> Henry and Russ refuse to discuss it.  Maybe somebody else has
> an opinion.
>                          Bye, Frank
> 
> msg-id       =  "<" id-local "@" id-domain ">"
> 
> id-local     = dot-atom-text / ( DQUOTE id-quote DQUOTE )
> id-quote     = ( "." [id-text] ) /
>                ( [id-text] "." ) /
>                ( [id-text] id-special [id-text] )
> 
> id-text      = 1*( atext / "." / id-special )
> id-special   = "(" / ")" / "," /       ; all specials, minus ">",
>                "[" / "]" / "@" /       ; minus DQUOTE, minus "\",
>                ":" / ";" / "<" /       ; minus single ".", plus:
>                ".." / "\\" / ( "\" DQUOTE )
> 
> id-domain    = dot-atom-text / ("[" id-literal "]")
> id-literal   = 1*( %d33-61 /           ; printable ASCII minus
>                    %d63-90 /           ; ">", "[", "\", "]"
>                    %d94-126 /          ; plus "\[", "\\, "\]"
>                    "\[" / "\\" / "\]" )

Usefor draft -04 section 1.6 states:

   This document uses a cite by reference methodology, rather than
   repeating the contents of other standards, which could otherwise
   result in subtle differences and interoperability challenges.
   Although this document is as a result rather short, it requires
   complete understanding and implementation of the normative references
   to be compliant.

Given the stated methodology and the rationale for it (especially
"otherwise result in subtle differences and interoperability challenges"),
as well as the basis of our document supposed to be 2822, and the
suggestion made years ago by Mark Crispin, viz:

  NNTP Transport Requirements

  � �The BLURDYBLOOP command in NNTP uses the message-id as an argument.
�   �The syntax for this command does not provide any way for spaces to
�   �appear in the message-id. �Therefore, message-ids in the Message-ID,
�   �In-Reply-To, and References headers of a message MUST NOT contain
�   �the space character.

I can see no reason for an incompatible redefinition in our document.

The sole purpose of the incompatible redefinition of msg-id, already
defined in RFC 2822 making concessions to specific Usenet issues,
seems to be to:
1. eliminate some legal, unchanged-since-822 local-part constructs that
   somebody doesn't like (but I've seen no description of real issues
   similar to those of Mark's suggestion) (N.B. the WS part of CFWS is
   already excluded by 2822).
2. force-fit the existing IP address literal restrictions into the
   syntax instead of simply saying "an IP address literal must conform
   to the syntax of STD 3 (IPv4) or RFC 3513 (IPv6) or any future IP
   address literal syntax that may be approved by the IESG".  Of
   course, that's already implicit in the semantic definition of RFC
   2822 domain-literal/no-fold-literal, so nothing need be said at all
   (avoiding the somewhat tricky issue of 3513 successors and future
   directions.
3. introduce incompatible use of 822/2822 "special"s in the LHS. That
   is not backwards compatible (it may break IMAP and other Standards
   Track protocols).  In particular (but by no mens the sole problem),
   introduction of unquoted parentheses in the LHS of a msg-id in
   structured fields (especially since comments were permitted by RFC
   822) is begging for interoperability problems.  Especially as the
   proposed syntax would permit *unbalanced* parentheses.  That's
   going to break all sorts of quick-and-dirty hacks, and we all
   know that there are plenty of those around.

My recommendation is to leave msg-id as defined in our primary normative
reference RFC 2822 and to add such notes along the lines suggested by
Mark as may be required, keeping those (in the format document) to the
bare minimum necessary to deal with specific external protocol issues
such as NNTP.  A personal dislike for some RFC 822/2822 feature wouldn't
qualify, nor would some specific broken UA that can't handle legal 822
syntax not explicitly forbidden by RFC 1036.  Length limits in some
ancient implementations might barely qualify for such a note (NNTP's
limit of ca. 500 octets would warrant a brief note, but doesn't encroach
on legal domain names), but said note should also observe that legal
domain names may be as long as 255 octets, and should strongly recommend
that software with such incompatible limits be upgraded ASAP.  Such
notes about ancient implementations might even be relegated to a
separate informational document.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Date:  Tue, 21 Jun 2005 20:05:42 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

Ken Murchison wrote:

> Software that accepts dates with unknown timezones SHOULD
> treat such timezones as equivalent to "+0000".

I'd say "-0000", the conventional "unknown whatever" notation.

If my guess what Harald meant was correct you've lost the MAY:

 Injecting agents MAY reject articles with unrecognized
 timezones.

But maybe that's not important, or I guessed wrong, bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  NEW: USEFOR 5. MIME boundary security considerations
Date:  Tue, 21 Jun 2005 19:45:55 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 20
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> If you want to raise a new issue, and want to make sure it
> gets a ticket, please use a subject line saying something
> like
> NEW: USEPRO 3.14.159 Section numbering is irrational

Last proposed "security considerations" addition was in
<http://mid.gmane.org/[email-masked]>

 MIME security considerations are discussed in [RFC2046].  Note that
 applying some [RFC2231] extensions for parameters like multi-line
 paramters on a boundary parameter as defined in [RFC2046] might be
 abused to bypass naive methods of handling parameters.  Two examples:

 Content-Type:
  multipart/mixed (; boundary=foo ; xyz=");bOuNdArY*=''next%20part(")
 Content-Type: multipart/digest;
  boundary (not=me) = ("yes ;-) simple (foo;bar") ; x-foo = xyzzy

Date: Tue, 21 Jun 2005 13:53:08 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Sender: [email-masked]

On Tue, 21 Jun 2005, Ken Murchison wrote:
> Based in the discussion of this ticket, here's what I currently have in
> the document...

Looks good, with one minor caveat...

> Software that accepts dates with unknown timezones SHOULD treat such
> timezones as equivalent to "+0000".

I go along with the earlier suggestion that this should be "-0000", which
is RFC2822ese for "treat as GMT but actual timezone uncertain".  In reader
agents, in particular, the distinction might be useful. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date:  Tue, 21 Jun 2005 19:31:08 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 58
Sender: [email-masked]

Charles Lindsey wrote:

>>> 2822 says more than that. A little further down:
>> [...]
>>>| some domain identifier

>> Yes, that's not good enough, it SHOULD be FQDN allowing
>> domain literal only for special cases.

> Why? If it's good enough for RFC 2822, it's good enough
> for us.

RfC 2822 missed the idea of Message-IDs, a globally unique
identifier forever.  Where collisions and abusing foreign
name spaces may have all sorts of undesirable side effects.

It's already clear that we cannot use the old 2822 msg-id
syntax.  For NetNews our starting point is RfC 1036 anyway,
that's based on 822, and 822 said:

| THE USE OF DOMAIN-LITERALS IS STRONGLY DISCOURAGED.

Everywhere to the right of an "@" or in path identities.

> Feel free to trawl the archives.

<http://mid.gmane.org/[email-masked]>
<http://mid.gmane.org/[email-masked]>
<http://mid.gmane.org/[email-masked]>

> All current UAs should (and AIUI do) send mail correctly
> to addresses using domain literals.

Not here. my UA sends mails to its configured smart hosts,
and these smart hosts reject all mails to domain literals:
"administrative restriction".

>> It's bad in an <id-domain> in the case of dynamic or
>> reserved IPs without a clear owner of this name space
[...]

> It won't cause havoc unless two sites have been allocated
> the same dynamic IP

No, that's wrong.  They only need a simple algorithm for
the LHS creating the <id-local> adding the same <id-domain>
like [127.0.0.1] (the worst case) or just whatever dynamic
IP they have.

If it's the same <id-local> and the same IP it could cause
harm.  Even after weeks, it's not only about simultaneously.

The only way to get a msg-id right is a proper name space,
and that's normally an FQDN in the <id-domain>.  A static
IP is also fine.  A dynamic IP is dangerous.  A private IP
might be FUBAR.  For IPv6 it depends, still a very bad idea
in the path or addresses, but maybe okay for a msg-id.  Bye

To: [email-masked]
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 21 Jun 2005 10:30:36 -0700
Sender: [email-masked]

Ken Murchison <[email-masked]> writes:

> The Date header is the same as that specified in Sections 3.3 and
> 3.6.1 of [RFC2822] with the added restrictions detailed in
> Section 2.2. However, the use of "GMT" as a time zone (part of <obs-
> zone>), although deprecated, is widespread in news articles today.
> Therefore, agents MUST accept <date-time> constructs which include
> the updated <zone> construct below.

> orig-date = "Date:" SP date-time CRLF

> zone = (( "+" / "-" ) 4DIGIT) / "GMT"

> Note that agents SHOULD NOT generate <date-time> constructs which
> include "GMT" and MUST NOT generate <date-time> constructs which
> include any other zone names defined by <obs-zone>.

> Software that accepts dates with unknown timezones SHOULD treat such
> timezones as equivalent to "+0000".

This looks great to me.

To: [email-masked]
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 21 Jun 2005 10:29:40 -0700
Sender: [email-masked]

Forrest J Cavalier <[email-masked]> writes:

> Charles offered no explanation for that change.

> Changing "unparsed" to "unparseable" is a complete change in meaning.
> "Unparseable" means that there is no possible way to parse.

> "Unparsed" seems correct to me, based on the "MAY accept other
> timezones" language.

I think he was copying a change I made in the wording.  "unparsed" doesn't
make any sense to me -- I'm not sure, reading the word, what it's supposed
to mean.

The idea, however it should be expressed, is that if you don't recognize
the time zone, you have to treat it as +0000 (or -0000, whatever).  Maybe
unrecognized would be a better word?

Date: Tue, 21 Jun 2005 13:10:09 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: [email-masked]
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Sender: [email-masked]

Since the ticket system doesn't appear to be able to CC to the list, I'm 
reposting here.

Based in the discussion of this ticket, here's what I currently have in
the document.  I removed the reference to "UT" and used Russ' text 
regarding unknown/unparsed zones:

The Date header is the same as that specified in Sections 3.3 and
3.6.1 of [RFC2822] with the added restrictions detailed in
Section 2.2. However, the use of "GMT" as a time zone (part of <obs-
zone>), although deprecated, is widespread in news articles today.
Therefore, agents MUST accept <date-time> constructs which include
the updated <zone> construct below.

orig-date = "Date:" SP date-time CRLF

zone = (( "+" / "-" ) 4DIGIT) / "GMT"

Note that agents SHOULD NOT generate <date-time> constructs which
include "GMT" and MUST NOT generate <date-time> constructs which
include any other zone names defined by <obs-zone>.

Software that accepts dates with unknown timezones SHOULD treat such
timezones as equivalent to "+0000".

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Tue, 21 Jun 2005 12:58:10 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Sender: [email-masked]

Frank Ellermann wrote:
> Yes.  Somebody fixed the "treat unparsed timezones as "GMT",
> it became "treat unparsed timezones as +0000".  Later Charles
> replaced "unparsed" by "unparseable".  I'd use "-0000" for
> this problem, and putting it all together we would get:

Charles offered no explanation for that change.

Changing "unparsed" to "unparseable" is a complete change
in meaning.   "Unparseable" means that there is no possible way
to parse.

"Unparsed" seems correct to me, based on the "MAY accept other timezones"
language.

Charles?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1042 USEFOR Newsgroups folding
Date:  Tue, 21 Jun 2005 18:45:55 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 11
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
 
> Makes sense?

Yes.  As far as I'm concerned somebody should replace all the
utter dubious [FWS] directly after the colon-space by a *WSP,
dito all [FWS] directly before the CRLF.  I don't believe in
an optional "F"-folding explicitly "verboten" by a MUST NOT.

                        Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Date:  Tue, 21 Jun 2005 18:34:05 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 38
Sender: [email-masked]

Russ Allbery wrote:
 
>> But is "EST" to be considered "unknown" for this purpose (I
>> would have no problem with that).
 
> I hope not; it's documented in RFC 2822.

Harald's text was:

| zone = (( "+" / "-" ) 4DIGIT) / "GMT"

GMT is the only obs-zone you MUST accept.  By a general rule
elsewhere you MAY accept all 2822 obs- syntax incl. EST.

>> we don't say that "GMT" is to be interpreted as "+0000".
>> OTOH, RFC 2822 says it for us, and we have mentioned that
>> our "GMT" is a relic of that <obs-zone>, so it _ought_ to
>> be clear.

Yes.  Somebody fixed the "treat unparsed timezones as "GMT",
it became "treat unparsed timezones as +0000".  Later Charles
replaced "unparsed" by "unparseable".  I'd use "-0000" for
this problem, and putting it all together we would get:

| ...treat unparseable timezones as -0000.

Charles proposed to remove the "Posting agents MAY reject".
I'm not sure what Harald meant here, but "posting agents"
make no sense, it probably should be "injecting agents":

| Injecting agents MAY reject articles with unparseable
| timezones.

>> Is everybody happy to leave it at that?

Somehow we're now talking about slightly different texts and
interpretations, my "1028 => resolved" was wrong... ;-)  Bye.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: OT: 78
Date:  Tue, 21 Jun 2005 18:00:55 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 22
Sender: [email-masked]

Charles Lindsey wrote:

> What is this tool that hits you, and exactly what is the
> nature of the breakage that you see?

It's a "norton commander clone" with among many other features
a built-in text viewer.  That beast folds lines if they reach
column 79.  I have many "magic" 78 and 79 in my REXX scripts.

> the reasons I have seen in attempting to justify a 78 or
> 79 limit have nothing to do with the number of characters
> needed to represent an EOL

Is there any other (and preferably less ridiculous) reason...

> the space occupied by visible characters in windows,
> terminals, etc.

...what's that, something with proportional fonts and 79 "M" ?

                          Bye, Frank

Date: Tue, 21 Jun 2005 17:33:53 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1042 USEFOR Newsgroups folding
Sender: [email-masked]

In reviewing the discussion of comments in the References header (#1029), I 
discovered that I had failed to create a ticket for the separate issue of 
folding the Newsgroups header.

I have filed the ticket below. Given the experiment and the result, I 
suggest that we say that agents SHOULD NOT generate folded Newsgroups 
headers, and that the valid reason to generate them is to test whether they 
propagate.

Makes sense?

                     Harald
------------------------------------------------------------
Subject: USEFOR 3.1.5: Newsgroups folding?

USEFOR -04 3.1.5 currently says:

newsgroups = "Newsgroups:" SP newsgroup-list CRLF

newsgroup-list = [FWS] newsgroup-name
*( [FWS] "," [FWS] newsgroup-name ) [FWS]

.....

NOTE: Observe that the syntax does not allow comments within the
Newsgroups header; this is to simplify processing by relaying and
serving agents which have a requirement to process this header
extremely rapidly.

It places no restriction on folding in the Newsgroups header.
It's not clear that there is consensus in the group on this.

On June 10, Charles Lindsey said:

I have just submitted an article with the following headers to a server
using INN 2.4.1. It certainly got stored on that server. I invite you
all to see whether it arrived on any of the groups it was sent to, and
especially indications of how long it took and the route it took would
be interesting, since propagation using only INN 2.4+ servers may be
rather slow. If you do not see it in the named groups, it might be worth
looking to see if your server has filed it under "junk".

Newsgroups: man.test,
uk.test,
misc.test,
de.test
Path: chl
From: "Charles Lindsey" <[email-masked]>
Subject: Testing folded newsgroups.
Message-ID: <[email-masked]>
Date: Fri, 10 Jun 2005 18:32:18 GMT
Lines: 14

It would also be helpful if others who have relaying privileges would
try the same experiment.

On June 12, Richard Clayton reported:

the article

>Message-ID: <[email-masked]>

is not currently (almost 2 days later) available on news.demon.co.uk
(well connected UK ISP server), news.mit.edu (MIT's server) or
news.cam.ac.uk (Cambridge University's server)

retention is sufficient for the article to be seen if it managed to
reach these servers by any flood-fill route (direct routes would have
meant that it arrived in seconds)

>It would also be helpful if others who have relaying privileges would
try
>the same experiment.

the existing result speaks for itself

"Broken" articles like this don't propagate at all well :(

hence any suggestion that folded newsgroups header lines are current
state of the art (rather than some future possibility) seems premature

To: [email-masked]
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 21 Jun 2005 06:59:46 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Current practice seems to be to reject single-component names (and
> various other such things), certainly in newgroup messages and maybe in
> injectors as well (you already told us INN does some of that).

INN only doesn't honor them in control messages in its default
configuration; it has no problem with them in all other contexts.  And, of
course, it creates two single-component names by default (junk and
control), which are then valid in Newsgroups, etc.

That's only one news server, and I think other servers are far less
conservative about things like that.

The INN control message policy is *explicitly* a matter of policy and is
intended to be a USEAGE-equivalent sort of configuration.  In other words,
it's *not* part of the protocol; it's part of what the local administrator
can configure as part of their personal preferences.

> So, in any regionally limited hierarchy, or other private or cooperating
> setup, the sofware is going to have to be modified to suit (a pretty
> easy hack, I imagine).

If they create the groups by applying a checkgroups, there is no problem
creating the single-component group.  If they create a single group via
ctlinnd, there is no problem creating the single-component group.  No
software modifications are needed.  Both of those methods are common.

> So it we say that certain newsgroup-names SHOULD NOT be used in Usenet
> (or outside of cooperating subnets, or however else we choose to say
> it), then surely we are just documenting existing practice.

No.  We're not.

To: [email-masked]
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 21 Jun 2005 06:56:00 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> But is "EST" to be considered "unknown" for this purpose (I would have no
> problem with that).

I hope not; it's documented in RFC 2822.

> BTW, we don't say that "GMT" is to be interpreted as "+0000". OTOH, RFC
> 2822 says it for us, and we have mentioned that our "GMT" is a relic of
> that <obs-zone>, so it _ought_ to be clear. Is everybody happy to leave
> it at that?

I am.  I don't want to duplicate information already in RFC 2822.

Date: Tue, 21 Jun 2005 14:29:05 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Definition of "private" (Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts)
Sender: [email-masked]

--On 20. juni 2005 10:23 +0000 Charles Lindsey <[email-masked]> wrote:

> In <[email-masked]> Russ Allbery <[email-masked]>
> writes:
>
>> You keep using this word "private."  I do not think this word means what
>> you think it means.
>
> I think it was Harald's term. If there is doubt about its meaning, then a
> definition would be needed. There is a clear enough definition of
> "cooperating subnet" in son-of-1036 and in our earlier drafts.

RFC 2434, guidelines for writing IANA sections:

      Private Use - For private or local use only, with the type and
           purpose defined by the local site. No attempt is made to
           prevent multiple sites from using the same value in different
           (and incompatible) ways. There is no need for IANA to review
           such assignments and assignments are not generally useful for
           interoperability.

           Examples: Site-specific options in DHCP [DHCP] have
           significance only within a single site.  "X-foo:" header
           lines in email messages.

RFC 2045 section 5.1, MIME content-type:

    (1)   Private values (starting with "X-") may be defined
          bilaterally between two cooperating agents without
          outside registration or standardization. Such values
          cannot be registered or standardized.

The essential point of "private" (and the one that has transfer value to 
this discussion) is "IETF does not try to make rules about what you do".
I think.

              Harald

Date: Tue, 21 Jun 2005 12:48:41 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

--On 20. juni 2005 11:12 +0000 Charles Lindsey <[email-masked]> wrote:

>> It's bad in an <id-domain> in the case of dynamic or reserved
>> IPs without a clear owner of this name space - discussing havoc
>> caused by msg-id collisions with say IANA is pointless.
>
> It won't cause havoc unless two sites have been allocated the same dynamic
> IP, in which case they already have severe problems with all sorts of
> protocols.

10.0.0.1.

Date: Tue, 21 Jun 2005 13:57:24 +0200
From: Eivind Tagseth <[email-masked]>
To: [email-masked]
Subject: Re: OT: 78 (was Forward compatibility)
Mail-Followup-To: Eivind Tagseth <[email-masked]>, [email-masked]
Sender: [email-masked]

* Charles Lindsey <[email-masked]> [2005-06-21 11:11:16 +0000]:
> In <[email-masked]> Eivind Tagseth <[email-masked]> writes:
> 
> >I always thought 78 was used to allow for a limited set of quotation 
> >characters?

> USEAGE recommends a limit of 72 if you want to allow for quotation, and 79
> if you don't really care about being quoted.

Ah... sorry, got those two numbers mixed up.

Eivind

To: [email-masked]
Xref: clerew local.usefor:21392
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Date: Tue, 21 Jun 2005 10:51:58 GMT
Lines: 40
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> I just proposed such a wording in reply to Harald. s/MUST NOT/SHOULD NOT/
>> in that wording.

>> And whatever wording is used for single-component names should be
>> applicable also to some of the other cases under discussion (e.g. "+"
>> and "-" components).

>This is all utter nonsense.  Drawing these arbitrary distinctions defeats
>the entire point of writing a standard, namely enabling interoperable
>software.

Current practice seems to be to reject single-component names (and various
other such things), certainly in newgroup messages and maybe in injectors
as well (you already told us INN does some of that).

So, in any regionally limited hierarchy, or other private or cooperating
setup, the sofware is going to have to be modified to suit (a pretty easy
hack, I imagine).

So it we say that certain newsgroup-names SHOULD NOT be used in Usenet (or
outside of cooperating subnets, or however else we choose to say it), then
surely we are just documenting existing practice.

BTW, I am now persuaded that it is SHOULD NOT rather than MUST NOT for
these cases (since no interoperability problem arises).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21391
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Date: Tue, 21 Jun 2005 10:39:23 GMT
Lines: 40
Sender: [email-masked]

>> 3.1.2 Date

>>  Relaying agents SHOULD treat unparsed timezones as "GMT".
                                 ^^^^^^^^
				 unparseable
>>  Posting agents MAY reject articles with unparsed timezones.

>I do wonder whether that last sentence is even necessary, though, or
>perhaps if it shouldn't instead be phrased as a caveat to the previous
>sentence.  Maybe something like:

>    Software that accepts dates with unknown timezones SHOULD treat such
>    timezones as equivalent to "GMT".

Yes, I prefer that, but I also accept Seth's point that it should be
equivalent to "+0000".

But is "EST" to be considered "unknown" for this purpose (I would have no
problem with that). Actually, it is not clear from Harald's text either.
Perhaps s/SHOULD/MAY/ if you want to be excessively liberal.

BTW, we don't say that "GMT" is to be interpreted as "+0000". OTOH, RFC
2822 says it for us, and we have mentioned that our "GMT" is a relic of
that <obs-zone>, so it _ought_ to be clear. Is everybody happy to leave
it at that?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21390
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 affects USEPRO followups (was: #1008 USEFOR 1.5 References: Final text on "followup")
Date: Tue, 21 Jun 2005 10:29:37 GMT
Lines: 29
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>The following text in 2822 is incorrect for your msg-id:

>| Where any quoted-pair appears, it is to be interpreted as the
>| text character alone.  That is to say, the "\" character that
>| appears as part of a quoted-pair is semantically "invisible".
>[...]
>| The only places in this standard where quoted-pair currently
>| appears are ccontent, qcontent, dcontlnt, no-fold-quote, and
>| no-fold-literal.

><no-fold-quote>   is used by <id-left>
><no-fold-literal> is used by <id-right>

But the syntax of <no-fold-quote> and <no-fold-literal> is different in
our draft than it is in RFC 2822. In particular, there is no <quoted-pair>
anywhere within it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21393
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Tue, 21 Jun 2005 10:59:41 GMT
Lines: 39
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>I find:

>   This specification uses the terms "header", "header name", and
>   "header content" which are synonymous with the [RFC2822] terms
>   "header field", "field name", and "field body" respectively.

>Ok, documented, but absolutely no justification.

Yes. There was justification in the old article-* drafts, but it has sadly
got lost in the rush to make USEFOR as brief (and unhelpful) as possible
:-( .

>Under section three, I see ABNF where we use the term "field" for what we
>want to call the "header name", so we we haven't completely changed usage.

Yes, that is in the ABNF, where we are giving a syntax rule that has to
interwork with the rest of the RFC 2822 syntax.

>When a site that is hosting spam tells me to send the "full header" of 
>email I report as spam, should I be asking them "which header do you want 
>in its entirety?"

I have seen lots of messages from abuse@... sites, but never one asking me
to send a "full header". Lots of them asking me to send "full headers",
though.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21395
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: OT: 78 (was Forward compatibility)
Date: Tue, 21 Jun 2005 11:11:16 GMT
Lines: 21
Sender: [email-masked]

In <[email-masked]> Eivind Tagseth <[email-masked]> writes:

>I always thought 78 was used to allow for a limited set of quotation 
>characters?

Hmmmm! I have never seen that explanation offered before, and it is pretty
weak insofar as it only allows for one level of quotation.

USEAGE recommends a limit of 72 if you want to allow for quotation, and 79
if you don't really care about being quoted.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21394
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: OT: 78 (was: Forward compatibility)
Date: Tue, 21 Jun 2005 11:06:24 GMT
Lines: 25
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>It's an ordinary "off by one" thing, I have exactly one tool
>where it hits me.  If you port *NIX software to DOS, Windows,
>or OS/2, CrLf is one character more than Lf, and then a simple
>if ( 80 <= n ) strikes for 78 + 2.

What is this tool that hits you, and exactly what is the nature of the
breakage that you see?

All the reasons I have seen in attempting to justify a 78 or 79 limit have
nothing to do with the number of characters needed to represent an EOL,
and have everthing to do with the space occupied by visible characters in
windows, terminals, etc.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 21 Jun 2005 09:04:15 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Seth Breidbart <[email-masked]>, [email-masked]
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Sender: [email-masked]

--On 20. juni 2005 13:31 -0400 Seth Breidbart <[email-masked]> wrote:

> Russ Allbery <[email-masked]> wrote:
>
>>    Software that accepts dates with unknown timezones SHOULD treat such
>>    timezones as equivalent to "GMT".
>
> Since GMT is a "SHOULD NOT" I'd prefer not to bless it.  Is there any
> reason not to write '... equivalent to "+0000".'?

None that I can see. Good catch!

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  #1003 affects USEPRO followups (was: #1008 USEFOR 1.5 References: Final text on "followup")
Date:  Mon, 20 Jun 2005 22:23:00 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 28
Sender: [email-masked]

Charles Lindsey wrote:

>> quoted-pair backslash _is_ "semantically invisible" in
>> both <id-left> and <id-right>, because 2822 says so.

> No, a "very simple parser" probably would not treat the
> backslash specially at all.

IBTD.  But this is actually a part of "#1003 fix msg-id".

The following text in 2822 is incorrect for your msg-id:

| Where any quoted-pair appears, it is to be interpreted as the
| text character alone.  That is to say, the "\" character that
| appears as part of a quoted-pair is semantically "invisible".
[...]
| The only places in this standard where quoted-pair currently
| appears are ccontent, qcontent, dcontlnt, no-fold-quote, and
| no-fold-literal.

<no-fold-quote>   is used by <id-left>
<no-fold-literal> is used by <id-right>

The "\"-problem does not exist in the <id-local>, <id-domain>,
<id-quote>, and <id-literal> incarnation of the <msg-id> ABNF.

                             Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Date:  Mon, 20 Jun 2005 21:57:16 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 7
Sender: [email-masked]

Henry Spencer wrote:
 
> you've got one remaining mention of "UT" (in the "SHOULD NOT
> generate") which is probably best removed

#1028 => resolved

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  OT: 78 (was: Forward compatibility)
Date:  Mon, 20 Jun 2005 21:33:50 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 13
Sender: [email-masked]

Charles Lindsey wrote:
 
> I have never seen such a window in which complicated things
> happened if you tried to put a character in column 79. Hence
> I never understood the "78" figure in RFC 2822, in spite of
> asking for a justification on several occasions.

It's an ordinary "off by one" thing, I have exactly one tool
where it hits me.  If you port *NIX software to DOS, Windows,
or OS/2, CrLf is one character more than Lf, and then a simple
if ( 80 <= n ) strikes for 78 + 2.
                                    Bye, Frank

Date: Mon, 20 Jun 2005 13:31:26 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Sender: [email-masked]

Russ Allbery <[email-masked]> wrote:

>    Software that accepts dates with unknown timezones SHOULD treat such
>    timezones as equivalent to "GMT".

Since GMT is a "SHOULD NOT" I'd prefer not to bless it.  Is there any
reason not to write '... equivalent to "+0000".'?

Seth

Date: Mon, 20 Jun 2005 09:33:38 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>If we use RFC2822 terms differently than RFC2822, then we must have
>>already documented the difference and the justification somewhere.

>Indeed. Which is why USEFOR-04 documents it.

Great. So, I'll go to the IETF-USEFOR working group web page and look 
under "latest drafts" to see what we say ... ummmm, nope, no USEFOR-04.
According to us, our latest draft is USEFOR-03.

No problem, IETF itself has documents we don't provide to group members, 
so I'll use their copy.

I find:

   This specification uses the terms "header", "header name", and
   "header content" which are synonymous with the [RFC2822] terms
   "header field", "field name", and "field body" respectively.

Ok, documented, but absolutely no justification.

Under section three, I see ABNF where we use the term "field" for what we
want to call the "header name", so we we haven't completely changed usage.

I find no mention at all of the term "body", so not only do we not 
justify a change to that term, we don't document it.
 
While scanning the entire USEFOR-04 document for the word "header" I find 
a lot of places where the word occurs, which could easily be changed to 
follow RFC2822 terms using a good editing program. It would take more 
effort to list all the places than to just make the changes. 

So, what is the justification?

>The justification is long standing usage in the News community (and in the
>email community too), and in previous News documents.

Really? I've always learned that a message consists of a header and a 
body, and that the header consists of one or more header fields. Sometimes 
the word "line" is used instead of "field", but the essence is the same.

When a site that is hosting spam tells me to send the "full header" of 
email I report as spam, should I be asking them "which header do you want 
in its entirety?"

This "we're news, they're email" nonsense doesn't justify confusing and 
contradictory use of terms between the two. The fact that we are working 
on a technical standard an not a "USENET for Dummies" book means we ought 
to use the correct technical terms and not slang.

>Either we change all of the relevent terms consistently, or we change none
>of them. Mix'n'match certainly won't work.

That's right. And since there is no valid reason to use different terms 
(other than "ohh, look how hard it would be to change all the places it 
occurs...") we should be consistent.

To: [email-masked]
Xref: clerew local.usefor:21379
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Mon, 20 Jun 2005 11:12:53 GMT
Lines: 70
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> 2822 says more than that. A little further down:
>[...]
>>| some domain identifier

>Yes, that's not good enough, it SHOULD be FQDN allowing domain
>literal only for special cases.

Why? If it's good enough for RFC 2822, it's good enough for us.

>> Hold on! We never agreed that domain literals were a Bad
>> Thing to use in an <id-right>.

>That's not how I recall it, ...

Feel free to trawl the archives.

>It's bad in the path-identifier where it breaks Bruce's B-News
>monster in the case of IPv6 colons.

By the time IPv6 addresses actually come to be used in a Path header, one
can reasonably suppose that software will have been upgraded. Even where it
is not, it is only the un-upgraded site that will experience any problem
(i.e. it may erroneously relay an article to a peer that has already had
it).

>It's bad in reply addresses where normal users with a normal
>smart host and a normal MUA (not-direct to IP, let alone IPv6)
>might be unable to use it.

All current UAs should (and AIUI do) send mail correctly to addresses using
domain literals.

>It's bad in an <id-domain> in the case of dynamic or reserved
>IPs without a clear owner of this name space - discussing havoc
>caused by msg-id collisions with say IANA is pointless.

It won't cause havoc unless two sites have been allocated the same dynamic
IP, in which case they already have severe problems with all sorts of
protocols.

>So it's a Bad Thing everywhere, exactly as STD 11 says.

No.

>only replace the bogus 2822 notation <id-right> by <id-domain>.

>STD 11 is also normative for RfC 1036, so if you introduce the
>unclear <id-right> concept in USEFOR this would also pop-up in
>a "differences from RfC 1036" appendix:  "random garbage in the
><id-right> of a <msg-id> is now allowed, but not recommended".

A domain literal (if it is syntactically a valid Internet address) is not
"random garbage".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 20 Jun 2005 08:25:46 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> 3.1.2 Date

>  The Date header is the same as that specified in Sections 3.3 and 3.6.1
>  of [RFC2822] with the added restrictions detailed in Section
>  2.2. However, the use of "GMT" as time zone (part of <obs-zone>),
>  although deprecated, is widespread in news articles today. Therefore,
>  agents MUST accept <date-time> constructs which include the updated
>  <zone> construct below.

>  orig-date = "Date:" SP date-time CRLF

>  zone = (( "+" / "-" ) 4DIGIT) / "GMT"

>  Note that agents SHOULD NOT generate <date-time> constructs which
>  include either "UT" or "GMT" and MUST NOT generate <date-time>
>  constructs which include any other zone names defined by <obs-zone>.

>  Relaying agents SHOULD treat unparsed timezones as "GMT".
>  Posting agents MAY reject articles with unparsed timezones.

> (the last part may belong in USEPRO. But as Russ argues in another
> context... it affects the way you want to write your article parser....)

The first sentence of the last paragraph is a format issue.  The second
sentence is somewhat borderline, I agree, particularly since (unless I'm
mistaken) the format document hasn't really defined what rejecting an
article means.  But the common-sense definition is applicable, so it's not
a huge deal.

I do wonder whether that last sentence is even necessary, though, or
perhaps if it shouldn't instead be phrased as a caveat to the previous
sentence.  Maybe something like:

    Software that accepts dates with unknown timezones SHOULD treat such
    timezones as equivalent to "GMT".

The rule isn't specific to relaying agents, after all, and that sentence
leaves open the possibility that software may decide to not deal with such
timezones at all.

To: [email-masked]
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 20 Jun 2005 08:21:08 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Is there such a thing as a "regionally limited" group?

Yes.

> I think you will find some servers (supernews, I imagine) which carry
> the groups for every known region.

And you would be wrong.  But really, it hardly matters; the success of
private peering arrangements in keeping certain regional hierarchies
regional is really entirely beside the point that we're discussing.

> But groups internal to a large organization, or only available by
> subscription (or other contractual arrangements, such as Usenet2) are
> definitely "private" or "cooperating subnets", and so they may have
> single-component names if they want. At any rate, that seems to be the
> current practice, more or less.

> I suspect that, as regards acceptance by injecting agents and
> propagation, the standard should say "SHOULD NOT" rather than "MUST NOT"
> (though I would leave it as MUST NOT for newgroup messages). But it
> should still be accompanied by wording to indicate that exceptions to
> the SHGOULD are really only for private/cooperating use.

> I just proposed such a wording in reply to Harald. s/MUST NOT/SHOULD NOT/
> in that wording.

> And whatever wording is used for single-component names should be
> applicable also to some of the other cases under discussion (e.g. "+"
> and "-" components).

This is all utter nonsense.  Drawing these arbitrary distinctions defeats
the entire point of writing a standard, namely enabling interoperable
software.  The software used for so-called "private" hierarchies is no
different than the software used for any other "public" hierarchy, and
saying that they have different requirements is like saying that certain
headers "SHOULD NOT" be used in a mail message when it's sent outside of
one's local organization.  It's an absurd rule; mail software has to be
able to deal with mail from any source, archives of mail messages have to
be portable and parsable, and mail software has no way of knowing whether
such an arbitrary distinction applies to a particular message.  Usenet is
no different.

Date: Mon, 20 Jun 2005 10:46:43 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1028 Usefor 3.1.2 Timezones: Suggested text
Sender: [email-masked]

On Mon, 20 Jun 2005, Harald Tveit Alvestrand wrote:
> Reviewing the discussion on #1028, I suggest the following modification:

Looks reasonable... except that you've got one remaining mention of "UT"
(in the "SHOULD NOT generate") which is probably best removed -- without
the earlier context, it's pretty cryptic and probably unnecessary. 

                                                          Henry Spencer
                                                       [email-masked]

Date: Mon, 20 Jun 2005 16:12:44 +0200
From: Eivind Tagseth <[email-masked]>
To: [email-masked]
Subject: Re: Forward compatibility
Mail-Followup-To: Eivind Tagseth <[email-masked]>, [email-masked]
Sender: [email-masked]

* Charles Lindsey <[email-masked]> [2005-06-20 09:53:34 +0000]:
> In <[email-masked]> Frank Ellermann <[email-masked]> writes:
> 
> Indeed. I have seen lots of 80 column windows which make life complicated
> if you actually try to put a character in column 80 (hence the advice to
> keep lines at or below 79).
> 
> But I have never seen such a window in which complicated things happened
> if you tried to put a character in column 79. Hence I never understood the
> "78" figure in RFC 2822, in spite of asking for a justification on several
> occasions.

I always thought 78 was used to allow for a limited set of quotation 
characters?

When following up on a message with 79/80 characters wide lines, formatting
any quoted text is a pain.

Eivind

To: [email-masked]
Xref: clerew local.usefor:21374
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Forward compatibility
Date: Mon, 20 Jun 2005 09:53:34 GMT
Lines: 27
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Kai Henningsen wrote:

>> 80 character screens certainly still exist.

>Every normal text mode window uses it, 80 is better than 40.

Indeed. I have seen lots of 80 column windows which make life complicated
if you actually try to put a character in column 80 (hence the advice to
keep lines at or below 79).

But I have never seen such a window in which complicated things happened
if you tried to put a character in column 79. Hence I never understood the
"78" figure in RFC 2822, in spite of asking for a justification on several
occasions.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21376
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Date: Mon, 20 Jun 2005 10:23:17 GMT
Lines: 56
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>You keep using this word "private."  I do not think this word means what
>you think it means.

I think it was Harald's term. If there is doubt about its meaning, then a
definition would be needed. There is a clear enough definition of
"cooperating subnet" in son-of-1036 and in our earlier drafts.

>People use the same software and the same protocol for both world-wide
>public groups, regionally limited groups, groups hosted by a single
>server, or groups internal to (often large) organizations or ISPs.  The
>protocol description should be the same for all of them, since it is the
>same protocol and the same interoperability concerns apply.  If you want
>to draw distinctions about best practices, that's what USEAGE is for.

Is there such a thing as a "regionally limited" group? I think you will
find some servers (supernews, I imagine) which carry the groups for every
known region. Anyway, I think Usenet = "world-wide public groups including
regional groups of every sort". They should be obeying all the rules
(including no single-component names).

Whether single server groups are part of Usenet is debateable. If they are
publicly available, then they will surely "leak" sooner of later (as
Micrsoft discovered). For that reason, I think the single-component
"spamcop" is regrettable.

But groups internal to a large organization, or only available by
subscription (or other contractual arrangements, such as Usenet2) are
definitely "private" or "cooperating subnets", and so they may have
single-component names if they want. At any rate, that seems to be the
current practice, more or less.

I suspect that, as regards acceptance by injecting agents and propagation,
the standard should say "SHOULD NOT" rather than "MUST NOT" (though I
would leave it as MUST NOT for newgroup messages). But it should still be
accompanied by wording to indicate that exceptions to the SHGOULD are
really only for private/cooperating use.

I just proposed such a wording in reply to Harald. s/MUST NOT/SHOULD NOT/
in that wording.

And whatever wording is used for single-component names should be
applicable also to some of the other cases under discussion (e.g. "+" and
"-" components).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21375
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Semantic & Header content (Re: #1008 USEFOR 1.5 References: Final text on "followup")
Date: Mon, 20 Jun 2005 10:02:14 GMT
Lines: 63
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>--On torsdag, juni 16, 2005 12:11:50 +0000 Charles Lindsey 
><[email-masked]> wrote:

>>    o  All agents MUST generate headers so that at least one space
>>       immediately follows the ':' separating the header name and the
>>       header contents (for compatibility with deployed software).  News
>>       agents MAY accept headers which do not contain the required space.
>>
>>    o  The header contents of every header line (including the first and
>>       any that are subsequently folded) MUST contain at least one non-
>>       whitespace character.
>>
>>          NOTE: This means that no header content defined by or
>>          referenced by this document can be empty.  As a result, this
>>          document updates the <unstructured> construct from Section
>>          3.2.6 of [RFC2822] as follows:
>>
>>    unstructured    =  1*( [FWS] utext ) [FWS]

>The first instance seems to say that the space after the colon is not 
>considered part of the header content.

Indeed, but the definition of "header field body" in RFC 2822 clearly
includes that SP (that was the initial example which lead to the need to
define something more precise for USEPRO).

>The second one is formally wrong (you can't have "header content" of a 
>"header line", since "header content" means the whole header field's 
>content).

Yes.

>The third one implicitly uses "empty" in the sense of "something that is 
>not whitespace". That's not clear either.

Perhaps the word "(in)visible" would help.

>Since this paragraph needs some rework anyway to be totally 
>self-consistent, perhaps we should free up "content" for what Charles 
>wants, and define as follows:

OK.

>  "header field content" ("content" for short) ...

Well it will be "header content" or "header field content" according to
whether we decide to revert to strict RFC 2822 termonology, or to retain
the "News-style" terminology in the present draft. I still vote for the
latter.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21378
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Mon, 20 Jun 2005 10:46:28 GMT
Lines: 47
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>  [backslash]
>>> ignoring these "invisible" characters in a msg-id-list is a
>>> bad plan for the References construction and trimming
> 
>> No, look at our current syntax for <msg-id>. There is not a
>> <quoted-pair> in sight.

>Are you sure that a valid 2822 <id-left> or <id-right> never
>shows up in the References of a Netnews article ?

It might, if some gateway lets it through. But if the References header
contains <ab\[email-masked]> and the article itself contains
<[email-masked]), do not expect a news reader to thread the article in
its proper place. Come to that, I doubt many mail readers would actually
thread it correctly.

>As an implementor of any combined MUA and news UA I'd probably
>pick a very simple parser, and then the quoted-pair backslash
>_is_ "semantically invisible" in both <id-left> and <id-right>,
>because 2822 says so.

No, a "very simple parser" probably would not treat the backslash
specially at all.

>It's no real problem for the References if some non-canoncial
>Message-IDs are mangled by a simple implementation.  But I'm
>stunned by your confidence that everybody implements USEFOR
>correctly only because we say MUST and hide major differences
>from 2822 deep within obscure <mqspecial> or <no-fold-literal>

They are not hidden. There is explanation, including the various
<[email-masked]> examples.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21377
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Mon, 20 Jun 2005 10:34:10 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>No, not in USEFOR as currently written.

>That's why I wrote "that's what RFC2822 tells us."

>If we use RFC2822 terms differently than RFC2822, then we must have
>already documented the difference and the justification somewhere.

Indeed. Which is why USEFOR-04 documents it.

The justification is long standing usage in the News community (and in the
email community too), and in previous News documents.

>According to RFC2822, it already is "header field body". Do we have a 
>reason for it to be something else?

Either we change all of the relevent terms consistently, or we change none
of them. Mix'n'match certainly won't work.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 20 Jun 2005 09:44:51 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1028 Usefor 3.1.2 Timezones: Suggested text
Sender: [email-masked]

Trying to get a simple one settled.....

Reviewing the discussion on #1028, I suggest the following modification:

OLD:

3.1.2 Date

 The Date header is the same as that specified in Sections 3.3 and
 3.6.1 of [RFC2822] with the added restrictions detailed in
 Section 2.2. However, the use of "UT" and "GMT" as time zones
(part
 of <obs-zone>), although deprecated, is widespread in news articles
 today. Therefore, agents MUST accept <date-time> constructs which
 include the updated <zone> construct below.

 orig-date = "Date:" SP date-time CRLF

 zone = (( "+" / "-" ) 4DIGIT) / "UT" / "GMT"

 Note that agents SHOULD NOT generate <date-time> constructs which
 include either "UT" or "GMT" and MUST NOT generate <date-time>
 constructs which include any other zone names defined by
<obs-zone>,
 some of which have ambiguous interpretations and would have adverse
 effects on the Netnews protocols.

NEW:

3.1.2 Date

 The Date header is the same as that specified in Sections 3.3 and
 3.6.1 of [RFC2822] with the added restrictions detailed in
 Section 2.2. However, the use of "GMT" as time zone
(part
 of <obs-zone>), although deprecated, is widespread in news articles
 today. Therefore, agents MUST accept <date-time> constructs which
 include the updated <zone> construct below.

 orig-date = "Date:" SP date-time CRLF

 zone = (( "+" / "-" ) 4DIGIT) / "GMT"

 Note that agents SHOULD NOT generate <date-time> constructs which
 include either "UT" or "GMT" and MUST NOT generate <date-time>
 constructs which include any other zone names defined by
<obs-zone>.

 Relaying agents SHOULD treat unparsed timezones as "GMT".
 Posting agents MAY reject articles with unparsed timezones.

(the last part may belong in USEPRO. But as Russ argues in another 
context... it affects the way you want to write your article parser....)

Date: Sun, 19 Jun 2005 09:04:54 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: ADMIN: Ticket system changes
Sender: [email-masked]

I've had a new field added to the ticket system configuration.
It's called "Discussion", and is intended to reflect the state of 
discussion on the issue.

The values are:

No discussion
No consensus
Text needed
Text proposed
Text accepted
Proposed no change
No change needed
Document updated

The intention is that "no consensus" is the state while we're still 
talking; "text needed" is when we think we know a solution but don't have a 
text yet; "text proposed" and "proposed no change" should be obvious; "text 
accepted" and "no change needed" are the states after consensus on the 
issue is declared.

Once the editor integrates the text into his running copy, he can change 
the value to "Document updated" and close the ticket.

This proved an effective means of working in the IASA BCP discussions, so 
I'd like to do it again.

As part of this change, I've changed ticket #1008 to be "Text accepted" 
rather than being a closed ticket.

                     Harald

Date: Sat, 18 Jun 2005 10:56:35 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Admin: Posters this week
Sender: [email-masked]

Just FYI....

Searching for msgs since 11-Jun-2005
  1   39  28.06 "Charles Lindsey" <[email-masked]>
  2   24  45.32 Frank Ellermann <[email-masked]>
  3   21  60.43 Russ Allbery <[email-masked]>
  4   17  72.66 Harald Tveit Alvestrand <[email-masked]>
  5    7  77.70 Henry Spencer <[email-masked]>
  6    6  82.01 Richard Clayton <[email-masked]>
  7    6  86.33 Bruce Lilly <[email-masked]>
  8    4  89.21 "Ruud H.G. van Tol" <[email-masked]>
  9    4  92.09 John Stanley <[email-masked]>
 10    2  93.53 Seth Breidbart <[email-masked]>
 11    2  94.96 [email-masked] (Kai Henningsen)
 12    2  96.40 "Forrest J. Cavalier III" <[email-masked]>
 13    2  97.84 Alexey Melnikov <[email-masked]>
 14    2  99.28 Graham Drabble <[email-masked]>
 15    1 100.00 Sebastian Brocks <[email-masked]>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Newsgroup names #1021
Date:  Fri, 17 Jun 2005 21:39:28 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 13
Sender: [email-masked]

Charles Lindsey wrote:

> Why-oh-why are we spending so much time and effort on
> rediscovering wheels which were invented years ago in our
> earlier drafts?

Because so far not one draft back to MESSFOR got the <msg-id>
right, and so it's the natural next step to visit the other
fundamental idea of NetNews again, the <newsgroup-name>.

Apparently Harald has a knack to ask the right questions, if
we couldn't answer it we'd be in serious trouble.  Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date:  Fri, 17 Jun 2005 21:26:50 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 37
Sender: [email-masked]

Charles Lindsey wrote:

  [backslash]
>> ignoring these "invisible" characters in a msg-id-list is a
>> bad plan for the References construction and trimming
 
> No, look at our current syntax for <msg-id>. There is not a
> <quoted-pair> in sight.

Are you sure that a valid 2822 <id-left> or <id-right> never
shows up in the References of a Netnews article ?

As an implementor of any combined MUA and news UA I'd probably
pick a very simple parser, and then the quoted-pair backslash
_is_ "semantically invisible" in both <id-left> and <id-right>,
because 2822 says so.

It's no real problem for the References if some non-canoncial
Message-IDs are mangled by a simple implementation.  But I'm
stunned by your confidence that everybody implements USEFOR
correctly only because we say MUST and hide major differences
from 2822 deep within obscure <mqspecial> or <no-fold-literal>

| The only places in this standard where quoted-pair currently
| appears are ccontent, qcontent, dcontent, no-fold-quote, and
| no-fold-literal.

How did the 2822 author get away with "this standard", Bruce on
vacation ?  Anyway, if you want to stick to a <no-fold-literal>
in your syntax you need again more prose explainig that you've
disabled this rule in USEFOR.  

With <id-quote>, <id-literal>, etc. in my syntax this problem
doesn't exist right from the start.  No extra prose needed.

                             Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date:  Fri, 17 Jun 2005 20:50:04 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 48
Sender: [email-masked]

Charles Lindsey wrote:

> 2822 says more than that. A little further down:
[...]
>| some domain identifier

Yes, that's not good enough, it SHOULD be FQDN allowing domain
literal only for special cases.

>> RfC 2822 also doesn't discourage domain literals generally
>> like STD 11.  We have to fix it by a clear SHOULD NOT as in
>> STD 11.

> Hold on! We never agreed that domain literals were a Bad
> Thing to use in an <id-right>.

That's not how I recall it, and a domain-literal is everywhere
a Bad Thing:

It's bad in the path-identifier where it breaks Bruce's B-News
monster in the case of IPv6 colons.

It's bad in reply addresses where normal users with a normal
smart host and a normal MUA (not-direct to IP, let alone IPv6)
might be unable to use it.

It's bad in an <id-domain> in the case of dynamic or reserved
IPs without a clear owner of this name space - discussing havoc
caused by msg-id collisions with say IANA is pointless.

So it's a Bad Thing everywhere, exactly as STD 11 says.

>| any <no-fold-literal> used for the <id-right> should be of
>| the form of an Internet address, in accordance with Section
>| 3.4.1 of [RFC2822].

All these tons of text are so obvious and unnecessary if you'd
only replace the bogus 2822 notation <id-right> by <id-domain>.

STD 11 is also normative for RfC 1036, so if you introduce the
unclear <id-right> concept in USEFOR this would also pop-up in
a "differences from RfC 1036" appendix:  "random garbage in the
<id-right> of a <msg-id> is now allowed, but not recommended".

No idea how you can hope to smuggle this through a "last call".

                         Bye, Frank

Date: Fri, 17 Jun 2005 11:32:11 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>No, not in USEFOR as currently written.

That's why I wrote "that's what RFC2822 tells us."

If we use RFC2822 terms differently than RFC2822, then we must have
already documented the difference and the justification somewhere.

>There is a discussion in progress
>as to whether to change USEFOR to adhere strictly to the RFC 2822
>terminology, and if that change is made then the "header body" that I have
>asked for will become "header field body".

According to RFC2822, it already is "header field body". Do we have a 
reason for it to be something else?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Forward compatibility
Date:  Fri, 17 Jun 2005 20:13:30 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 31
Sender: [email-masked]

Kai Henningsen wrote:

> When doing my small bit of C programming on that machine, I
> certainly need to cope with fixed-80 format for C source.

And with the RfC format they could punch a serial card number
in the columns 73..80, and that is exactly as it should be :-)

> 80 character screens certainly still exist.

Every normal text mode window uses it, 80 is better than 40.

>> making column 80 "dangerous territory".
> Curses fortunately can handle that

On a PC with a straight forward BIOS interface, virtual or
real, I found no way around "stay clear of row 25 column 80
in the string output function".  Yes, PCcurses handles it,
but internally it has to care about it.

> The one annoying problem left is that these terminals
> usually cannot write into the lower right corner (because
> doing so causes immediate scrolling).

It works if you use single char output in that position -
test Mark's PDcurses or for DOS test PCcurses (sorry, the
latter has no mouse support).  For ncurses it would be a
problem in the termcap-layer, not in ncurses.

                       Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Date:  Fri, 17 Jun 2005 19:49:47 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 20
Sender: [email-masked]

Charles Lindsey wrote:

> See also separate discussion between Russ and Henry as to
> whether "cooperating subnets" are a useful concept to
> mention.

Nowhere I've seen a concept "something like a path MUST have
at least two components", the only exception I know is 2821
with its "at least one dot rule", and the author suggested to
fix it in a future 2821bis for the rare cases like `host ai`
ai = 209.88.68.34

Essentially the same idea or problem as a newsgroup spamcop,
or any filesystem with interesting ideas about /dev/ etc.

And for a filesystem you wouldn't forbid files in the root
directory only to protect pseudo-subdirectories like /dev/.

                          Bye, Frank

To: Usefor Mailing List <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 17 Jun 2005 10:39:17 -0700
Sender: [email-masked]

Henry Spencer <[email-masked]> writes:
> On Fri, 17 Jun 2005, Russ Allbery wrote:

>> People use the same software and the same protocol for both world-wide
>> public groups, regionally limited groups, groups hosted by a single
>> server, or groups internal to (often large) organizations or ISPs.  The
>> protocol description should be the same for all of them...

> Except that it *can't* be, because practices which we *cannot* endorse
> as the worldwide standard are common in some of those cooperating
> subnets -- for example, the "just send eight" approach to
> greater-than-ASCII character sets is widespread in non-English
> monolingual subnets, but any standard which aims at full
> interoperability has to be more cautious.

"Just send eight" is common in world-wide newsgroups just as much as it is
in more limited hierarchies.  That issue is not related to cooperating
subnets and the concept of cooperating subnets is not particularly helpful
in resolving or working around it.

Do you have any other examples of places where you feel this concept is
useful?

To: Usefor Mailing List <[email-masked]>
Subject: Re: Newsgroup names #1021
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 17 Jun 2005 10:37:02 -0700
Sender: [email-masked]

Henry Spencer <[email-masked]> writes:

> One of the things which certainly has slowed this process down greatly
> is people who obsess about (e.g.) how a "format" document isn't supposed
> to talk about anything but "format" (defined in some narrow way),
> instead of being willing to make pragmatic compromises to convey the
> necessary information most effectively.

I find format vs. protocol to be a useful distinction for writing software
but only if it's done right.  If we just jumble things between the two
documents, it becomes much harder to read than if we had one document.  If
we want to abandon the idea and just write one document, that's something
that can be argued for, but if not, we should do the split in a way that
actually helps keep concepts separate rather than just confuse the issue.

The idea, at least in my opinion, is that one should be able to write an
article parser off the format document, and then put the policy layer of
the software around it while referencing the protocol document.

I know that you opposed splitting the document in the first place, so I'm
not surprised that you consider the discussions that result from trying to
implement the split to be a waste of time.  Certainly, if your feeling is
that it should be one document, it all *is* a waste of time.

However, my understanding of why we split the document in the beginning
was to try to speed up the process of dealing with the format document by
avoiding creeping featurism and try to tighten the scope.  Accordingly,
I'm taking the tack of resisting attempts that I see as trying to
reintroduce all the creeping featurism problems we were having into the
format document, thus miring it in all of the debates we were trying to
sidestep by splitting the documents in the first place.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Semantic & Header content
Date:  Fri, 17 Jun 2005 19:28:07 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 89
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> The term "header content" (RFC 2822 "field body") is used in
> 3 places in USEFOR, all of which are in this paragraph:

I try to translate this for you, otherwise you'd be forced to
read several hundred rounds of ping pong in the [FWS]-threads:

>| All agents MUST generate headers so that at least one space
>| immediately follows the ':' separating the header name and
>| the header contents (for compatibility with deployed
>| software).

That says we really want SP (not WSP or FWS) after the colon.

>| News agents MAY accept headers which do not contain the
>| required space.

Optionally we'd accept no separator (or a HT) after the colon,
but we never generate header fields this way.

>| The header contents of every header line (including the
>| first and any that are subsequently folded) MUST contain at
>| least one non-whitespace character.

That says we don't allow 2*(CRLF 1*WSP) as in LWSP with an
apparently empty line.  Same concept as in 2822 FWS or CFWS.

It also says that we don't allow a first folding before the
first non-whitespace character in the header field body.

So you can have a Subject:<SP><WSP>hello<CRLF>
<WSP>world but not a Subject:<SP><CRLF>
<WSP>hello world.

That's a major difference from RfC 2822, and it's important.

It's an extension of an 2822 "MUST NOT" about CFWS resulting
in apparently empty lines in the header.

>| NOTE: This means that no header content defined by or
>| referenced by this document can be empty.

That says that we don't have e.g. empty Bcc: headers like mail.

>| As a result, this document updates the <unstructured>
>| construct from Section 3.2.6 of [RFC2822] as follows:

>| unstructured    =  1*( [FWS] utext ) [FWS]

We also have no apparently empty Subject, Summary, or Comments.

We want at least one VCHAR or NO-WS-CTL in every line of the
folded header field body.  Actually some of "us" (= me) think
that NO-WS-CTL is madness, but it's no syntax problem here:

Users are free to experiment with FF, VT, etc. as they see fit,
only for the msg-id we explicitly killed this weird NO-WS-CTL.

Replacing <utext> by ( VCHAR / NO-WS-CTL ) is better:  Without
<obs-utext> that's just what it is.  With a "note" to send all
complaints to the persons listed in RfC 2822 chapter 8.  <beg>

The <obs-utext> bit is about isolated CR or LF outside of CRLF,
so we really don't want this anymore in Netnews header fields.

> The second one is formally wrong (you can't have "header
> content" of a "header line", since "header content" means the
> whole header field's content).

It's about the part of the header field body in a folded header
field line.  Maybe modulo terminology that makes sense for me.

> The third one implicitly uses "empty" in the sense of
> "something that is not whitespace". That's not clear either.

It uses "not empty" in the sense of ( VCHAR / NO-WS-CTL ), "no
header content" [...] "can be empty".  Where "empty" is simply
( WSP / CR / LF ) which is just the same as ( FWS / CR / LF ).

> perhaps we should free up "content" for what Charles wants,
[...]
> Does that make sense?

Yes.  You've almost hit another USEFOR rathole from my POV:
Using [FWS] instead of *WSP only because RfC 2822 says [CFWS].

                            Bye, Frank

Date: Fri, 17 Jun 2005 12:55:48 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Newsgroup names #1021
Sender: [email-masked]

On Fri, 17 Jun 2005, Russ Allbery wrote:
> > I found those words in the old draft-13. Why-oh-why are we spending so
> > much time and effort on rediscovering wheels which were invented years
> > ago in our earlier drafts?
> 
> Because what we had years ago in our earlier drafts was not publishable.
> There is no magic pixie dust in the above wording that makes me happy with
> calling to.* newsgroups a format issue.

One of the things which certainly has slowed this process down greatly is
people who obsess about (e.g.) how a "format" document isn't supposed to
talk about anything but "format" (defined in some narrow way), instead of
being willing to make pragmatic compromises to convey the necessary
information most effectively.

                                                          Henry Spencer
                                                       [email-masked]

Date: Fri, 17 Jun 2005 12:50:48 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Sender: [email-masked]

On Fri, 17 Jun 2005, Russ Allbery wrote:
> People use the same software and the same protocol for both world-wide
> public groups, regionally limited groups, groups hosted by a single
> server, or groups internal to (often large) organizations or ISPs.  The
> protocol description should be the same for all of them...

Except that it *can't* be, because practices which we *cannot* endorse as
the worldwide standard are common in some of those cooperating subnets --
for example, the "just send eight" approach to greater-than-ASCII
character sets is widespread in non-English monolingual subnets, but any
standard which aims at full interoperability has to be more cautious.

("Just send eight" is common in email, too, but RFC 2822 explicitly
repudiates it.)

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
Subject: Re: Newsgroup names #1021
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 17 Jun 2005 09:44:36 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> In other words, I'm looking at the distinction between the ABNF for a
>> newsgroup name, which has to allow such things as junk or to.* or
>> control.* and which is used for the Newsgroups header, the Xref header,
>> and so on, and the rules about what newsgroups you should use for what
>> purpose.  The latter feels very strongly like a USEPRO issue to me.

> Except that several people have expressed a strong preference for
> keeping all restrictions of this nature at one place, in USEFOR.

I know.  I understand their point, but I disagree with those people.  If
we have to put all of the restrictions in the same place, I think putting
them all in USEPRO is actually the best of a couple of bad options and
only put the permissive ABNF into USEFOR (since in practice that's a
common implementation strategy anyway -- namely, to allow anything in the
software but then add restrictions in policy).

> But, of course, any prohibition of "To" should be accompanied by a
> wording such as:

>     (reserved for the ihave/sendme protocol described in [USEPRO], and
>     for test articles sent on an essentially point-to-point basis).

> I found those words in the old draft-13. Why-oh-why are we spending so
> much time and effort on rediscovering wheels which were invented years
> ago in our earlier drafts?

Because what we had years ago in our earlier drafts was not publishable.
There is no magic pixie dust in the above wording that makes me happy with
calling to.* newsgroups a format issue.

To: [email-masked]
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 17 Jun 2005 09:41:19 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> The standard is meant to apply to both. But there is no reason why it
> should not make more stringent rules for the publicly available Usenet
> than for private use.

You keep using this word "private."  I do not think this word means what
you think it means.

People use the same software and the same protocol for both world-wide
public groups, regionally limited groups, groups hosted by a single
server, or groups internal to (often large) organizations or ISPs.  The
protocol description should be the same for all of them, since it is the
same protocol and the same interoperability concerns apply.  If you want
to draw distinctions about best practices, that's what USEAGE is for.

To: [email-masked]
Xref: clerew local.usefor:21350
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Date: Fri, 17 Jun 2005 11:19:28 GMT
Lines: 65
Sender: [email-masked]

In <56F1BBE49C633BB0655C3AE3@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>--On 15. juni 2005 13:54 +0000 Charles Lindsey <[email-masked]> wrote:

>>>   use and MUST NOT be generated by posting agents (whether in
>>>   Newsgroups headers or in newgroup control
>>>   messages [USEPRO]) without a private prior agreement to do so.
>>>   However, such names MUST be accepted by
>>>   relaying and serving agents.
>>
>> I think I would say "reserved for private arrangements between serving
>> agents and the reading agents which they serve".
>>
>> Or it could say "reserved for read-only newsgroups, made available by
>> each serving agent to those reading agents which it serves".

>I don't think we have a consensus that this is the only possible scope for 
>these characters - I could be wrong.
>The principle of the thing is: Either we limit them to private use (in 
>which case we should say NOTHING about how they are used), or we limit them 
>to a specific purpose (in which case we need text describing that purpose 
>in enough detail to make sure we understand it).

OK, I think we can satisfy both viewpoints with a little care. But first
some terminology:

Son-of-1036 (and also our earlier drafts) defined the term "cooperating
subnet", which is envisioned as a complete bunch of injectors, relayers,
servers and the lot. So a "private arrangement" just between a serving
agent and its community of clients hardly amounts to a "cooperating
subnet". I.e. "cooperating subnets" is a subset of "private arrangements".

So you say that "[some list of things, such as control.*, junk,
single-component names and components starting with "+" or "-"] are
reserved for private use (for example read-only newsgroups, made available
by a serving agent to those reading agents which it serves)."

>> And I think they should not be accepted by relaying agents etc. So it
>> should then say
>>     "hence they MUST NOT appear in Newsgroups and Followup-To headers,
>> though they MAY appear in Xref headers".

>Relaying agents within or outside the private-agreement area?

And then you say "hence [same list of things] MUST NOT appear in
Newsgroups and Followup-To headers (except in cooperating subnets), though
they MAY appear in Xref headers".

Exactly which of the forbidden things are included in that list is a
separate discussion.

See also separate discussion between Russ and Henry as to whether
"cooperating subnets" are a useful concept to mention. I agree with Henry
on this one.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21352
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Fri, 17 Jun 2005 11:30:51 GMT
Lines: 25
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> The words "semantically invisible" are used at several places
>> in RFC 2822.

>Found it, the other places are backslashes in quoted-pairs, and
>ignoring these "invisible" characters in a msg-id-list is a bad
>plan for the References construction and trimming algorithm.

No, look at our current syntax for <msg-id>. There is not a <quoted-pair>
in sight. All you have is <mqspecial> and nobody has ever said that
anything in an <mqspecial> is "invisible".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21351
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Date: Fri, 17 Jun 2005 11:26:08 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>If this standard, even the protocol pieces (!), is only intended to be
>applicable to the world-wide public Usenet and not any private server or
>other use of the protocols, we're all wasting our time.  A substantial
>portion of the use of the Usenet protocols is outside the publically
>distributed hierarchies, and those uses are frequently the most
>interesting to support.  They have the same interoperability requirements
>and need for a solid protocol as the public use of the Usenet article
>format.

The standard is meant to apply to both. But there is no reason why it
should not make more stringent rules for the publicly available Usenet
than for private use.

So you could quite well say that "single-component newsgroups" (and maybe
various other things) MUST NOT appear {on the public Usenet} except in
prior agreement in cooperating subnets.

>This cooperating subnet nonsense really has to go.

No, I agree with Henry on this one.

>If you want to limit USEAGE to just the public Usenet, ...

Again, USEAGE applies to both, but it may be useful to distinguish Best
Practice on Usenet from what might be acceptable elsewhere.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21354
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Fri, 17 Jun 2005 13:22:27 GMT
Lines: 62
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:
> 
>>> we also agreed to add something along the line of STD 11:
>>>| Note:  THE USE OF DOMAIN-LITERALS IS STRONGLY DISCOURAGED.
> 
>> I think we already inherit that advice from RFC 2822 (along
>> with the RECOMMENDATION to use domains).

>No, we would inherit the erroneous contrary, 2822 says:

>| a good method is to put the domain name (or a domain literal
>| IP address) of the host on which the message identifier was
>| created on the right hand side of the "@"

2822 says more than that. A little further down:

   ....  Though other algorithms will work, it is
   RECOMMENDED that the right hand side contain some domain identifier
   (either of the host itself or otherwise) such that the generator of
   the message identifier can guarantee the uniqueness of the left hand
   side within the scope of that domain.

>Domain literal is _not_ a good method.  RfC 2822 also doesn't
>discourage domain literals generally like STD 11.  We have to
>fix it by a clear SHOULD NOT as in STD 11.

Hold on! We never agreed that domain literals were a Bad Thing to use in
an <id-right>. What we DID agree was that such literals had better be an
IP address in some accepted notation (which is hinted at, but not made
absolutely clear, by some wording in RFC 2822 in connection with
<addr=spec>s, where it says "In the domain-literal form, the domain is
interpreted as the literal Internet address of the particular host").

So what we wrote (and it is now in USEFOR-04) was:

      NOTE: It is RECOMMENDED in [RFC2822] that, for ensuring global
      uniqueness, the <id-right> be some domain identifier within whose
      scope the uniqueness of the <id-left> can be guaranteed.  When
      following this recommendation, any <dot-atom-text> or <no-fold-
      literal> used for the <id-right> are to be interpreted as
      <domain>s as described in Section 3.4.1 of [RFC2822].

Now you might make that clearer by changing the last sentence:

      Hence, when following this recommendation, any <no-fold-literal>
      used for the <id-right> should be of the form of an Internet
      address, in accordance with Section 3.4.1 of [RFC2822].

That would be fine by me if Harald and Ken are agreeable.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21355
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Newsgroup names #1021
Date: Fri, 17 Jun 2005 13:29:53 GMT
Lines: 31
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>In other words, I'm looking at the distinction between the ABNF for a
>newsgroup name, which has to allow such things as junk or to.* or
>control.* and which is used for the Newsgroups header, the Xref header,
>and so on, and the rules about what newsgroups you should use for what
>purpose.  The latter feels very strongly like a USEPRO issue to me.

Except that several people have expressed a strong preference for keeping
all restrictions of this nature at one place, in USEFOR.

But, of course, any prohibition of "To" should be accompanied by a wording
such as:

    (reserved for the ihave/sendme protocol described in [USEPRO], and
    for test articles sent on an essentially point-to-point basis).

I found those words in the old draft-13. Why-oh-why are we spending so
much time and effort on rediscovering wheels which were invented years ago
in our earlier drafts?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21353
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Fri, 17 Jun 2005 11:36:22 GMT
Lines: 26
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>> I want USEFOR to use the term "header body" for the syntactic object,

>There are two parts to a message: the header and the (optional) body.

No, not in USEFOR as currently written. There is a discussion in progress
as to whether to change USEFOR to adhere strictly to the RFC 2822
terminology, and if that change is made then the "header body" that I have
asked for will become "header field body".

But that change has not yet been agreed, and hence I am continuing to use
the present terminology.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Fri, 17 Jun 2005 12:05:11 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Semantic & Header content (Re: #1008 USEFOR 1.5 References: Final text on "followup")
Sender: [email-masked]

--On torsdag, juni 16, 2005 12:11:50 +0000 Charles Lindsey 
<[email-masked]> wrote:

>  am not sure yet whether that bit is needed in USEPRO. I need to go
> through and see all the places where "content" is used. When I have done
> that, I may be able to simplify that definition.
>
>>> [That assumes that "header content" in USEFOR will be
>>> redefined as "header body".]
>
>> I'm on the brink to join Bruce's "hacker slang" camp if we now
>> get different terms for the same "structured header field body"
>> depending on the draft in question.
>
> I want USEFOR to use the term "header body" for the syntactic object,
> before any unfolding or other semantic simplifications. I have looked at
> all the places where USEFOR currently used "header content", and in every
> case it is the syntactic, not the semantic, object that is meant.

The term "header content" (RFC 2822 "field body") is used in 3 places in 
USEFOR, all of which are in this paragraph:

>    o  All agents MUST generate headers so that at least one space
>       immediately follows the ':' separating the header name and the
>       header contents (for compatibility with deployed software).  News
>       agents MAY accept headers which do not contain the required space.
>
>    o  The header contents of every header line (including the first and
>       any that are subsequently folded) MUST contain at least one non-
>       whitespace character.
>
>          NOTE: This means that no header content defined by or
>          referenced by this document can be empty.  As a result, this
>          document updates the <unstructured> construct from Section
>          3.2.6 of [RFC2822] as follows:
>
>    unstructured    =  1*( [FWS] utext ) [FWS]

The first instance seems to say that the space after the colon is not 
considered part of the header content.
The second one is formally wrong (you can't have "header content" of a 
"header line", since "header content" means the whole header field's 
content).
The third one implicitly uses "empty" in the sense of "something that is 
not whitespace". That's not clear either.

Since this paragraph needs some rework anyway to be totally 
self-consistent, perhaps we should free up "content" for what Charles 
wants, and define as follows:

  "field body" (as in RFC 2822) - the characters after the colon up to the 
end of the header field; we don't need to define this, since 2822 does.
The only place we need that is probably the paragraph above.

  "header field content" ("content" for short) - the stuff inside the 
header that is not leading or trailing whitespace, is not a comment, and is 
not the CRLF between header lines (we can surely make up better text for 
this!)

Does that make sense?

Date: 17 Jun 2005 08:58:00 +0200
From: [email-masked] (Kai Henningsen)
To: [email-masked]
Subject: Re: Forward compatibility (Re: CFWS in References header)
Organization: Organisation? Me?! Are you kidding?
Comment: Unsolicited commercial mail will incur an US$100 handling fee per received mail.
Sender: [email-masked]

[email-masked] (Harald Tveit Alvestrand)  wrote on 13.06.05 in <A73AD83D9BEE949094D0BFB9@B50854F0A9192E8EC6CDA126>:

> In the Beginning Of Time, certain email systems ran on IBM mainframes, and
> stored email in record-oriented files, one record per line, which had a
> hard limit of 80 characters per line (think punch-card hard).
> I believe (hope!) that all these systems are now dead.

Well, the filesystems on which these files are stored insist on records -  
there is no non-record-oriented format. The best you could do is variable  
length format, and IIRC you still need a max length ...

And while I don't know about CMS or TSO or similar mainframe systems, the  
AS/400 is alive and well and has a very similar filesystem (actually, it  
seems to be something like a thin layer above a SQL database), and I'm  
pretty certain there are people doing mail on it. When doing my small bit  
of C programming on that machine, I certainly need to cope with fixed-80  
format for C source.

And yes, that's all EBCDIC.

> A number of email readers ran by dumping their output on 80-character
> terminals, some of which had an irritating bug where, if you sent it 80

80 character screens certainly still exist.

> characters and a CRLF, the cursor would move 2 lines down, not one - making
> column 80 "dangerous territory".

Curses fortunately can handle that. However, I'm not at all sure I'd agree  
to calling that a bug - it's pretty natural behaviour if you automatically  
wrap the line at the end. You actually need to do unnatural stuff to avoid  
that.

> I hope these terminals are now disposed of in an environmentally friendly
> fashion, and no longer interesting.

I certainly doubt that, too. But see the comment about curses - the  
*problem* ought to be gone, even if the terminals aren't.

The one annoying problem left is that these terminals usually cannot write  
into the lower right corner (because doing so causes immediate scrolling).  
Sometimes, this can be done with a trick (using horizontal or vertical  
insertion, if that capability is available).

MfG Kai

Date: 17 Jun 2005 09:05:00 +0200
From: [email-masked] (Kai Henningsen)
To: [email-masked]
Subject: Re: #1004 USEFOR 3.2.14 - X-Headers text suggestion
Organization: Organisation? Me?! Are you kidding?
Comment: Unsolicited commercial mail will incur an US$100 handling fee per received mail.
Sender: [email-masked]

[email-masked] (Alexey Melnikov)  wrote on 15.06.05 in <[email-masked]>:

> One issue I should have raised earlier is about X-Mailer. X-Mailer is
> supported by majority of mail clients and is a de-facto standard for
> email. Many of those mail clients are also news clients. So I think it
> is too late (and even harmful) to deprecate X-Mailer.

I don't see why. (And I'm not even sure I believe that! User-Agent is  
certainly not an invention of this WG; it's copied from HTTP, and I expect  
(didn't look) that Mozilla and derived software (Thunderbird, for example)  
already support *that*.)

MfG Kai

Date: Thu, 16 Jun 2005 15:04:34 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

> I want USEFOR to use the term "header body" for the syntactic object,

There are two parts to a message: the header and the (optional) body. 

The header consists of header fields.

A header field consists of a field name and a field body.

That's what RFC2822 tells us. And that part of RFC2822 is hardly email 
specific.

"Header body" is joining two terms that are at the same level of
definition for two different parts of a message, and hoping that the
reader guesses it means something related to only one specific line in one
specific part of a message. 

"Header field body" is the term for the contents of the header field that 
are not the field name or separator. When we initialize an inherited 
header field in an outgoing message, we want to copy the header field 
body, typically, of the parent. We don't need a new term for it. 

If you want to keep comments out of References header field bodies, define
them without comments. End of problem.

And, by the way, this discussion is proof that discussing USEFOR and 
USEPRO separately will not work: here's a definition that is supposed to
go into USEFOR so USEPRO can use something other than RFC2822-defined 
terms.

Date: Thu, 16 Jun 2005 23:26:33 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Sender: [email-masked]

--On torsdag, juni 16, 2005 22:22:40 +0200 Frank Ellermann 
<[email-masked]> wrote:

> Henry Spencer wrote:
>
>>> This cooperating subnet nonsense really has to go.
>
>> No, it remains important.  We want the standard to be as
>> broadly applicable as we can, but that doesn't mean we have
>> to bless every stupidity that's in existing use anywhere
>
> If the USEFOR memo says "UAs MUST NOT use single component
> names", then a conforming UA cannot post in the spamcop NG,
> it cannot use Followup-To: spamcop in other spamcop.* NGs.
>
> That's wrong.  Such a "conforming" UA would be broken for a
> trivial practical task, by an arbitrary restriction.
>
> Of course I prefer a structure with NGs at the leaves of a
> tree, just beause that's as it always was.  But if it also
> works with NGs at the nodes, and obviously that's the case,
> why try to forbid it on a _syntactical_ level ?
>
> And if I had to choose between "Usenet" vs. GMaNe for some
> strange reason I'd pick GMaNe.  I want standards allowing
> UA, server, and gateway implementors to get it right.  Bye.

You seem to be making a convincing argument that the single-component 
restriction should be pushed all the way over to USEAGE.....

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date:  Thu, 16 Jun 2005 22:42:45 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 23
Sender: [email-masked]

Charles Lindsey wrote:
 
>> we also agreed to add something along the line of STD 11:
>>| Note:  THE USE OF DOMAIN-LITERALS IS STRONGLY DISCOURAGED.
 
> I think we already inherit that advice from RFC 2822 (along
> with the RECOMMENDATION to use domains).

No, we would inherit the erroneous contrary, 2822 says:

| a good method is to put the domain name (or a domain literal
| IP address) of the host on which the message identifier was
| created on the right hand side of the "@"

Domain literal is _not_ a good method.  RfC 2822 also doesn't
discourage domain literals generally like STD 11.  We have to
fix it by a clear SHOULD NOT as in STD 11.

Section 3.6.4 in 2822 is not good enough for NetNews or other
serious applications of Message-IDs like mailing list archives.

                        Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Date:  Thu, 16 Jun 2005 22:22:40 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 24
Sender: [email-masked]

Henry Spencer wrote:

>> This cooperating subnet nonsense really has to go.
 
> No, it remains important.  We want the standard to be as
> broadly applicable as we can, but that doesn't mean we have
> to bless every stupidity that's in existing use anywhere

If the USEFOR memo says "UAs MUST NOT use single component
names", then a conforming UA cannot post in the spamcop NG,
it cannot use Followup-To: spamcop in other spamcop.* NGs.

That's wrong.  Such a "conforming" UA would be broken for a
trivial practical task, by an arbitrary restriction.

Of course I prefer a structure with NGs at the leaves of a
tree, just beause that's as it always was.  But if it also 
works with NGs at the nodes, and obviously that's the case,
why try to forbid it on a _syntactical_ level ?

And if I had to choose between "Usenet" vs. GMaNe for some
strange reason I'd pick GMaNe.  I want standards allowing
UA, server, and gateway implementors to get it right.  Bye.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date:  Thu, 16 Jun 2005 21:55:25 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 22
Sender: [email-masked]

Charles Lindsey wrote:

> The words "semantically invisible" are used at several places
> in RFC 2822.

Found it, the other places are backslashes in quoted-pairs, and
ignoring these "invisible" characters in a msg-id-list is a bad
plan for the References construction and trimming algorithm.

 [2047 decoding] 
> I am not sure yet whether that bit is needed in USEPRO. I
> need to go through and see all the places where "content" is
> used. When I have done that, I may be able to simplify that
> definition.

Better stay away from "semantical content" for the References,
otherwise you'd first remove the backslashes and later reinsert
them.  With Murphy the output differs from the input, and we're
not very interested to fix a non-canonical msg-id in this case.

                       Bye, Frank

To: Usefor Mailing List <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 16 Jun 2005 10:24:57 -0700
Sender: [email-masked]

Henry Spencer <[email-masked]> writes:
> On Thu, 16 Jun 2005, Russ Allbery wrote:

>> This cooperating subnet nonsense really has to go.

> No, it remains important.  We want the standard to be as broadly
> applicable as we can, but that doesn't mean we have to bless every
> stupidity that's in existing use anywhere, just because some moron once
> thought it was a good idea.

Mail manages to avoid doing that while not introducing this cooperating
subnet concept.

It's not a crime to be noncompliant with our standard.  We don't need to
write in an escape clause that says that you're allowed to be noncompliant
if you talk to everyone in advance; that's just common sense.  The IETF
has no enforcement powers.  And in the meantime, this cooperating subnet
concept is constantly used within the WG to excuse some ignoring a huge
percentage of netnews deployments and argue for limiting the scope of our
work to the point where there would be no purpose served in continuing.

> It's perfectly reasonable to say "using a one-component newsgroup name
> for spamcop was a dumb idea and we aren't going to endorse it, but doing
> stupid things in private is okay so long as it stays private; if you get
> a chance to do it over again, do it right next time".

What part of the *protocol* does it break?

I don't see any protocol reason why the general discussion group for
Australia had to be aus.general instead of aus.  It would have been less
convenient to configure in particular news servers, INN included., but
while the convenience argument may (or may not) be a good one at a BCP
level, it's not a protocol argument.

Date: Thu, 16 Jun 2005 13:08:36 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Sender: [email-masked]

On Thu, 16 Jun 2005, Russ Allbery wrote:
> This cooperating subnet nonsense really has to go.

No, it remains important.  We want the standard to be as broadly
applicable as we can, but that doesn't mean we have to bless every
stupidity that's in existing use anywhere, just because some moron
once thought it was a good idea.

It's perfectly reasonable to say "using a one-component newsgroup name for
spamcop was a dumb idea and we aren't going to endorse it, but doing
stupid things in private is okay so long as it stays private; if you get a
chance to do it over again, do it right next time".

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 16 Jun 2005 09:46:39 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Frank Ellermann <[email-masked]> writes:

>> That's not about my UA using a "Followup-To: spamcop", right ?  I've no
>> "private arrangements" with news.spamcop.net, it's an ordinary public
>> news server, but not a part of Usenet.

> If spamcop is not a part of Usenet, then it is a cooperating subnet and
> it can do whatever it pleases.

If this standard, even the protocol pieces (!), is only intended to be
applicable to the world-wide public Usenet and not any private server or
other use of the protocols, we're all wasting our time.  A substantial
portion of the use of the Usenet protocols is outside the publically
distributed hierarchies, and those uses are frequently the most
interesting to support.  They have the same interoperability requirements
and need for a solid protocol as the public use of the Usenet article
format.

This cooperating subnet nonsense really has to go.

If you want to limit USEAGE to just the public Usenet, I don't really
care.  I've pretty much given up on USEAGE as a lost cause already.  But
please leave the protocol alone.  It applies to all use of netnews, not
just publically distributed hierarchies.

To: [email-masked]
Subject: Re: Newsgroup names #1021
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 16 Jun 2005 09:41:35 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> The distinction that I'm making here is between the syntactic
>> definition of a newsgroup name that can be broadly applied in all
>> contexts and the protocol use of a newsgroup name where particular
>> newsgroups are reserved in some contexts but not others.  The latter
>> cannot be explained without reference to the news protocol, and
>> therefore is to me clearly a protocol issue.

> Ah! You are saying that "ctl" and "all" MUST NEVER EVER appear, but "To"
> is different because it MAY appear if you are running a UUCP feed with
> the old IHAVE messages. I still think the distinction is somewhat
> artificial when it comes to deciding where to deprecate/discourage these
> things, though the exact wording used must be chosen with care.

I'm saying that "all" and "ctl" are not permissable anywhere, whereas
"to.*" (for example) is a perfectly valid thing to put into the Newsgroups
header (and in fact has to be allowed in the Newsgroups header to work at
all), just not a valid target for control messages and some other
purposes.  (And I don't think the control message restriction should be
handled in the description of the syntax of the control header; that's
really just the wrong place for such things, IMO.)

In other words, I'm looking at the distinction between the ABNF for a
newsgroup name, which has to allow such things as junk or to.* or
control.* and which is used for the Newsgroups header, the Xref header,
and so on, and the rules about what newsgroups you should use for what
purpose.  The latter feels very strongly like a USEPRO issue to me.

To: [email-masked]
Xref: clerew local.usefor:21335
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Thu, 16 Jun 2005 11:48:23 GMT
Lines: 104
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>In message <[email-masked]>, Charles Lindsey
><[email-masked]> writes

>>OK, but before doing that I want to redefine the term "header content",
>>within USEPRO as follows:
>>
>>   The "semantic content" (often abbreviated to just "content") of a
>>   structured header is its semantic interpretation (cf. use of that
>>   term in [RFC 2822]); i.e. after unfolding it, ignoring comments and
>>   other semantically invisible items, replacing white space by a single
>>   SP, and after decoding and/or recombining any <encoded-word>s or
>>   <extended-parameter>s [RFC 2231].
>>[That assumes that "header content" in USEFOR will be redefined as
>>"header body".]
>>
>>With that, I can tidy up a whole bunch of places where USEPRO currently
>>uses the term "content". 

>Accepting that a word for "the meat of what's really there" is useful;
>then it's a shame that it isn't a word that resembles the RFC2822 "field
>body" terminology;

What we actually call the new term can be adjusted when Harald has ruled
on that matter. But keeping "content" in it somewhere seems to agree with
some existing usage of that term.

> but I've no immediate suggestions that are any better
>than "semantic content"   ....  though I really don't think you need to
>explicitly appeal to 2822 for the meaning of "semantic" :)

Well RFC 2822 uses "semantic in many places, and I was just trying to show
my credentials :-) . In fact, the precise wording of that definition is
probably an overkill. E.g. the RFC 2231 stuff is not necessary for
applying it to the References header, but until I have gone through the
whole of USEPRO examining existing use of "content", I cannot be sure.

>>So, let us start with References headers:

>OK so far

>>That gives you carte blanche to refold the new header, to retain or remove
>>the old comments, and even to insert new ones.

>OK, but surely I don't need permission to fold anything, I just can.

The old version told you to take syntactic objects, such as the header
body (aka header field body, if you must) of an existing References
header, and simply to concatenate them. That buys you whatever folding and
comments were already built into those objects. So, legally speaking, you
were obliged to retain them. Yes, that law was "a ass"; which is why I
want to change it.

>>>If the total length of the resulting References header field .....
>>
>>No, you have omitted the bit about it being a limit on the unfolded length

>erm... the header field _is_ the unfolded form :)

No, "header field" as defined in RFC 2822 is the syntactic object as it
appears on the wire - folds and all.

>>   If the total length of the resulting References header after
>>   unfolding (including its header name but not the final CRLF) exceeds
>>   998 characters, it MUST be trimmed (and otherwise it MAY be trimmed).

>this wording is backwards :(  You've just created the new References
>header, so why would you have even thought about folding it yet ?

No, I have just created a References header whose *semantic*content*
contains all the right msg-ids. Who knows whether it is folded or not? Now
I want to say that you had better check that it would be less than 998 if
you were to unfold it (because some poorly designed implementation might
try to unfold it and process it further in a 998 buffer).

Note that there is a general permission in USEPRO that you don't have to
do things in the order stated, so long as the effects achieved are
indistinguishable. So a sensible implementor will probably do his counting
before folding, and indeed before attempting the concatenation.

>.... So what the specification should explain is what the
>measurement rule is...

Yes. That is what I was trying to do: "do a test unfold, and see if it is
too long". Here is another attempt:

   If the resulting References header would, after unfolding, exceed 998
   characters in length (including its header name but not the final
   CRLF), it MUST be trimmed (and otherwise it MAY be trimmed). Trimming
   involves removing any number of message identifiers from its content,
   except that the first message identifier and the last two MUST NOT be
   removed.

Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21338
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Newsgroup names #1021
Date: Thu, 16 Jun 2005 12:05:10 GMT
Lines: 25
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>The distinction that I'm making here is between the syntactic definition
>of a newsgroup name that can be broadly applied in all contexts and the
>protocol use of a newsgroup name where particular newsgroups are reserved
>in some contexts but not others.  The latter cannot be explained without
>reference to the news protocol, and therefore is to me clearly a protocol
>issue.

Ah! You are saying that "ctl" and "all" MUST NEVER EVER appear, but "To"
is different because it MAY appear if you are running a UUCP feed with the
old IHAVE messages. I still think the distinction is somewhat artificial
when it comes to deciding where to deprecate/discourage these things,
though the exact wording used must be chosen with care.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21339
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Thu, 16 Jun 2005 12:11:50 GMT
Lines: 47
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>>| The "semantic content" (often abbreviated to just "content")
>>| of a structured header is its semantic interpretation (cf.
>>| use of that term in [RFC 2822]); i.e. after unfolding it,
>>| ignoring comments and other semantically invisible items,
>>| replacing white space by a single SP

>What else is "invisible" except from replacing as much FWS as
>possible by a single SP ?

The words "semantically invisible" are used at several places in RFC 2822.

>>| and after decoding and/or recombining any <encoded-word>s
>>| or <extended-parameter>s [RFC 2231].

>Argh...  the beauty of MIME is that this decoding is normally
>unnecessary, and especially in-transit it's _never_ necessary.

I am not sure yet whether that bit is needed in USEPRO. I need to go
through and see all the places where "content" is used. When I have done
that, I may be able to simplify that definition.

>> [That assumes that "header content" in USEFOR will be
>> redefined as "header body".]

>I'm on the brink to join Bruce's "hacker slang" camp if we now
>get different terms for the same "structured header field body"
>depending on the draft in question.

I want USEFOR to use the term "header body" for the syntactic object,
before any unfolding or other semantic simplifications. I have looked at
all the places where USEFOR currently used "header content", and in every
case it is the syntactic, not the semantic, object that is meant.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21337
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Date: Thu, 16 Jun 2005 12:01:13 GMT
Lines: 29
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>That's not about my UA using a "Followup-To: spamcop", right ?
>I've no "private arrangements" with news.spamcop.net, it's an
>ordinary public news server, but not a part of Usenet.

If spamcop is not a part of Usenet, then it is a cooperating subnet and it
can do whatever it pleases.

But are you sure that it has not leaked out onto usenet? There is nothing
to stop other servers from taking it and propagating it.

I still think it better to say that single-component groups are a Bad
Thing on Usenet. But it is one of my 'category 2' things. It is stupid,
but nothing breaks. So if the category 2 stuff is included in USEFOR (as
people seem to want), then a weaker form of words can be used against it.

In any case, USEPRO will discourage/deprecate newgroup messages for it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21336
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Thu, 16 Jun 2005 11:52:43 GMT
Lines: 24
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>IIRC we also agreed to add something along the line of STD 11:
>| Note:  THE USE OF DOMAIN-LITERALS IS STRONGLY DISCOURAGED.

I think we already inherit that advice from RFC 2822 (along with the
RECOMMENDATION to use domains).

But I would have no problem with repeating the advice in USEFOR (but some
people are dead set against repeating _anything_ from RFC 2822 - not that
I am against it if if it helps make thing clearer).

I think it's up to Harald now.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Thu, 16 Jun 2005 09:12:36 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Sender: [email-masked]

--On 15. juni 2005 13:54 +0000 Charles Lindsey <[email-masked]> wrote:

>>   use and MUST NOT be generated by posting agents (whether in
>>   Newsgroups headers or in newgroup control
>>   messages [USEPRO]) without a private prior agreement to do so.
>>   However, such names MUST be accepted by
>>   relaying and serving agents.
>
> I think I would say "reserved for private arrangements between serving
> agents and the reading agents which they serve".
>
> Or it could say "reserved for read-only newsgroups, made available by
> each serving agent to those reading agents which it serves".

I don't think we have a consensus that this is the only possible scope for 
these characters - I could be wrong.
The principle of the thing is: Either we limit them to private use (in 
which case we should say NOTHING about how they are used), or we limit them 
to a specific purpose (in which case we need text describing that purpose 
in enough detail to make sure we understand it).
I'd prefer to say nothing.

> And I think they should not be accepted by relaying agents etc. So it
> should then say
>     "hence they MUST NOT appear in Newsgroups and Followup-To headers,
> though they MAY appear in Xref headers".

Relaying agents within or outside the private-agreement area?

Date: Wed, 15 Jun 2005 22:00:31 +0100
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes

>>The field body for the new article's References header field MUST be
>>constructed as follows. The field body from the parent's References
>>header field (if any) is placed first followed by CFWS. The field body
>>is completed by adding the field body from the Message-ID header field
>>of the parent. Any comments that are present within the field body MAY
>>be removed.
>
>OK, but before doing that I want to redefine the term "header content",
>within USEPRO as follows:
>
>   The "semantic content" (often abbreviated to just "content") of a
>   structured header is its semantic interpretation (cf. use of that
>   term in [RFC 2822]); i.e. after unfolding it, ignoring comments and
>   other semantically invisible items, replacing white space by a single
>   SP, and after decoding and/or recombining any <encoded-word>s or
>   <extended-parameter>s [RFC 2231].
>[That assumes that "header content" in USEFOR will be redefined as
>"header body".]
>
>With that, I can tidy up a whole bunch of places where USEPRO currently
>uses the term "content". 

Accepting that a word for "the meat of what's really there" is useful;
then it's a shame that it isn't a word that resembles the RFC2822 "field
body" terminology; but I've no immediate suggestions that are any better
than "semantic content"   ....  though I really don't think you need to
explicitly appeal to 2822 for the meaning of "semantic" :)

>So, let us start with References headers:
>
>   The (semantic) content of the new article's References header
>   consists of the content of the Message-ID header of the parent
>   preceded, if the parent had a References header, by the content of
>   that References header and a SP (subject to trimming as described
>   below).

OK so far

>That gives you carte blanche to refold the new header, to retain or remove
>the old comments, and even to insert new ones.

OK, but surely I don't need permission to fold anything, I just can.

>>>   If the resulting References header is excessively long it MAY (and if
>>>   its total length, whether folded or not, and including the header
>>>   name but not the final CRLF, exceeds 998 chacacters it MUST) then be
>>>   trimmed, but in such a manner that the first and the last two message
>>>   identifiers always remain.
>
>>If the total length of the resulting References header field (which
>>includes the 12 character header field name at the start, but not the
>>terminating CRLF) exceeds 998 octets it MUST be trimmed and it MAY be
>>trimmed at any length. Trimming involves the removal of any number of
>>message identifiers from the field body except that the first and the
>>last two message identifiers MUST NOT be removed.
>
>No, you have omitted the bit about it being a limit on the unfolded length

erm... the header field _is_ the unfolded form :)

>(because some broken agents out there try to do the whole thing in a
>single 998 character buffer).
>
>   If the total length of the resulting References header after
>   unfolding (including its header name but not the final CRLF) exceeds
>   998 characters, it MUST be trimmed (and otherwise it MAY be trimmed).

this wording is backwards :(  You've just created the new References
header, so why would you have even thought about folding it yet ? Any
implementation is bound to measure it, trim it, and only then can it
fold it as needed. So what the specification should explain is what the
measurement rule is...

You could just say that the "semantic content" mustn't exceed 986 but I
suspect you don't mean that, you mean the total length including any
comments (ugh!) but you don't want to include anything extra for when
you fold it...    As far as I can see that's just the "field body" plus
12 or in fact the header field (which includes the field name).

>   Trimming involves removing any number of message identifiers from its
>   content, except that the first message identifier and the last two
>   MUST NOT be removed.

OK with content here

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date:  Wed, 15 Jun 2005 22:57:24 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 47
Sender: [email-masked]

Charles Lindsey wrote:

> I want to redefine the term "header content", within USEPRO
> as follows:

>| The "semantic content" (often abbreviated to just "content")
>| of a structured header is its semantic interpretation (cf.
>| use of that term in [RFC 2822]); i.e. after unfolding it,
>| ignoring comments and other semantically invisible items,
>| replacing white space by a single SP

What else is "invisible" except from replacing as much FWS as
possible by a single SP ?

>| and after decoding and/or recombining any <encoded-word>s
>| or <extended-parameter>s [RFC 2231].

Argh...  the beauty of MIME is that this decoding is normally
unnecessary, and especially in-transit it's _never_ necessary.

> [That assumes that "header content" in USEFOR will be
> redefined as "header body".]

I'm on the brink to join Bruce's "hacker slang" camp if we now
get different terms for the same "structured header field body"
depending on the draft in question.

>| The (semantic) content of the new article's References
>| header consists of the content of the Message-ID header of
>| the parent preceded, if the parent had a References header,
>| by the content of that References header and a SP (subject
>| to trimming as described below).

A completely new terminology instead of simply saying "grab
old msg-id-list, add SP and parent's msg-id, then trim it as
described below" ?  Of course you can replace CFWS by SP, but
that's not a feature restricted to References, it's normal.

> you have omitted the bit about it being a limit on the
> unfolded length (because some broken agents out there try
> to do the whole thing in a single 998 character buffer).

In my case it was an injection-agent (= news server) telling
me when I have to trim excessively long references manually.

                       Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date:  Wed, 15 Jun 2005 22:03:03 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 18
Sender: [email-masked]

Charles Lindsey wrote:
 
> Frank has been flogging this for months, but appears to have
> gathered no support.

Now you did correct the broken msg-id syntax wrt to the weird
NO-WS-CTL, you also corrected the missing leading and trailing
dot issue.  

Yes, that took some months while we discussed what the msg-id
actually _is_, analyzing different versions of domain-literal.

IIRC we also agreed to add something along the line of STD 11:
| Note:  THE USE OF DOMAIN-LITERALS IS STRONGLY DISCOURAGED.

One issue where I didn't gather support were the names of the
productions.  At the moment it appears to be a tie (2:2), bye.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Date:  Wed, 15 Jun 2005 21:43:44 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 9
Sender: [email-masked]

Charles Lindsey wrote:

> "hence they MUST NOT appear in Newsgroups and Followup-To
>  headers, though they MAY appear in Xref headers".

That's not about my UA using a "Followup-To: spamcop", right ?
I've no "private arrangements" with news.spamcop.net, it's an
ordinary public news server, but not a part of Usenet.  Bye.

To: [email-masked]
Subject: Re: #1004 USEFOR 3.2.14 - X-Headers text suggestion
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 15 Jun 2005 11:12:59 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> text suggestion from me:

I like this a lot better.  Deprecating X-* headers did feel a bit odd, and
I think this will accomplish the same thing in practice.

To: [email-masked]
Subject: Re: INN implementation clarifications
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 15 Jun 2005 11:09:14 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:

> Does INN treat numeric/hexadecimal/dotted path components as IP
> addresses (e.g. comparing against the server's IP address(es) and/or
> doing reverse DNS lookups), or are they treated purely as "names"?  I.e
> in the example above (ignoring ::45af:89 since we know there's a colon
> issue), is "10.10.10.10" treated as an IP address?

INN currently does no verification of path identities whatsoever.  All
path identities are used only as opaque tokens that are compared
case-insensitively against the recorded path identity for peers to avoid
sending them articles they already got.  DNS is never used (and in fact
anything at all related to DNS is carefully avoided in all core INN code,
since DNS lookups can take minutes and INN is single-threaded).

To: [email-masked]
Subject: Re: Newsgroup names #1021
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 15 Jun 2005 11:06:53 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> The special things that happen with to.* are protocol things, not
>> format things, and should be put into the protocol document; otherwise,
>> the split that we're creating is rather meaningless.

>> "ctl" and "all" *are* USEFOR things, ...

> I don't follow the distinction you are trying to make here. They are all
> things that can cause unexpected things to happen if you try to use them
> in normal newsgroup names. It seems artificial to try to distinguish
> between them.

The distinction that I'm making here is between the syntactic definition
of a newsgroup name that can be broadly applied in all contexts and the
protocol use of a newsgroup name where particular newsgroups are reserved
in some contexts but not others.  The latter cannot be explained without
reference to the news protocol, and therefore is to me clearly a protocol
issue.

To: [email-masked]
Subject: Re: Newsgroup names #1021
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 15 Jun 2005 11:05:17 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:
>> Charles Lindsey <[email-masked]> writes:

>>> Indeed, but these are intended to be "Read Only" groups as far as
>>> Usenet is concerned. So by all means use those URI in order to read
>>> the groups, but don't expect to be able to POST to them.

>> I am fairly certain this is not the case for spamcop.

> Then it must be a "private arrengement".

Why, because you peresonally think that single-level newsgroup names
shouldn't be allowed?  I see no reason to classify this as a "private
arrangement" rather than a normal use of the news protocols.

> Indeed - hence the 'category 2' I just suggested. But is there any
> actual purpose in including "junk" in the Newsgroups header (apart from
> Trolls just showing that they can do it).

Why is this question even relevant?

> For sure, the contents of "junk" will be entirely different on each
> server, so it will hardly function as a normal group.

On servers that don't use junk for rejected articles (the vast majority of
them), the contents will likely be about as much the same as any other
poorly propagating newsgroup.

To: [email-masked]
Xref: clerew local.usefor:21319
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Newsgroup names #1021
Date: Wed, 15 Jun 2005 14:09:56 GMT
Lines: 42
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Frank Ellermann <[email-masked]> writes:

>>> There's IMHO nothing wrong (syntactically) with the news-URLs
>>>        <news://news.spamcop.net/spamcop>
>>>        <news://news.gmane.org/junk>

>> Indeed, but these are intended to be "Read Only" groups as far as Usenet
>> is concerned. So by all means use those URI in order to read the groups,
>> but don't expect to be able to POST to them.

>I am fairly certain this is not the case for spamcop.

Then it must be a "private arrengement". I think it would have been better
in that case if they had constructed a conventional 2-level hierarchy, but
if it's there, then it's there.

>> If it's a Read Only group, then its name MUST NOT appear in a Newsgroups
>> header (thogh it MAY appear in an Xref header).

>There are plenty of articles out there that have "junk" in the Newsgroups
>header.  I understand why you don't like the practice, but the fact of the
>matter is that it doesn't break the protocol.  => USEPRO or even USEAGE.

Indeed - hence the 'category 2' I just suggested. But is there any actual
purpose in including "junk" in the Newsgroups header (apart from Trolls
just showing that they can do it). For sure, the contents of "junk" will
be entirely different on each server, so it will hardly function as a
normal group.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21315
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1004 USEFOR 3.2.14 - X-Headers text suggestion
Date: Wed, 15 Jun 2005 12:12:53 GMT
Lines: 74
Sender: [email-masked]

In <[email-masked]> Alexey Melnikov <[email-masked]> writes:

>Harald Tveit Alvestrand wrote:

>> text suggestion from me:
>>
>> OLD (3.2.13):
>>
>>      NOTE: This header supersedes the role performed redundantly by
>>      experimental headers such as X-Newsreader, X-Mailer, X-Posting-
>>      Agent, X-Http-User-Agent, and other headers previously used on
>>      Usenet and in Email for this purpose.  Use of these experimental
>>      headers SHOULD be discontinued in favor of the single, standard
>>      User-Agent header.
>>
>> NEW:
>>
>>      NOTE: Some of this information has previously been sent in
>>      non-standardized headers such as X-Newsreader, X-Mailer, X-Posting-
>>      Agent, X-Http-User-Agent, and others.
>>      Once an agent uses User-Agent, it should have no need to
>>      send these non-standard headers.

I think I prefer the original wording; or if not, something in between.
What we are trying to bring about is that User-Agent will become the norm,
and that usage of the others will fade away. If "SHOULD be discontinued"
is considered to be too strong, then something like "it is
intended/expected that ...", or "continued use of ... is discouraged". And
the word "redundant(ly)" in the original could usefully be retained, as
providing motivation for the intention/expectation/discouragement.

>One issue I should have raised earlier is about X-Mailer. X-Mailer is 
>supported by majority of mail clients and is a de-facto standard for 
>email. Many of those mail clients are also news clients. So I think it 
>is too late (and even harmful) to deprecate X-Mailer.

Again, I think the intent is that it will fade away (note that one could
never standardize it in that form). More and more user agents are using
User-Agent (though with some divergence as regards its exact syntax, which
our draft should clarify). And when you say X-Mailer is "supported",
surely what you mean is that it is "generated". I don't think much
software actually looks at it, apart from ad hoc scripts, which are going
to have to look at User-Agent as well. Indeed, the whole point of
encouraging those old headers to fade away is that eventually the writers
of ad hoc scripts will just need to look at User-Agent.

>> OLD (3.2.14):
>>
>>   This header is intended to replace various currently-used but
>>   undocumented headers such as "NNTP-Posting-Host", "X-Trace" and
>>   "X-Complaints-To".  These headers are thus deprecated.
>>
>> NEW:
>>
>>      NOTE: Some of this information has previously been sent in
>>      non-standardized headers such as NNTP-Posting-Host, X-trace,
>>      X-Complaints-To, and others.
>>      Once an injecting agent uses Injection-Info, it should have no 
>> need to
>>      send these non-standard headers.

Same remarks as for User-Agent. I prefer the Old, and if not that then
something a bit stronger than the New.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21317
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Date: Wed, 15 Jun 2005 13:54:52 GMT
Lines: 134
Sender: [email-masked]

In <A3144E13755E7D3CACB6EA26@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>After checking all the mail on the #1021 thread, a suggestion...

>Suggestion for a resolution in 3.1.5:

>Let the ABNF be "rather liberal". In fact use the existing ABNF.
>Let the text explain the various restrictions.

Yes, I like that.

>Alternatively, one could introduce the lowercase restriction using the 
>"obs-" convention, like this:

No, leave it as it is, and let the verbiage do the talking.

>And restrict them like this:

>  A newsgroup component SHOULD NOT consist of digits only, and
>  SHOULD NOT contain uppercase letters. Such components MAY be
>  used only to refer to existing groups that do not conform to this
>  naming scheme.

Some motivation would be helpful:
   (because of bad interactions with a widely deployed storage technique)
   (because some existing agents treat them case insensitievly)
and perhaps
   ... to existing groups created prior to this standard.

I shall make it clear in USEPRO that this rule is to be enforced in
newgroup control messages.

>The second part is the restriction (if any) on + - _.
>I suggest (in order to be concrete; I'm not sure it's right):

>   Components beginning with underline ("_") are reserved for use by
>   future versions of this standard and MUST NOT be generated by
>   posting agents (whether in Newsgroups headers or in newgroup control
>   messages [USEPRO]).  However, such names MUST be accepted by
>   relaying and serving agents.

Actually, the "(whether in Newsgroups headers ...)" could be omitted
without loss. Again, I will ensure that USEPRO keeps them out of control
messages.

As worded, it seems that injecting agents MAY/SHOULD/MUST reject them.
Perhaps that last sentence is a USEPRO matter, and it needs simple
"However, such names MUST be accepted by other agents" (modulo query about
injecting agents).

>   Components beginning with "+" and "-" ("_") are reserved for private
                                          ^^^^^
					   Eh?
>   use and MUST NOT be generated by posting agents (whether in
>   Newsgroups headers or in newgroup control
>   messages [USEPRO]) without a private prior agreement to do so.
>   However, such names MUST be accepted by
>   relaying and serving agents.

I think I would say "reserved for private arrangements between serving
agents and the reading agents which they serve".

Or it could say "reserved for read-only newsgroups, made available by
each serving agent to those reading agents which it serves".

And I think they should not be accepted by relaying agents etc. So it
should then say
    "hence they MUST NOT appear in Newsgroups and Followup-To headers,
though they MAY appear in Xref headers".

>Suggestions for modifications are in order....

>The third part is the restriction on component names. I've gathered the 
>following from the discussion (the below is a suggestion for text):

>The following newsgroup names are reserved, and MUST NOT be used as
>the name of a newsgroup:

>  Groups starting with the component "example"
>  The group "poster"

>The following newsgroup names have been used for specific purposes in 
>various implementations and protocols, and MUST therefore not be used for 
>the names of normal newsgroups. They MAY be used for their specific 
>purpose, or by local agreement.

>  Groups with one component
>  Groups starting with "junk"
                             ^
			     .
>  Groups starting with "to"
                           ^
			   .
>  Groups containing the component "all"
>  Groups containing the component "ctl"
>  The group "control"
>  The group "junk"

Add "Groups starting with "control." And I would omit groups of the form
"junk.*" (they are mighty peculiar, but do not actually conflict).

There seems to be some consensus that all these should be mentioned in
USEFOR. However, they could be split into two categories:

Ones that cause real harm:
   to.* *.all.* *.ctl.* poster

and ones that are just plain stupid
   example.* junk control control.*

Russ seems to want the second category to be relegated to USEFOR, but I
think it is better to mention them all here, even if the prohibitions
against them are somewhat milder.

>NOTE: Some newsgroups violating these restrictions exist. It is not a 
>protocol violation to use these names to access those newsgroups.

That applies particularly to the second category (but their existence is
restricted to particular servers). Indeed, the treatment of
the second category should be more like I have suggested for the "+" and
"-" ones.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21321
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Forward compatibility (Re: CFWS in References header)
Date: Wed, 15 Jun 2005 14:26:43 GMT
Lines: 37
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>I guarantee that software breaks if you fold the path header.  Software
>will break with anything other than host names separated by !.  Now, we
>may decide that we don't care about a lot of that software (such as ad-hoc
>parsing scripts), and major news software should not break, but the Path
>header format *in practice*, and therefore what much software has been
>written to during this long interregnum without an effective Usenet
>standard, is significantly more restrictive than what RFC 1036 says.

It all the major news software does not break, then I could live with
problems with a few ad hoc scripts.

Two things are needed.

1. The folded Path header must propagate without bits of it getting lost.
That is a problem for relaying and serving agents to get right.

2. When a relaying agent reads through the Path, to avoid sending an
article where it has already been sent before, it must search the whole of
the Path. That also is a problem for relaying and serving agents to get
right.

If existing relaying agents in fact satisy both of those, then there is no
problem. Moreover, one can tolerate occasional failure of #2, because it
causes nothing to break beyond a bit of redundant transmission.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21320
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Newsgroup names #1021
Date: Wed, 15 Jun 2005 14:17:45 GMT
Lines: 25
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>The special things that happen with to.* are protocol things, not format
>things, and should be put into the protocol document; otherwise, the split
>that we're creating is rather meaningless.

>"ctl" and "all" *are* USEFOR things, ...

I don't follow the distinction you are trying to make here. They are all
things that can cause unexpected things to happen if you try to use them
in normal newsgroup names. It seems artificial to try to distinguish
between them.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21318
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Date: Wed, 15 Jun 2005 14:03:33 GMT
Lines: 34
Sender: [email-masked]

>> And restrict them like this:

>>   A newsgroup component SHOULD NOT consist of digits only, and
>>   SHOULD NOT contain uppercase letters. Such components MAY be
>>   used only to refer to existing groups that do not conform to this
>>   naming scheme.

>I like this.  I would add "A newsgroup component SHOULD NOT be the string
>"all" or "ctl", for compatibility with older news software" or something
>along those lines, since the all and ctl thing is historically a format
>consideration, not a protocol consideration.

But isn't that covered in Harald's list of forbidden things further down
(though I would agree that some motivation should be given for each
forbidden item)?

And I am not sure the format/protocol distinction means much here. These
things have been excluded from the format because they would cause the
protocols to screw up in various ways.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21316
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date: Wed, 15 Jun 2005 13:11:08 GMT
Lines: 32
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Harald Tveit Alvestrand wrote:

>> I'm going to resolve the ticket as "just replace mqtext with
>> *mqtext".

>> OK?

Fine with me.

>There's a dispute about the meaning of <id-right>.  I say it's
>in fact an <id-domain>, a proper name space with an owner.

Frank has been flogging this for months, but appears to have gathered no
support.

RFC 2822 RECOMMENDS that a domain should be used as the <id-right>, and
this is indeed good practice. But once the <id-right> has been generated,
it is a domain no longer, but just a string of characters to be compared
for equality, or copied (as into a References header).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Russ Allbery <[email-masked]>
Subject: Re: INN implementation clarifications
Date: Wed, 15 Jun 2005 12:11:05 -0400
Cc: [email-masked]
Sender: [email-masked]

Russ, thanks for the clarifications.  One more:

On Thu June 9 2005 17:59, Russ Allbery wrote:

> INN considers [a-zA-Z0-9._-] to be valid hostname characters in the Path
> header and all other characters are treated as delimiters.  So in other
> words, the following path header:
> 
>     Path: f-oo.com%  bar_bar.com*10.10.10.10![::45af:89]=not-for-mail
> 
> is treated by INN as completely identical to:
> 
>     Path: f-oo.com!bar_bar.com!10.10.10.10!45af!89!not-for-mail

Does INN treat numeric/hexadecimal/dotted path components as IP
addresses (e.g. comparing against the server's IP address(es) and/or
doing reverse DNS lookups), or are they treated purely as "names"?
I.e in the example above (ignoring ::45af:89 since we know there's a
colon issue), is "10.10.10.10" treated as an IP address?

Date: Wed, 15 Jun 2005 10:35:46 -0400
From: Harald Tveit Alvestrand <[email-masked]>
To: "Forrest J. Cavalier III" <[email-masked]>, [email-masked]
Subject: Tickets for USEPRO (Re: Ticket system mechanics)
Sender: [email-masked]

After exchanging mail with Alexey on the subject, we have agreed that he'll 
take on the task of entering tickets for USEPRO, while I continue entering 
tickets for USEFOR.

We'll continue to follow the <document> <section> naming, so all USEFOR 
tickets will start with USEFOR, while all USEPRO tickets will start with 
USEPRO.

A request from the ticket enterer.....
If you want to raise a new issue, and want to make sure it gets a ticket, 
please use a subject line saying something like

NEW: USEPRO 3.14.159 Section numbering is irrational

This will make the job of entering tickets easier.....

              Harald

--On 13. juni 2005 17:14 -0400 Harald Tveit Alvestrand 
<[email-masked]> wrote:

>> Why are you not entering tickets for USEPRO?  The point of a ticket
>> system is to record issues so that work on them can occur at time other
>> than when they are raised.  Delaying they entry of tickets is completely
>> wrong.
>
> Limited time on my hands. I have a hard time just keeping up with the
> USEFOR issues.
> More people entering tickets (and tracking followup on tickets) would
> help, if we could avoid stumbling over each others' feet.

Date: Wed, 15 Jun 2005 08:21:58 -0400
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: Guaranteed MUSTard
Sender: [email-masked]

--On 14. juni 2005 20:00 +0000 Charles Lindsey <[email-masked]> wrote:

> No, that is not the meaning of that SHOULD in RFC 2822. They say 'reply
> messages SHOULD have "In-Reply-To:" and "References:' fields. I don't
> think it ever occurred to them that someone might want to use References
> in messages that were not replies. They do not forbid it, but they give no
> guidance on how to do it.

Ned was there at the time; he wrote the text about References in MIME 
message/partial. I don't think we should assume he had forgotten it by that 
time......

                   Harald

Date: Wed, 15 Jun 2005 12:33:50 +0100
From: Alexey Melnikov <[email-masked]>
To: Harald Tveit Alvestrand <[email-masked]>
CC: [email-masked]
Subject: Re: #1004 USEFOR 3.2.14 - X-Headers text suggestion
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> --On 15. juni 2005 00:24 +0100 Alexey Melnikov 
> <[email-masked]> wrote:
>
>>>      NOTE: Some of this information has previously been sent in
>>>      non-standardized headers such as X-Newsreader, X-Mailer, 
>>> X-Posting-
>>>      Agent, X-Http-User-Agent, and others.
>>>      Once an agent uses User-Agent, it should have no need to
>>>      send these non-standard headers.
>>
>> One issue I should have raised earlier is about X-Mailer. X-Mailer is
>> supported by majority of mail clients and is a de-facto standard for
>> email. Many of those mail clients are also news clients. So I think 
>> it is
>> too late (and even harmful) to deprecate X-Mailer.
>
> We can't standardize it (due to the X- rule; see John Klensin's appeal 
> on the LEMONADE MMS document on the IETF list for details).
> So it's going to remain undefined with regard to format, handling 
> rules and so on. And messages without it are going to continue to be 
> acceptable, no matter what.
>
> So we've got 2 options:
>
> - Keep it on the list of "don't see why you'd continue to send it"
> - Not mention it at all

Considering the choices I would prefer not mentioning it.

> I don't want to say anything that seems like *recommending* sending
>
> User-agent: foobar/1.2
> X-Mailer: foobar/1.2
>
> That's just useless header bloat in my opinion. But just my opinion.

Date: Wed, 15 Jun 2005 07:13:11 -0400
From: Harald Tveit Alvestrand <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: [email-masked]
Subject: Re: #1004 USEFOR 3.2.14 - X-Headers text suggestion
Sender: [email-masked]

--On 15. juni 2005 00:24 +0100 Alexey Melnikov 
<[email-masked]> wrote:

>>      NOTE: Some of this information has previously been sent in
>>      non-standardized headers such as X-Newsreader, X-Mailer, X-Posting-
>>      Agent, X-Http-User-Agent, and others.
>>      Once an agent uses User-Agent, it should have no need to
>>      send these non-standard headers.
>
> One issue I should have raised earlier is about X-Mailer. X-Mailer is
> supported by majority of mail clients and is a de-facto standard for
> email. Many of those mail clients are also news clients. So I think it is
> too late (and even harmful) to deprecate X-Mailer.

We can't standardize it (due to the X- rule; see John Klensin's appeal on 
the LEMONADE MMS document on the IETF list for details).
So it's going to remain undefined with regard to format, handling rules and 
so on. And messages without it are going to continue to be acceptable, no 
matter what.

So we've got 2 options:

- Keep it on the list of "don't see why you'd continue to send it"
- Not mention it at all

I don't want to say anything that seems like *recommending* sending

User-agent: foobar/1.2
X-Mailer: foobar/1.2

That's just useless header bloat in my opinion. But just my opinion.

Date: Wed, 15 Jun 2005 00:24:54 +0100
From: Alexey Melnikov <[email-masked]>
To: Harald Tveit Alvestrand <[email-masked]>
CC: [email-masked]
Subject: Re: #1004 USEFOR 3.2.14 - X-Headers text suggestion
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> Ticket text and comments so far:
>
>> From Frank Ellermann:
>>
>> Is deprecating X- headers allowed ?
>>
>> From Charles Lindsey, May 26:
>>
>>> Is deprecating X- headers allowed ?
>>
>> I don't see why not, if you are providing a standard header to fulfil
>> the same role.
>
>> From Frank Ellermann:
>>
>> Examples could include "X-NoArchive: yes, (c) 2005 by xyzzy"
>> or "X-ComplaintsTo: not me". The "same role" is unclear for
>> X- header fields.
>>
>> Not an important point, but if it helps to avoid a warning
>> from Bruce's review tool I'd just delete the last statement
>> "These headers are thus deprecated." - the intended effect of
>> the complainto-param is obvious without it.
>
>
> text suggestion from me:
>
> OLD (3.2.13):
>
>      NOTE: This header supersedes the role performed redundantly by
>      experimental headers such as X-Newsreader, X-Mailer, X-Posting-
>      Agent, X-Http-User-Agent, and other headers previously used on
>      Usenet and in Email for this purpose.  Use of these experimental
>      headers SHOULD be discontinued in favor of the single, standard
>      User-Agent header.
>
> NEW:
>
>      NOTE: Some of this information has previously been sent in
>      non-standardized headers such as X-Newsreader, X-Mailer, X-Posting-
>      Agent, X-Http-User-Agent, and others.
>      Once an agent uses User-Agent, it should have no need to
>      send these non-standard headers.

One issue I should have raised earlier is about X-Mailer. X-Mailer is 
supported by majority of mail clients and is a de-facto standard for 
email. Many of those mail clients are also news clients. So I think it 
is too late (and even harmful) to deprecate X-Mailer.

> OLD (3.2.14):
>
>   This header is intended to replace various currently-used but
>   undocumented headers such as "NNTP-Posting-Host", "X-Trace" and
>   "X-Complaints-To".  These headers are thus deprecated.
>
> NEW:
>
>      NOTE: Some of this information has previously been sent in
>      non-standardized headers such as NNTP-Posting-Host, X-trace,
>      X-Complaints-To, and others.
>      Once an injecting agent uses Injection-Info, it should have no 
> need to
>      send these non-standard headers.
>
> Makes sense?

The rest looks good to me.

Date: Tue, 14 Jun 2005 15:37:59 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Guaranteed MUSTard
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>And 4 - we MUST have them in followups; 2822 says SHOULD have them in
>replies.

USEFOR lists no such difference. The Chair has closed discussion of this,
IIRC; nobody has yet shown an interoperability issue justifying RFC2119
language so putting in a MUST would be wrong, and thus there is no
difference number 4.

To: [email-masked]
Subject: Re: Forward compatibility (Re: CFWS in References header)
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 14 Jun 2005 14:37:00 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> We haven't fixed that yet?  Consider the same objections lodged as to
>> folding Newsgroups, but with considerably more vehemence.  Consider,
>> for instance, the behavior of:

>>    Control: newgroup example.some.really.long.newsgroup.name
>>      moderated

>> in different existing implementations.

> We discussed this recently, and it was agreed AIUI to replace CFWS by
> FWS in the Control header.

It needs to be WS if you want to have a prayer of interoperating with
existing software.  Not even the latest development version of INN can
cope with a folded Control header.

> You may well be right about regular use, but certainly I have seen
> folded Paths propagate in previous experiments, and I had always assumed
> this was OK because it seems clear from RFC 1036 that the Path header
> may be folded.

I guarantee that software breaks if you fold the path header.  Software
will break with anything other than host names separated by !.  Now, we
may decide that we don't care about a lot of that software (such as ad-hoc
parsing scripts), and major news software should not break, but the Path
header format *in practice*, and therefore what much software has been
written to during this long interregnum without an effective Usenet
standard, is significantly more restrictive than what RFC 1036 says.

To: [email-masked]
Xref: clerew local.usefor:21295
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Tue, 14 Jun 2005 18:10:29 GMT
Lines: 48
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>Charles Lindsey wrote:

>> In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:
>>>5. What should an implementor of a user Agent make of the 7.6
>>>language regarding the References header?  What can that implementor
>>>assume about articles created by agents that pre-date the draft?  Or
>>>is there a flag day?  What can they assume about a multi-part FAQ?
>> 
>> 
>> There is no need for any flag day. There is nothing that an implementor
>> using the References header has to do in consequence of our draft that is
>> in any way different from what he was doing beforehand.
>> 

>These documents will obsolete previous standards. They must be a complete
>specification, suitable for implementation of all software which processes
>articles.

Yes, and there _is_ a complete specification, because USEFOR tells you the
syntax of the References header, and USEPRO tells you how to construct it.
So long as implementors can assume it has been constructed in that way,
implementors (of User Agents, in particular) can go ahead and use it in
any way they see fit. The corresponding header in RFC 2822 was defined in
much the same way, and implementors have managed to figure out how to use
that one.

Now I will grant you that what is said is minimal, and it is said in the
wrong place. Will you join me in asking the Chair to raise a ticket to put
some reasonable semantics back into USEFOR?

>"Nothing....in any way different from what he was doing beforehand"
>is not a specification, much less a complete specification.

That sentence was not intended as a spedification. It was a simple
statement of fact.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21296
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Tue, 14 Jun 2005 19:39:54 GMT
Lines: 113
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>In message <[email-masked]>, Charles Lindsey
><[email-masked]> writes

>>>     It would be helpful to make that crystal clear. I can suggest some
>>>     words, if that's necessary....

>looks as if it might be :(

>I've tried to use header field etc as they should be used,

No, our Chair has not ruled on whether to make that change or not. Let us
deal with one issue at once.

>>   If the parent did not have a References header, the content of the
>>   new article's References header MUST be inherited from that of the
>>   Message-ID header of the parent, otherwise (there already was a
>>   References header) it MUST be comprised of the parent's References
>>   header followed by CFWS and the Message-ID header content of the
>>   parent (subject to trimming as described below).

>The field body for the new article's References header field MUST be
>constructed as follows. The field body from the parent's References
>header field (if any) is placed first followed by CFWS. The field body
>is completed by adding the field body from the Message-ID header field
>of the parent. Any comments that are present within the field body MAY
>be removed.

OK, but before doing that I want to redefine the term "header content",
within USEPRO as follows:

   The "semantic content" (often abbreviated to just "content") of a
   structured header is its semantic interpretation (cf. use of that
   term in [RFC 2822]); i.e. after unfolding it, ignoring comments and
   other semantically invisible items, replacing white space by a single
   SP, and after decoding and/or recombining any <encoded-word>s or
   <extended-parameter>s [RFC 2231].
[That assumes that "header content" in USEFOR will be redefined as
"header body".]

With that, I can tidy up a whole bunch of places where USEPRO currently
uses the term "content". So, let us start with References headers:

   The (semantic) content of the new article's References header
   consists of the content of the Message-ID header of the parent
   preceded, if the parent had a References header, by the content of
   that References header and a SP (subject to trimming as described
   below).

That gives you carte blanche to refold the new header, to retain or remove
the old comments, and even to insert new ones.

>>   If the resulting References header is excessively long it MAY (and if
>>   its total length, whether folded or not, and including the header
>>   name but not the final CRLF, exceeds 998 chacacters it MUST) then be
>>   trimmed, but in such a manner that the first and the last two message
>>   identifiers always remain.

>If the total length of the resulting References header field (which
>includes the 12 character header field name at the start, but not the
>terminating CRLF) exceeds 998 octets it MUST be trimmed and it MAY be
>trimmed at any length. Trimming involves the removal of any number of
>message identifiers from the field body except that the first and the
>last two message identifiers MUST NOT be removed.

No, you have omitted the bit about it being a limit on the unfolded length
(because some broken agents out there try to do the whole thing in a
single 998 character buffer).

   If the total length of the resulting References header after
   unfolding (including its header name but not the final CRLF) exceeds
   998 characters, it MUST be trimmed (and otherwise it MAY be trimmed).
   Trimming involves removing any number of message identifiers from its
   content, except that the first message identifier and the last two
   MUST NOT be removed.

OK, I expect all that still needs a little polishing.

>>>>The essential property of the
>>>>        References header, guaranteed by the procedure above and to be
>>>>        preserved in any future extension, is that no article can ever
>>>>        precede one of its own parents.
>>

>I would prefer it not to be said at all, except perhaps as a general
>hand-waving comment

Well, being in a NOTE, it was _meant_ to be hand waving ...

> to say that the intent of the References header
>field and the rules on trimming are to make it easy to assign articles
>to the correct thread and to place them in an appropriate tree-like
>structure provided that propagation is not too awful. Of course, we
>would add, there may be mangling going on, so programs should take steps
>not to be caught out by nonsense.

And I would like to say all those things too, but in USEFOR. See my reply
to Forrest, and let us see where that leads to.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21294
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Forward compatibility (Re: CFWS in References header)
Date: Tue, 14 Jun 2005 17:02:55 GMT
Lines: 43
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> Control _can_ be folded (it has FWS in it).

>We haven't fixed that yet?  Consider the same objections lodged as to
>folding Newsgroups, but with considerably more vehemence.  Consider, for
>instance, the behavior of:

>    Control: newgroup example.some.really.long.newsgroup.name
>      moderated

>in different existing implementations.

We discussed this recently, and it was agreed AIUI to replace CFWS by FWS
in the Control header.

>> I believe Paths are regularly folded in the Real Usenet.

>I think you're confusing what was written into this draft years ago with
>what actually happens out on the wire where no one is paying attention to
>these drafts.  :)

>Since it happens to work with at least some major servers, a few people
>may fold Path headers just for grins, but it is not in my experience an
>event that could be described with the term "regularly."

You may well be right about regular use, but certainly I have seen folded
Paths propagate in previous experiments, and I had always assumed this was
OK because it seems clear from RFC 1036 that the Path header may be
folded.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21297
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Tue, 14 Jun 2005 19:42:38 GMT
Lines: 29
Sender: [email-masked]

In <[email-masked]> Seth Breidbart <[email-masked]> writes:

>"Ruud H.G. van Tol" <[email-masked]> wrote:
>>
>> I still can read that as if the first two and the last two message
>> identifiers (so four) must remain.

Ah! Now I see the ambiguity.

>To avoid that possible (incorrect, but possible) ambiguity, how about
>wording it as "trimmed, but in such a manner that the first and the
>two last message identifiers always remain."?  (I would think that
>"first and two last" is harder to misinterpret than "first and last
>two" due to the lack of possible parallelism.)

What I have actually put now is

"... that the first message identifier and the last two always remain."

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21298
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Guaranteed MUSTard
Date: Tue, 14 Jun 2005 20:00:51 GMT
Lines: 70
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> if you are generating a mail reply, you use the rules given
>> in RFC 2822, not the rules I have written for USEPRO (though
>> they are near enough the same).

>Followup-To: poster is an issue for news.

USEPRO tells you to email it in that case. It is not the job of our drafts
to tell you how to email things.

>    [...] in the course of generating a followup.

>Delete "or for some other reason" plus examples, because it's
>unrelated to Followup agents.

No, that section applies to *all* References headers, not just to
followups.

>The prose about a "Mail-Copies-To" in usepro-03 (7.6) is also
>obsolete,

Yes, it is gone now.

>> if we do things differently than in RFC 2822, then this needs
>> to be pointed out

>The three differences:  1 - we trim, they don't.  2 - they have
>an In-Reply-To, we don't.  3 - they say [CFWS], we say CFWS.

And 4 - we MUST have them in followups; 2822 says SHOULD have them in
replies.

>The fourth point, the worst references could be anything, is no
>difference, we have it now in the definition, they have it as a
>SHOULD (not a MUST, so obviously other references might exist).

No, that is not the meaning of that SHOULD in RFC 2822. They say 'reply
messages SHOULD have "In-Reply-To:" and "References:' fields. I don't
think it ever occurred to them that someone might want to use References
in messages that were not replies. They do not forbid it, but they give no
guidance on how to do it.

>> If you follow the procedure given, then that is the result
>> that you will get.

>No, it is not.  The given procedure gets it right for the LAST
>msg-id added to the references.  It cannot guarantee anything
>about OLDER msg-ids in the msg-id-list.  Among other problems
>there's no guarantee that the FIRST msg-id properly identifies
>the start of the thread.

If the OLDER msg-ids were not created by the proper procedure, then they
were garbage already. If _every_ msg-id in the header was put there by
some previous agent using the procedure I gave, then the result is indeed
guaranteed.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 14 Jun 2005 15:23:46 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Sender: [email-masked]

On Tue, 14 Jun 2005, Harald Tveit Alvestrand wrote:
> Alternatively, one could introduce the lowercase restriction using the 
> "obs-" convention...  And restrict them like this:
>   A newsgroup component SHOULD NOT consist of digits only, and
>   SHOULD NOT contain uppercase letters. Such components MAY be
>   used only to refer to existing groups that do not conform to this
>   naming scheme.

This sounds like a good idea to me.

> The following newsgroup names are reserved, and MUST NOT be used...
> The following newsgroup names have been used for specific purposes in 
> various implementations and protocols, and MUST therefore not be used for 
> the names of normal newsgroups...
> NOTE: Some newsgroups violating these restrictions exist. It is not a 
> protocol violation to use these names to access those newsgroups.

And that also sounds good.

> Suggestions for edits, or parts of this that belong in USEPRO not USEFOR?

Put it all in USEFOR.  As per previous posting, putting it in one place is
*much* better than arbitrarily subdividing it.

                                                          Henry Spencer
                                                       [email-masked]

Date: Tue, 14 Jun 2005 15:13:07 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Newsgroup names #1021
Sender: [email-masked]

On Tue, 14 Jun 2005, Seth Breidbart wrote:
> Deprecate them all in one place (even if some of them are "officially"
> deprecated there, and the others are mentioned in a NOTE as being
> deprecated elsewhere).  Failing to do so will confuse implementors.
> 
> Anyplace that ANY newsgroup names are deprecated MUST mention (at
> least by pointer) ALL deprecated newsgroup names.

I concur.  It is very important that all restrictions and prohibitions be
together, at least logically (i.e., after following cross-reference
pointers).  If the discussion is split, each and every part of it MUST
cross-reference all other parts, no exceptions.

Keeping it all together in one place would make the most sense.  The
format/protocol split is supposed to be *helpful*, not a religious law
which must be followed no matter how much it hurts readability. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
Subject: Re: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 14 Jun 2005 11:15:00 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> Suggestion for a resolution in 3.1.5:

> Let the ABNF be "rather liberal". In fact use the existing ABNF.
> Let the text explain the various restrictions.

> Alternatively, one could introduce the lowercase restriction using the
> "obs-" convention, like this:

>    component-char = %x61-7A ; lowercase letters
>                   / DIGIT / "+" / "-" / "_" / obs-component-char
>    obs-component-char = %x41-5A ; uppercase letters

> And restrict them like this:

>   A newsgroup component SHOULD NOT consist of digits only, and
>   SHOULD NOT contain uppercase letters. Such components MAY be
>   used only to refer to existing groups that do not conform to this
>   naming scheme.

I like this.  I would add "A newsgroup component SHOULD NOT be the string
"all" or "ctl", for compatibility with older news software" or something
along those lines, since the all and ctl thing is historically a format
consideration, not a protocol consideration.  .all is equivalent to .* in
that news software and is treated as a wildcard, so it actually has
different textual semantics, and posting to a group ending in .ctl was a
different way of designating control messages.  Here's the relevant text
from RFC 1036:

    Wildcards (e.g., the word "all") are never allowed in a "News-
    groups" line.  For example, a newsgroup comp.all is illegal,
    although a newsgroup rec.sport.football is permitted.

    For upward compatibility, messages that match the newsgroup pattern
    "all.all.ctl" should also be interpreted as control messages.  If no
    "Control" header is present on such messages, the subject is used as
    the control message.  However, messages on newsgroups matching this
    pattern do not conform to this standard.

> The second part is the restriction (if any) on + - _.  I suggest (in
> order to be concrete; I'm not sure it's right):

>    Components beginning with underline ("_") are reserved for use by
>    future versions of this standard and MUST NOT be generated by
>    posting agents (whether in Newsgroups headers or in newgroup control
>    messages [USEPRO]).  However, such names MUST be accepted by
>    relaying and serving agents.

>    Components beginning with "+" and "-" ("_") are reserved for private
>    use and MUST NOT be generated by posting agents (whether in
>    Newsgroups headers or in newgroup control
>    messages [USEPRO]) without a private prior agreement to do so.
>    However, such names MUST be accepted by
>    relaying and serving agents.

Works for me.

> The following newsgroup names are reserved, and MUST NOT be used as
> the name of a newsgroup:

>   Groups starting with the component "example"
>   The group "poster"

Right.

> The following newsgroup names have been used for specific purposes in
> various implementations and protocols, and MUST therefore not be used
> for the names of normal newsgroups. They MAY be used for their specific
> purpose, or by local agreement.

>   Groups with one component
>   Groups starting with "junk"

Hm, I'm not sure what to do with junk.foo.  It's actually a normal
newsgroup with no special properties, just a horribly confusing name
that's likely to run afoul of common newsfeed patterns.  To USEAGE, maybe?

>   Groups starting with "to"

Including the group "to".

>   Groups containing the component "all"
>   Groups containing the component "ctl"
>   The group "control"

And also groups starting with "control".  The whole hierarchy is reserved
for special purposes.

>   The group "junk"

> NOTE: Some newsgroups violating these restrictions exist. It is not a
> protocol violation to use these names to access those newsgroups.

Right.

> Suggestions for edits, or parts of this that belong in USEPRO not
> USEFOR?

I would take everything other than the uppercase, digits, all, ctl,
"poster", and special character notes and move it all to USEPRO, except
for junk.* which could probably go to USEAGE (although I wouldn't care too
much if it ended up in USEFOR).  USEPRO can then also include the
explanations of what junk is for, what control.* is for, how to.* is used,
etc.  There's enough material for a whole section on special newsgroup
names.

To: [email-masked]
Subject: Re: Newsgroup names #1021
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 14 Jun 2005 11:08:15 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> I would presume that any INN site will have refused to create that
> group, and any others like it.

INN will refuse to honor a control message for it with the default
controlchan configuration (which can be changed).  If the newsgroup were
created manually with ctlinnd, INN would have no problems with it.

INN assigns no special meaning to "all" or "ctl" name components (with one
exception) and treats such groups like any other group internally; the
only place they're handled separately is in auditing newsgroup names in
control messages.  The one exception is that if a control message (with a
Control header) is posted to a group whose name ends in example.foo.ctl,
it is treated as if it were posted to example.foo for propagation
purposes.

To: [email-masked]
Subject: Re: Newsgroup names #1021
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 14 Jun 2005 11:05:00 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Whilst "junk", "control.*", "example.*", plus the ones with conponents
> starting with "+" or "-" (if we retain that distinction) are reserved to
> special purposes, and might be dealt with in USEPRO, I think "poster"
> and "to.*" need to be put alongside "ctl" and "all" in USEFOR, since
> some very funny things can happen if people try to post articles into
> them.

The special things that happen with to.* are protocol things, not format
things, and should be put into the protocol document; otherwise, the split
that we're creating is rather meaningless.

"ctl" and "all" *are* USEFOR things, but I think it may be an open
question at this point whether we really care to continue to cater to the
obsolete wildcard format of C News, given that apparently some people have
started to ignore this with alt.* groups that are propagating.  Maybe it's
time to move that restriction to a SHOULD NOT instead of a MUST NOT with a
note that the restriction is likely to be removed entirely in a future
revision of the standard.

> My view is that it is simpler to deprecate them all in one place, but if
> the WG or our Chair wants them split between USEFOR and USEPRO in that
> manner, then it could be done. More opinions?

I think it should be split.

> The current wording in USEFOR is:

>    Components beginning with underline ("_") are reserved for use by
>    future versions of this standard and MUST NOT occur in <newsgroup-
>    name>s (whether in Newsgroups headers or in newgroup control messages
>    [USEPRO]).  However, such names MUST be accepted.

> Is that OK? It seems to allow a new standard to say "OK, you can now
> start to generate them with the following meaning".

I like Harald's wording better.

To: [email-masked]
Subject: Re: Newsgroup names #1021
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 14 Jun 2005 11:02:11 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Frank Ellermann <[email-masked]> writes:

>> There's IMHO nothing wrong (syntactically) with the news-URLs
>>        <news://news.spamcop.net/spamcop>
>>        <news://news.gmane.org/junk>

> Indeed, but these are intended to be "Read Only" groups as far as Usenet
> is concerned. So by all means use those URI in order to read the groups,
> but don't expect to be able to POST to them.

I am fairly certain this is not the case for spamcop.

> If it's a Read Only group, then its name MUST NOT appear in a Newsgroups
> header (thogh it MAY appear in an Xref header).

There are plenty of articles out there that have "junk" in the Newsgroups
header.  I understand why you don't like the practice, but the fact of the
matter is that it doesn't break the protocol.  => USEPRO or even USEAGE.

Date: Tue, 14 Jun 2005 12:54:57 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: Newsgroup names #1021
Sender: [email-masked]

"Charles Lindsey" <[email-masked]> wrote:

> Whilst "junk", "control.*", "example.*", plus the ones with conponents
> starting with "+" or "-" (if we retain that distinction) are reserved to
> special purposes, and might be dealt with in USEPRO, I think "poster" and
> "to.*" need to be put alongside "ctl" and "all" in USEFOR, since some very
> funny things can happen if people try to post articles into them.
>
> My view is that it is simpler to deprecate them all in one place, but if
> the WG or our Chair wants them split between USEFOR and USEPRO in that
> manner, then it could be done. More opinions?

Deprecate them all in one place (even if some of them are "officially"
deprecated there, and the others are mentioned in a NOTE as being
deprecated elsewhere).  Failing to do so will confuse implementors.

Anyplace that ANY newsgroup names are deprecated MUST mention (at
least by pointer) ALL deprecated newsgroup names.

Seth

To: [email-masked]
Xref: clerew local.usefor:21290
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Newsgroup names #1021
Date: Tue, 14 Jun 2005 13:43:51 GMT
Lines: 76
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> we concluded that a ban on uppercase was the safest option.

>Please propose ABNF if that's about USEFOR-05.

   component       =  1*component-char ; but containing at least
                                       ; one non-DIGIT

   component-char  =  %x61-7a / DIGIT / "+" / "-" / "_"
                                       ; uppercase letters excluded

>> perhaps a NOTE to the effect that some existing groups with
>> UC names (and also some with all-digit components) DO exist
>> on Usenet

     NOTE: the above syntax excludes uppercase letters (because of
     uncertainties as to whether they will be treated case insensitively
     by existing software) and all-digit components (because this can
     cause difficulties for a widely deployed storage technique). The fact
     that a small number of newsgroups have been formed on Usenet in
     violation of these restrictions should not be seen as legitimizing
     the creation of further non-compliant newsgroups.

>Depending on what we pick our news-nntp-uri drafts have to be
>updated.  At the moment ALPHA as in 1738 appears to be wrong.

No, those drafts (well, mine at least) allows the wider set of
newsgroup-names that the latest NNTP draft allows (which deliberately
leaves it to the news format standards to specify a more restricted set
for actual use).

So you are entitled to ask for <news://news.example.net/alt.2600> and, in
that particular case, you will likely get it.

>>| o Newsgroup-names having only one component.  These are
>>|   reserved for newsgroups whose propagation is restricted to
>>|   a single host or local network

>There's IMHO nothing wrong (syntactically) with the news-URLs
>        <news://news.spamcop.net/spamcop>
>        <news://news.gmane.org/junk>

Indeed, but these are intended to be "Read Only" groups as far as Usenet
is concerned. So by all means use those URI in order to read the groups,
but don't expect to be able to POST to them.

>So that MUST NOT is not what I'd expect in the ABNF.  Maybe it
>is a bad idea like fancy References, but not a syntax problem.

If it's a Read Only group, then its name MUST NOT appear in a Newsgroups
header (thogh it MAY appear in an Xref header).

>Some prose should cover it, I tend to SHOULD NOT for the class.
>In theory a newsgroup "poster" is possible.  Utter dubious and
>most probably a very bad idea, but not really MUST NOT harmful.
>Or is it harmful ?

I think it probably is. Suppose an article has
   Newsgroups: poster,misc.test
and you want to followup to the "poster" group only :-( .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21289
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Newsgroup names #1021
Date: Tue, 14 Jun 2005 12:14:26 GMT
Lines: 61
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Here is the relevant text from draft-13, which could usefully be copied
>> into USEFOR more-or-less as it stands:

>Hm, that sure looks mostly like USEPRO material to me.

>USEFOR should deal with what newsgroup names are *syntactically* valid and
>discuss newsgroup names that are *always* forbidden.  If the newsgroup
>name is just reserved for special purposes, that's exactly the sort of
>thing that USEPRO is for.  Given that reasoning, the only ones of these
>that I would put into USEFOR are:

>>      o Any newsgroup-name containing the component "ctl" (likewise);
>>      o Any newsgroup-name containing the component "all" (because this
>>        is used as a wildcard in some implementations).

Whilst "junk", "control.*", "example.*", plus the ones with conponents
starting with "+" or "-" (if we retain that distinction) are reserved to
special purposes, and might be dealt with in USEPRO, I think "poster" and
"to.*" need to be put alongside "ctl" and "all" in USEFOR, since some very
funny things can happen if people try to post articles into them.

My view is that it is simpler to deprecate them all in one place, but if
the WG or our Chair wants them split between USEFOR and USEPRO in that
manner, then it could be done. More opinions?

>> There was also provision for reserved components begining with "_" for
>> "future extensions". It is not so clear whether those should be
>> prohibited from propagating.

>If we're going to reserve one of these characters for future
>standardization purposes, it needs to propagate.  All of the reasons for
>reserving a group name that I can think of would need them to propagate
>(such as using it for punycode).

The current wording in USEFOR is:

   Components beginning with underline ("_") are reserved for use by
   future versions of this standard and MUST NOT occur in <newsgroup-
   name>s (whether in Newsgroups headers or in newgroup control messages
   [USEPRO]).  However, such names MUST be accepted.

Is that OK? It seems to allow a new standard to say "OK, you can now start
to generate them with the following meaning".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21291
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Newsgroup names #1021
Date: Tue, 14 Jun 2005 13:49:59 GMT
Lines: 24
Sender: [email-masked]

In <[email-masked]> "Ruud H.G. van Tol" <[email-masked]> writes:

>No doubt old news, but I see several in my newsgroups-list.
>Most in alt.*
>Other example: sol.lists.freebsd.cvs.all.digest

Which is a pity, because alt.* is one hierarchy where you may want to be
extremely picky as to what groups you allow for in your sys file, but in
CNews you could not do so if you wanted to distinguish between similar
groups with and without "all" in them.

I would presume that any INN site will have refused to create that group,
and any others like it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 14 Jun 2005 09:33:50 -0400
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1021: USEFOR 3.1.5 Newsgroups - suggested texts
Sender: [email-masked]

After checking all the mail on the #1021 thread, a suggestion...

- a number of restrictions are Good Ideas on newsgroup names.
  Particularly:
  - No all-digit components
  - No to., all, junk, test, example et al
  - Lowercase letters only
  - No leading + - _ (until assigned standard meaning)
  - No single-component newsgroup names

- however, a number of these constructs are *necessary* in order
  to access existing newsgroups such as alt.2600,
  sol.lists.freebsd.cvs.all.digest, "junk" and so on

Suggestion for a resolution in 3.1.5:

Let the ABNF be "rather liberal". In fact use the existing ABNF.
Let the text explain the various restrictions.

Alternatively, one could introduce the lowercase restriction using the 
"obs-" convention, like this:

   component-char = %x61-7A ; lowercase letters
                  / DIGIT / "+" / "-" / "_" / obs-component-char
   obs-component-char = %x41-5A ; uppercase letters

And restrict them like this:

  A newsgroup component SHOULD NOT consist of digits only, and
  SHOULD NOT contain uppercase letters. Such components MAY be
  used only to refer to existing groups that do not conform to this
  naming scheme.

The second part is the restriction (if any) on + - _.
I suggest (in order to be concrete; I'm not sure it's right):

   Components beginning with underline ("_") are reserved for use by
   future versions of this standard and MUST NOT be generated by
   posting agents (whether in Newsgroups headers or in newgroup control
   messages [USEPRO]).  However, such names MUST be accepted by
   relaying and serving agents.

   Components beginning with "+" and "-" ("_") are reserved for private
   use and MUST NOT be generated by posting agents (whether in
   Newsgroups headers or in newgroup control
   messages [USEPRO]) without a private prior agreement to do so.
   However, such names MUST be accepted by
   relaying and serving agents.

Suggestions for modifications are in order....

The third part is the restriction on component names. I've gathered the 
following from the discussion (the below is a suggestion for text):

The following newsgroup names are reserved, and MUST NOT be used as
the name of a newsgroup:

  Groups starting with the component "example"
  The group "poster"

The following newsgroup names have been used for specific purposes in 
various implementations and protocols, and MUST therefore not be used for 
the names of normal newsgroups. They MAY be used for their specific 
purpose, or by local agreement.

  Groups with one component
  Groups starting with "junk"
  Groups starting with "to"
  Groups containing the component "all"
  Groups containing the component "ctl"
  The group "control"
  The group "junk"

NOTE: Some newsgroups violating these restrictions exist. It is not a 
protocol violation to use these names to access those newsgroups.
------------------------------------------------------------

Suggestions for edits, or parts of this that belong in USEPRO not USEFOR?

                      Harald

Date: Tue, 14 Jun 2005 15:22:24 +0200
From: Sebastian Brocks <[email-masked]>
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

Frank Ellermann schrieb:

> There's a dispute about the meaning of <id-right>.  I say it's
> in fact an <id-domain>, a proper name space with an owner.

(...)

> Maybe somebody else has
> an opinion.

I agree with your position.

greetings, Sebastian

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Date:  Tue, 14 Jun 2005 15:05:29 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 40
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> I'm going to resolve the ticket as "just replace mqtext with
> *mqtext".

> OK?

There's a dispute about the meaning of <id-right>.  I say it's
in fact an <id-domain>, a proper name space with an owner.

TTBOMK Charles says, whatever it is, in a msg-id it's case
sensitive.  These positions aren't incompatible, but somewhat
different.  I'm obsessed with the (2)822 semantics, Charles is
more interested in a visual similarity to RfC 2822 <id-right>.

Message-IDs are at the very core of Netnews.  More than three
opinions about it for a clearer rough consensus might help.
Henry and Russ refuse to discuss it.  Maybe somebody else has
an opinion.
                         Bye, Frank

msg-id       =  "<" id-local "@" id-domain ">"

id-local     = dot-atom-text / ( DQUOTE id-quote DQUOTE )
id-quote     = ( "." [id-text] ) /
               ( [id-text] "." ) /
               ( [id-text] id-special [id-text] )

id-text      = 1*( atext / "." / id-special )
id-special   = "(" / ")" / "," /       ; all specials, minus ">",
               "[" / "]" / "@" /       ; minus DQUOTE, minus "\",
               ":" / ";" / "<" /       ; minus single ".", plus:
               ".." / "\\" / ( "\" DQUOTE )

id-domain    = dot-atom-text / ("[" id-literal "]")
id-literal   = 1*( %d33-61 /           ; printable ASCII minus
                   %d63-90 /           ; ">", "[", "\", "]"
                   %d94-126 /          ; plus "\[", "\\, "\]"
                   "\[" / "\\" / "\]" )

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1002 USEFOR 6. References - propose to accept and leave to editor to write
Date:  Tue, 14 Jun 2005 14:47:31 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
 
> I suggest we close this ticket and expect to see the new
> references in -05.
 
> OK?

For these references nits you could keep #1002 open until it's
actually fixed in -05.  Sometimes closing tickets because all
agree has no visible effect in the next draft, if it was only
a typo or here some old RfCs - at least I tend to forget such
mostly irrelevant details until I see them again.  Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1004 USEFOR 3.2.14 - X-Headers text suggestion
Date:  Tue, 14 Jun 2005 14:11:10 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 6
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> Makes sense?

Yes, good riddance of this X-header-zoo => #1004 resolved.

Date: Tue, 14 Jun 2005 07:52:56 -0400
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1004 USEFOR 3.2.14 - X-Headers text suggestion
Sender: [email-masked]

Ticket text and comments so far:

> From Frank Ellermann:
>
> Is deprecating X- headers allowed ?
>
> From Charles Lindsey, May 26:
>
>> Is deprecating X- headers allowed ?
>
> I don't see why not, if you are providing a standard header to fulfil
> the same role.

> From Frank Ellermann:
>
> Examples could include "X-NoArchive: yes, (c) 2005 by xyzzy"
> or "X-ComplaintsTo: not me". The "same role" is unclear for
> X- header fields.
>
> Not an important point, but if it helps to avoid a warning
> from Bruce's review tool I'd just delete the last statement
> "These headers are thus deprecated." - the intended effect of
> the complainto-param is obvious without it.

text suggestion from me:

OLD (3.2.13):

      NOTE: This header supersedes the role performed redundantly by
      experimental headers such as X-Newsreader, X-Mailer, X-Posting-
      Agent, X-Http-User-Agent, and other headers previously used on
      Usenet and in Email for this purpose.  Use of these experimental
      headers SHOULD be discontinued in favor of the single, standard
      User-Agent header.

NEW:

      NOTE: Some of this information has previously been sent in
      non-standardized headers such as X-Newsreader, X-Mailer, X-Posting-
      Agent, X-Http-User-Agent, and others.
      Once an agent uses User-Agent, it should have no need to
      send these non-standard headers.

OLD (3.2.14):

   This header is intended to replace various currently-used but
   undocumented headers such as "NNTP-Posting-Host", "X-Trace" and
   "X-Complaints-To".  These headers are thus deprecated.

NEW:

      NOTE: Some of this information has previously been sent in
      non-standardized headers such as NNTP-Posting-Host, X-trace,
      X-Complaints-To, and others.
      Once an injecting agent uses Injection-Info, it should have no need to
      send these non-standard headers.

Makes sense?

Date: Tue, 14 Jun 2005 07:32:23 -0400
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1003 USEFOR 3.1.3 - Resolution of msg-id issues
Sender: [email-masked]

Ticket #1003 says:

> From Frank Ellermann:
>
>| no-fold-quote = DQUOTE
>| ( "." mqtext /
>| *mqtext "." /
>| *mqtext mqspecial *mqtext )
>| DQUOTE
>
> s/mqtext/*mqtext/
>
> The msg-id = id-left "@" id-right style of syntax is ugly and
> doesn't indicate the intended semantics id-local "@" id-domain
>
> Overloading 2822 no-fold-quote by something else is confusing.
> a simple ( DQUOTE id-quote DQuOTE ) is clearer.
>
> Overloading 2822 no-fold-literal by something else is ugly, a
> simple id-literal is clearer
>
> The terms mqtext, mqspecial, and mdtext are cryptic (mq means
> "something in msg-id no-fold-quote", the d in md probably means
> "domain-literal").
>
> id-text, id-special, and id-literal are clearer (stuff needed
> only in a msg-id, no separate no-fold-literal vs. mdtext). See
> also <http://mid.gmane.org/[email-masked]>

I recorded 2 comments - from Charles Lindsey and Frank Ellermann - 
indicating that the replacement mqtext/*mqtext was correct, but on the 
naming issue Charles said:

> you have raised all these before, but nobody supported you
> and our Chair said to leave it be.

If nobody else wishes to reopen the issue of the names of the productions, 
I'm going to resolve the ticket as "just replace mqtext with *mqtext".

OK?

Date: Tue, 14 Jun 2005 07:14:27 -0400
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1002 USEFOR 6. References - propose to accept and leave to editor to write
Sender: [email-masked]

The text of ticket #1002 is the following:

>From Frank Ellermann:

2234 => 2234bis
2366 => 3851
---------------
2373 => 3513 (missing in the references)

3513 has no IPv4address / IPv6address ABNF, therefore let's
replace 2373 by 3986 (STD 66).

If Ken is clear on what needs to be done, I suggest we close this ticket 
and expect to see the new references in -05.

OK?

               Harald

Date: Mon, 13 Jun 2005 17:14:45 -0400
From: Harald Tveit Alvestrand <[email-masked]>
To: "Forrest J. Cavalier III" <[email-masked]>, [email-masked]
Subject: Ticket system mechanics (Re: References semantics (Re: #1008 USEFOR 1.5 References: Final text on "followup"))
Sender: [email-masked]

--On 13. juni 2005 08:26 -0400 "Forrest J. Cavalier III" 
<[email-masked]> wrote:

> Why are you not entering tickets for USEPRO?  The point of a ticket
> system is to record issues so that work on them can occur at time other
> than when they are raised.  Delaying they entry of tickets is completely
> wrong.

Limited time on my hands. I have a hard time just keeping up with the 
USEFOR issues.
More people entering tickets (and tracking followup on tickets) would help, 
if we could avoid stumbling over each others' feet.

> If you don't want to discuss USEPRO now, and you don't want to enter
> tickets,
> then I think it is appropriate to convince Charles that the documents must
> be considered independently, and then instruct him not to post USEPRO text
> until you will allow discussion.

So far, I'm unconvinced of the need for the strong linkage that Charles 
seems to assume is needed; since he's holding the pen on USEPRO, I can 
understand that he tends to see things in terms of USEPRO changes, but I 
haven't seen a case yet where people (apart from Charles) argued one way or 
another about what was needed in USEFOR based on the USEPRO changes.

                   Harald

Date: Mon, 13 Jun 2005 17:09:54 -0400
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: #1031 Re: standard terminology and text
Sender: [email-masked]

--On 13. juni 2005 11:54 +0000 Charles Lindsey <[email-masked]> wrote:

>> (I'm somewhat unsure what a "header line" means - it isn't listed in the
>> definitions, and is not used in 2822. I'm assuming that it's a line (as
>> defined in RFC 2822) that occurs in the header (as defined in RFC 2822))
>
> It is used variously in various places (sometimes when speaking of the
> individual lines after a header is folded). Therefore, it SHOULD NOT be
> used, unless the context makes it absolutely clear what is intended.

I propose that we define "header line" explicitly as the part of a header 
(field) that starts at the start of a line and ends with CRLF.
That's the interpretation that seems consistent with its usage in USEFOR.

Date: Mon, 13 Jun 2005 17:07:19 -0400
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: Forward compatibility (Re: CFWS in References header)
Sender: [email-masked]

--On 13. juni 2005 14:42 +0000 Charles Lindsey <[email-masked]> wrote:

>> Surely this is a quality of implementation issue.
>
> Then why does RFC 2822 make such an issue of it, giving encouragement to
> fold everything within 78 characters (rather than 998), presumablt because
> that's how user agents prefer it.

In the Beginning Of Time, certain email systems ran on IBM mainframes, and 
stored email in record-oriented files, one record per line, which had a 
hard limit of 80 characters per line (think punch-card hard).
I believe (hope!) that all these systems are now dead.

A number of email readers ran by dumping their output on 80-character 
terminals, some of which had an irritating bug where, if you sent it 80 
characters and a CRLF, the cursor would move 2 lines down, not one - making 
column 80 "dangerous territory".
I hope these terminals are now disposed of in an environmentally friendly 
fashion, and no longer interesting.

When variable-width textboxes became available, non-folding headers 
sometimes caused a long scrollbar at the bottom of the header-display box. 
Uncomfortable.
(This happens also if you pull up a message with a long header in text 
format inside a text editor, btw)
Thus, a number of people grew to like everything folding within 80 
characters, because it gave a consistent display when you could guarantee 
that you had 80 characters available.

Summary: It used to be an interoperability issue. Nowadays I hope it's just 
an user preference issue.
(note on terminology: in my parlance, user agents are software; they 
execute, and work or fail to work. users are the ones who have 
preferences....)

                    Harald

Date: Mon, 13 Jun 2005 16:59:00 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

"Ruud H.G. van Tol" <[email-masked]> wrote:
> Charles Lindsey schreef:

>> trimmed, but in such a manner that the first and the
>> last two message identifiers always remain.
>
> I still can read that as if the first two and the last two message
> identifiers (so four) must remain.

To avoid that possible (incorrect, but possible) ambiguity, how about
wording it as "trimmed, but in such a manner that the first and the
two last message identifiers always remain."?  (I would think that
"first and two last" is harder to misinterpret than "first and last
two" due to the lack of possible parallelism.)

Seth

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Mon, 13 Jun 2005 22:48:34 +0200
Organization: Chaos rules.
Sender: [email-masked]

Charles Lindsey schreef:

> trimmed, but in such a manner that the first and the
> last two message identifiers always remain.

I still can read that as if the first two and the last two message
identifiers (so four) must remain. Of course it is better that they do,
because that enhances the possibility that one of those articles is
still known and/or received. Maybe just remove the 'two'.

-- 
Grtz, Ruud

Date: Mon, 13 Jun 2005 21:21:17 +0100
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes

>>>7.6.1.  Construction of the References header
>>>
>>>   If the resulting References header is excessively long it MAY (and if
>>>   its total length, whether folded or not, exceeds 998 chacacters it
>>>   MUST) then be trimmed, but the first and the last two message
>>>   identifiers MUST NOT be removed.

[snip]

>>     It would be helpful to make that crystal clear. I can suggest some
>>     words, if that's necessary....

looks as if it might be :(

I've tried to use header field etc as they should be used, but have not
checked if the other jargon I've chucked in is entirely consistent. That
should however be reasonably simple to adjust

>   If the parent did not have a References header, the content of the
>   new article's References header MUST be inherited from that of the
>   Message-ID header of the parent, otherwise (there already was a
>   References header) it MUST be comprised of the parent's References
>   header followed by CFWS and the Message-ID header content of the
>   parent (subject to trimming as described below).

The field body for the new article's References header field MUST be
constructed as follows. The field body from the parent's References
header field (if any) is placed first followed by CFWS. The field body
is completed by adding the field body from the Message-ID header field
of the parent. Any comments that are present within the field body MAY
be removed.

>   If the resulting References header is excessively long it MAY (and if
>   its total length, whether folded or not, and including the header
>   name but not the final CRLF, exceeds 998 chacacters it MUST) then be
>   trimmed, but in such a manner that the first and the last two message
>   identifiers always remain.

If the total length of the resulting References header field (which
includes the 12 character header field name at the start, but not the
terminating CRLF) exceeds 998 octets it MUST be trimmed and it MAY be
trimmed at any length. Trimming involves the removal of any number of
message identifiers from the field body except that the first and the
last two message identifiers MUST NOT be removed.

>I am not sure that it really better (and it is certainly longer).

well I propose my wording as more direct (and addressing the issues that
I raised, as well as the comments issue someone else mentioned)

>>>The essential property of the
>>>        References header, guaranteed by the procedure above and to be
>>>        preserved in any future extension, is that no article can ever
>>>        precede one of its own parents.
>
>>nope :(   if you're given rubbish (eg an article with existing circular
>>references) then this statement suggests that you have an obligation to
>>fix it up. I don't think that's a reasonable requirement. I suggest that
>>since this is merely meant to be a helpful deduction then it is either
>>toned down or removed.
>
>If all References headers are constructed according to the preceding
>paragraphs, then circular references cannot arise. 

correct, but not the point I was making

>If, by some accident,
>one does arise, then "garbage in, garbage out" applies as usual.

that also wasn't the point I was making :(

I was suggesting that implementors who realised that there was a problem
might feel that the note obligated them to arrange that what went out
was not garbage (as would sometimes be possible). Clearly that's not the
intent, so we should arrange that it is clear.

>But that paragpraph is just a NOTE, so it has no normative effect (and I
>would prefer it to have been said in USEFOR - see my reply to Forrest).

I would prefer it not to be said at all, except perhaps as a general
hand-waving comment to say that the intent of the References header
field and the rules on trimming are to make it easy to assign articles
to the correct thread and to place them in an appropriate tree-like
structure provided that propagation is not too awful. Of course, we
would add, there may be mangling going on, so programs should take steps
not to be caught out by nonsense.

- -- 
richard @ highwayman . com                       "Nothing seems the same
                          Still you never see the change from day to day
                                And no-one notices the customs slip away"

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Subject: Re: Newsgroup names #1021
Date: Mon, 13 Jun 2005 22:15:22 +0200
Organization: Chaos rules.
Sender: [email-masked]

Charles Lindsey schreef:

>      o Any newsgroup-name containing the component "all" (because this
>        is used as a wildcard in some implementations).

No doubt old news, but I see several in my newsgroups-list.
Most in alt.*
Other example: sol.lists.freebsd.cvs.all.digest

-- 
Grtz, Ruud

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Newsgroup names #1021
Date:  Mon, 13 Jun 2005 19:55:21 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 52
Sender: [email-masked]

Charles Lindsey wrote:

> we concluded that a ban on uppercase was the safest option.

Please propose ABNF if that's about USEFOR-05.

> perhaps a NOTE to the effect that some existing groups with
> UC names (and also some with all-digit components) DO exist
> on Usenet

There are apparently three options for the US-ASCII letters:

      ALPHA                ; dubious with NNTP 27
      %x41-5A              ; conflict with RfC 1738
      %x41-5A / %61-7A     ; CAVEAT, case sensitive

Depending on what we pick our news-nntp-uri drafts have to be
updated.  At the moment ALPHA as in 1738 appears to be wrong.

>| The following forms of newsgroup-name MUST NOT be used
>| except for the specific purposes indicated:

>| o Newsgroup-names having only one component.  These are
>|   reserved for newsgroups whose propagation is restricted to
>|   a single host or local network

There's IMHO nothing wrong (syntactically) with the news-URLs
        <news://news.spamcop.net/spamcop>
        <news://news.gmane.org/junk>

So that MUST NOT is not what I'd expect in the ABNF.  Maybe it
is a bad idea like fancy References, but not a syntax problem.

Some prose should cover it, I tend to SHOULD NOT for the class.
In theory a newsgroup "poster" is possible.  Utter dubious and
most probably a very bad idea, but not really MUST NOT harmful.
Or is it harmful ?

> MUST NOT appear in Newsgroups and Followup-To headers; MAY
> appear in Xref headers.

I don't see a technical problem with "junk" in the Newsgroups.

 [only digits]
> I agree this prohibition should remain

Corresponding ABNF proposed elsewhere, essentially two options:
At least one non-digit or (as in s-o-1036) at least one letter.

                           Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Guaranteed MUSTard
Date:  Mon, 13 Jun 2005 20:35:03 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 51
Sender: [email-masked]

Charles Lindsey wrote:

> if you are generating a mail reply, you use the rules given
> in RFC 2822, not the rules I have written for USEPRO (though
> they are near enough the same).

Followup-To: poster is an issue for news.  If you think that
2822 is good enough, then 2046 is also good enough for partial,
and we get this short statement in USEPRO 7.6:

    [...] in the course of generating a followup.

Delete "or for some other reason" plus examples, because it's
unrelated to Followup agents.

The prose about a "Mail-Copies-To" in usepro-03 (7.6) is also
obsolete,

> if we do things differently than in RFC 2822, then this needs
> to be pointed out

The three differences:  1 - we trim, they don't.  2 - they have
an In-Reply-To, we don't.  3 - they say [CFWS], we say CFWS.

The fourth point, the worst references could be anything, is no
difference, we have it now in the definition, they have it as a
SHOULD (not a MUST, so obviously other references might exist).

> If you follow the procedure given, then that is the result
> that you will get.

No, it is not.  The given procedure gets it right for the LAST
msg-id added to the references.  It cannot guarantee anything
about OLDER msg-ids in the msg-id-list.  Among other problems
there's no guarantee that the FIRST msg-id properly identifies
the start of the thread.

> Of course, I cannot "guarantee" what will happen it you don't
> follow the procedure.

Therefore I recommend to remove this false "guarantee" before
it could confuse implementors.

> Garbage in -> garbage out.

Exactly.  Garbage in -> trimmed garbage with the precusor out.
No guarantees, no graph theory, just a msg-id-list as defined
in USEFOR.
                        Bye, Frank

Date: Mon, 13 Jun 2005 10:42:45 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Newsgroup names #1021 
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>John, how much does it take to make you see that the door you're trying to 
>bash in is already open?

Your response is condescending and insulting.

>The fact that this issue made it onto the ticket system without the list 
>being notified is an error;

That's right. And that is the first time I've seen you say that. So your
statement above about "how much does it take" is insulting. It implies 
you've been saying this all along and I've just not "gotten it".

>it is the list that is the WG.

Which list? The trouble ticket list, or this group? Since issues are being
raised on the trouble ticket list and discussion is taking place there,
I have to assume you mean "the trouble ticket list".

>Because this system is run as a voluntary gift to the IETF by Randy Bush, 
>and he thinks the entire idea of trusting "root certificates" just because 
>they got shipped with your browser is ludicrous.
>I can see his point.

We aren't talking about trusting root certificates, we are talking about
accepting SSL certificates that are self-certified for a service that has
NO need for certs in the first place. That begs the question of why you
think it is appropriate to run an off-WG discussion list just because it
was a gift. 

>Take it up with Randy. I wish you luck.

No, I take it up with YOU because YOU are the Chair of this WG and YOU are 
responsible for the creation and use of the off-WG discussion forum where 
"off-list" issues are being raised and discussed. Randy has nothing to do 
with this; he's not the one forcing these invalid certs down our throats 
if we want to participate in the discussion of issues relevant to this WG.
You are.

>John, I'll be impolite for once.

You've already been impolite. Don't pretend otherwise.

>This group has demonstrated conclusively that just having discussions on 
>the list does not lead to the group finishing.

Irrelevant. Either this working group is the USEFOR working group or it is 
not. IETF says it is, but the chair of this group is running another 
discussion forum that is handling USEFOR issues, so there is some 
question.

>A trouble ticket system is one means of helping a group close.

Like I already said, if it were just "a trouble ticket system" where you 
were creating and even publishing a list of problem points that needed to 
be discussed, that's fine. But this is NOT just "a trouble ticket system", 
it is a full-blown alternate discussion forum where issues are being taken 
directly without coming through here first. 

That is not acceptable. "We cannot finish in this group" is not an excuse.

>Your behaviour is part of the reason the group is not finishing.

Not agreeing with the "consensus" when the consensus changes from week to
week is hardly the cause of the problem.

>You didn't get the point that "6." referred to section 6 in the document, 
>did you?

Of course I understood that "6" referred to section 6 of a document. Stop 
being insulting. YOU didn't get the point that "No discussion, ticket 
seems clear" is demonstrating that the "trouble ticket system" is being 
used to bypass the discussion that IS taking place in THIS group. 

>Feel free to leave, John.

I'll not hold my breath waiting for you to censur yourself like you did 
Bruce. This WG is working on more than one standard, both literally and 
figuratively.

To: [email-masked]
Subject: Re: Forward compatibility (Re: CFWS in References header)
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 13 Jun 2005 09:37:04 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:
>> Charles Lindsey <[email-masked]> writes:

>>> IMO, Newsgroups folding would be very useful. Newsgroups headers in
>>> excess of 80 characters are very common. They display badly in many
>>> user agents; e.g. there is just a '!' to indicate "rest of line
>>> truncated", or they are are wrapped arbitrarily in the middle of a
>>> newsgroup (or between newsgroups if you are lucky).

>> Surely this is a quality of implementation issue.

> Then why does RFC 2822 make such an issue of it, giving encouragement to
> fold everything within 78 characters (rather than 998), presumablt
> because that's how user agents prefer it.

Because, all other things being equal, it's better to fold stuff.  As RFC
822 allowed folding all over the place, all other things are equal.  All
other things are most decidedly not equal for Usenet, which has never
allowed folding in some headers.

> Generally speaking, I have a great dislike if what user agents show me
> has been munged beyond ehat was actually on the wire.

That's really not a good reason to require in the protocol that articles
be pre-formatted for your personal preferences.  :)

> Control _can_ be folded (it has FWS in it).

We haven't fixed that yet?  Consider the same objections lodged as to
folding Newsgroups, but with considerably more vehemence.  Consider, for
instance, the behavior of:

    Control: newgroup example.some.really.long.newsgroup.name
      moderated

in different existing implementations.

> I believe Paths are regularly folded in the Real Usenet.

I think you're confusing what was written into this draft years ago with
what actually happens out on the wire where no one is paying attention to
these drafts.  :)

Since it happens to work with at least some major servers, a few people
may fold Path headers just for grins, but it is not in my experience an
event that could be described with the term "regularly."

To: [email-masked]
Xref: clerew local.usefor:21272
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Newsgroups folding test
Date: Mon, 13 Jun 2005 14:18:38 GMT
Lines: 35
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> OK, let's try it.

>I don't find these exercises useful.  I don't think they really prove what
>you're trying to prove.

It wasn't meant to _prove_ anything. It was intended to find out how well
such articles would propagate at the present time.

Not as well as I would have expected, it seems, but since it got as far as
Google it certainly travelled some considerable distance from where I sent
it. But it would be nice to see some Paths.

>  Propagation only measures whether or not the vast
>majority of servers reject it; if only a smaller number of servers reject
>the message, it could still be a significant problem in practice but won't
>show on this sort of a test.

What I am curious to know is whether some servers decided not to store it
even after they had received it (or whether they stored it in "junk").
Looking in server logs might be instructive.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21273
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Guaranteed MUSTard (was: #1008 USEFOR 1.5 References: Final text on "followup")
Date: Mon, 13 Jun 2005 14:28:40 GMT
Lines: 60
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> in the course of generating a followup or for some other
>> reason (e.g. the later parts of a multipart posting such
>> as a FAQ, or the later parts of a message/partial as
>> suggested in [RFC 2046])

>I'd remove these irrelevant examples where other algorithms
>might also work, the most important "other reason" is a reply,
>not limited to Followup-To: poster.  So that would be:

>   in the course of generating a followup or for some other
>   reason (e.g. a mail reply).

No, if you are generating a mail reply, you use the rules given in RFC
2822, not the rules I have written for USEPRO (though they are near enough
the same).

>> NOTE: There is no provision in this standard for an article
>> to have more than one parent.

>Then don't talk about it, or add the In-Reply-To blurb here:

RFC 2822 makes a comparable remark.

And if we do things differently than in RFC 2822, then this needs to be
pointed out (and since this is a protocol difference, not a format
difference, USEPRO is the proper place to do it).

>> The essential property of the References header, guaranteed
>> by the procedure above and to be preserved in any future
>> extension, is that no article can ever precede one of its
>> own parents.

>There is no "guarantee", this is bad advice for implementors.

If you follow the procedure given, then that is the result that you will
get. Of course, I cannot "guarantee" what will happen it you don't follow
the procedure.

>My UA cannot handle self-referencing articles, and Harald has
>reported another case, where this "guarantee" simply crashed.

No, somebody had evidently not followed the procedure. Garbage in ->
garbage out.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21274
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Forward compatibility (Re: CFWS in References header)
Date: Mon, 13 Jun 2005 14:42:54 GMT
Lines: 71
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> IMO, Newsgroups folding would be very useful. Newsgroups headers in
>> excess of 80 characters are very common. They display badly in many user
>> agents; e.g. there is just a '!' to indicate "rest of line truncated",
>> or they are are wrapped arbitrarily in the middle of a newsgroup (or
>> between newsgroups if you are lucky).

>Surely this is a quality of implementation issue.

Then why does RFC 2822 make such an issue of it, giving encouragement to
fold everything within 78 characters (rather than 998), presumablt because
that's how user agents prefer it.

Generally speaking, I have a great dislike if what user agents show me has
been munged beyond ehat was actually on the wire.

>> Moreover, this is the only header, within News or Mail, that cannot be
>> folded, so it sticks out like a sore thumb.

>Er, no.  Message-ID cannot be folded, Control cannot be folded, Date will
>probably break in practice if folded, Followup-To similarly cannot be
>folded, and that's just off the top of my head.  And I'm still concerned
>that folding Path might not actually work in practice, although it does
>work with the software I personally have checked.

Message-ID cannot be folded simply because the only folding points are at
the beginning and end, and we don't allow empty header content or empty
folded lines.

Control _can_ be folded (it has FWS in it).

Folding of Date is possible, but hevily deprecated in RFC 2822 (except
before any final comment). We inherit that deprecation.

Folding of Followup-To stands or falls with folding of Newsgroups.

I believe Paths are regularly folded in the Real Usenet.

>> Note that the problem is with relaying agents. I think all current user
>> agents would accept and display them properly.

>I'm pretty sure you're wrong.

See reply to Richard.

>> Therefore, I wish to see as much incentive as possible to get relaying
>> agents upgraded. MUST accept in our draft is a start. SHOULD NOT
>> generate would give more incentive than MUST NOT generate, and something
>> even weaker (as in the article-* drafts) would give more incentive
>> again.

>If a new RFC differs from existing practice by too much, more requirements
>that differ from existing practice cease to be motivation to change
>software and become motivation to just ignore the document.

There is a balance to be struck. Hence this discussion.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Subject: Re: Newsgroup names #1021
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 13 Jun 2005 08:49:04 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> It's not completely clear to me whether this is the right thing to do,
>> since there are some uses of mixed-case newsgroup names in the wild
>> already and they don't actually break anything beyond making things
>> more confusing.  The ftp.isc.org list refuses to add them, but the
>> protocol actually has historically supported them.

> When the WG last discussed this (many years ago) it was claimed that
> some software treated newsgroup-names case insensitively and some
> didn't. We rejected the idea of allowing injecting agents to "fix it
> up", and so we concluded that a ban on uppercase was the safest option.

Yeah, I'm leaning more and more towards prohibiting them because of the
case-insensitivity mess.

> Here is the relevant text from draft-13, which could usefully be copied
> into USEFOR more-or-less as it stands:

[...]

Hm, that sure looks mostly like USEPRO material to me.

USEFOR should deal with what newsgroup names are *syntactically* valid and
discuss newsgroup names that are *always* forbidden.  If the newsgroup
name is just reserved for special purposes, that's exactly the sort of
thing that USEPRO is for.  Given that reasoning, the only ones of these
that I would put into USEFOR are:

>      o Any newsgroup-name containing the component "ctl" (likewise);
>      o Any newsgroup-name containing the component "all" (because this
>        is used as a wildcard in some implementations).

> {There used to be a Usenet group "general" which even contained stuff
> when I first started to read news, but I don't think it propagates any
> more.}

Why would it have stopped?

newsfeed:~/log> grep ' general$' unwanted.log 
322514 general

Old newsgroups never die; they only become spam traps.

> I think articles with "control*" and "junk" in the Newsgroups header
> should definitely be prohibited, and hence should not be propagated
> (trolls will continue to post to control.*, but that's just what trolls
> do :-( ).

I'm not really seeing what harm is caused by it, other than it being
confusing.  Servers that don't want to can easily refuse to propagate such
articles.

> In draft-13, we also had names containing <component>s starting with "+"
> and "-" which were reserved in the same way, the intention being that
> servers who wanted to provide some special service to their own
> community of clients (say filtered and unfiltered versions of the same
> group) could do so without polluting the official namespace (so, with
> hindsight, the "junk" group could have been called "+junk"). I think
> this would be a useful feature to retain in USEFOR just in order to
> prevent further namespace pollution by "enterprising" server admins.

> There was also provision for reserved components begining with "_" for
> "future extensions". It is not so clear whether those should be
> prohibited from propagating.

If we're going to reserve one of these characters for future
standardization purposes, it needs to propagate.  All of the reasons for
reserving a group name that I can think of would need them to propagate
(such as using it for punycode).

Date: Mon, 13 Jun 2005 10:56:34 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

Charles Lindsey wrote:

> In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:
>>5. What should an implementor of a user Agent make of the 7.6
>>language regarding the References header?  What can that implementor
>>assume about articles created by agents that pre-date the draft?  Or
>>is there a flag day?  What can they assume about a multi-part FAQ?
> 
> 
> There is no need for any flag day. There is nothing that an implementor
> using the References header has to do in consequence of our draft that is
> in any way different from what he was doing beforehand.
> 

These documents will obsolete previous standards. They must be a complete
specification, suitable for implementation of all software which processes
articles.

"Nothing....in any way different from what he was doing beforehand"
is not a specification, much less a complete specification.

To: [email-masked]
Xref: clerew local.usefor:21266
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1031 Re: standard terminology and text
Date: Mon, 13 Jun 2005 11:54:41 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>After looing through draft-ietf-usefor-usefor-04, I've found that of the 
>"redefined" terms:

>   This specification uses the terms "header", "header name", and
>   "header content" which are synonymous with the [RFC2822] terms
>   "header field", "field name", and "field body" respectively.

>"header name" occurs once.
>"header content" occurs three times.

You need to count the occurrences in USEPRO and USEAGE too.

But "content" is a special case and needs redefining to suit the way it is
used in USEPRO. So it should be "header body" in USEFOR, if we retain the
"header" terminology (IMO we should retain it).

>(I'm somewhat unsure what a "header line" means - it isn't listed in the 
>definitions, and is not used in 2822. I'm assuming that it's a line (as 
>defined in RFC 2822) that occurs in the header (as defined in RFC 2822))

It is used variously in various places (sometimes when speaking of the
individual lines after a header is folded). Therefore, it SHOULD NOT be
used, unless the context makes it absolutely clear what is intended.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21265
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Newsgroup names #1021
Date: Mon, 13 Jun 2005 11:48:39 GMT
Lines: 153
Sender: [email-masked]

>> This was an issue that Charles raised off-list, I believe.

It was raised off list because it related to matters that were contained
in the old draft-13 and which were not believed to be controversial, but
which had not yet found their way into USEFOR.

>> The text of the ticket is the following:
>> ------------------------------
>> I suggest to update the Newsgroup ABNF as follows:
>> 1). uppercase letters must be prohibited in newsgroup names

>It's not completely clear to me whether this is the right thing to do,
>since there are some uses of mixed-case newsgroup names in the wild
>already and they don't actually break anything beyond making things more
>confusing.  The ftp.isc.org list refuses to add them, but the protocol
>actually has historically supported them.

When the WG last discussed this (many years ago) it was claimed that some
software treated newsgroup-names case insensitively and some didn't. We
rejected the idea of allowing injecting agents to "fix it up", and so we
concluded that a ban on uppercase was the safest option.

However, perhaps a NOTE to the effect that some existing groups with UC
names (and also some with all-digit components) DO exist on Usenet, with a
warning that their propagation may be poor and that creation of further
such groups would be inappropriate.

>> 2). single component newsgroups must be prohibited (this also
>> addresses conflict with the special value "poster")

>I think they should be allowed; "poster" and "junk" just have to be
>reserved.  (Getting the wording right for "junk" without going into what
>"junk" is used for is going to be a bit tricky.)

Here is the relevant text from draft-13, which could usefully be copied
into USEFOR more-or-less as it stands:

   The following forms of newsgroup-name MUST NOT be used except for the 
   specific purposes indicated:

     o Newsgroup-names having only one component. These are reserved for 
       newsgroups whose propagation is restricted to a single host or
       local network, and for pseudo-newsgroups such as "poster" (which
       has special meaning in the Followup-To-header - see section 6.7), 
       "junk" (often used by serving agents), and "control" (likewise);
     o Any newsgroup-name beginning with "control." (used as pseudo-
       newsgroups by many serving agents);
     o Any newsgroup-name containing the component "ctl" (likewise);
     o "to" or any newsgroup-name beginning with "to." (reserved for the 
       ihave/sendme protocol described in section 7.4, and for test
       articles sent on an essentially point-to-point basis);
     o Any newsgroup-name beginning with "example." (reserved for
       examples in this and other standards);
     o Any newsgroup-name containing the component "all" (because this
       is used as a wildcard in some implementations).

I think that gives the correct rule for single component newsgroups.

{There used to be a Usenet group "general" which even contained stuff when
I first started to read news, but I don't think it propagates any more.}

>>> 2). single component newsgroups must be prohibited (this
>>> also addresses conflict with the special value "poster")

>>  But not with pseudo-groups like "junk" or "control", and
>> the real NG "spamcop" <news://news.spamcop.net/spamcop>

>It depends on how one prohibits them, basically.  Definitely, news readers
>have to be able to read control and junk, so it has to be able to appear
>in the Xref header.  In practice, people post to control and junk
>directly, but this often doesn't really do what one wants and I'm not sure
>it's a good idea to allow in the standard because the behavior is somewhat
>undefined.  Crossposting to those groups is generally fairly harmless,
>just confusing.  Creating groups by that name with control messages is a
>bad idea and shouldn't happen.

I think articles with "control*" and "junk" in the Newsgroups header should
definitely be prohibited, and hence should not be propagated (trolls will
continue to post to control.*, but that's just what trolls do :-( ).

But such "pseudo-groups" can be read from servers, and hence can appear in
Xref headers. Therefore, the proper rule for such <newsgroup-name>s should
be

  MUST NOT appear in Newsgroups and Followup-To headers; MAY appear in
  Xref headers.

In draft-13, we also had names containing <component>s starting with "+"
and "-" which were reserved in the same way, the intention being that
servers who wanted to provide some special service to their own community
of clients (say filtered and unfiltered versions of the same group) could
do so without polluting the official namespace (so, with hindsight, the
"junk" group could have been called "+junk"). I think this would be a
useful feature to retain in USEFOR just in order to prevent further
namespace pollution by "enterprising" server admins.

There was also provision for reserved components begining with "_" for
"future extensions". It is not so clear whether those should be prohibited
from propagating.

>>> 3). prohibiting all digit components can be done in ABNF,
>>> but it would be ugly.

>>  Ugly if necessary is acceptable, but why is it necessary ?

>It's necessary to retain backward compatibility with any news server that
>uses the traditional directory structure for the news spool, since the
>group example.foo.600 and article 600 in example.foo both want to use the
>same file name.  Many older news servers will essentially crash if one has
>created example.foo.600 and article 600 arrives in example.foo.

I agree this prohibition should remain (but see suggestion for a NOTE
above).

>....  It's just hard to work
>around if you want to retain the standard spool structure because normally
>the part of the server that numbers articles and the part of the server
>that stores them don't have a two-way conversation, and there's a fair bit
>of existing software out there that just isn't ever going to cope.

We sometimes forget that newsreaders sometimes consult the newsspool
directly, without using NNTP as a conduit. For example, you can do this in
nn (I do) and no doubt in things like trn also.

>> And pseudo-TLH "to" is some mail-by-news trick, from a user's POV like
>> an NG.

>The to.* newsgroups have extremely special handling with older news
>software and were used for point-to-point UUCP feeds and similar sorts of
>tricks.  They are special-cased in numerous places in INN and I believe
>also in C News,...

And USEPRO still documents how to use them. And there was certainly
evidence of them still being used a couple of years ago.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21267
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1031 Re: standard terminology and text
Date: Mon, 13 Jun 2005 11:56:39 GMT
Lines: 22
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Fri June 10 2005 12:50, Henry Spencer wrote:

>> We should be consistent:  either use the terminology that is common in
>> news, or use the RFC 2822 terminology from mail.

>1. RFC 2822 isn't just "mail". This has been discussed before.

And your particular interpretation of it has been dismissed on each
occasion.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21269
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Mon, 13 Jun 2005 12:39:48 GMT
Lines: 82
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>Charles Lindsey wrote:

>> 7.6.1.  Construction of the References header
>> 
>>    The following procedure is to be used whenever some previous article
>>    (the "parent") is to be referred to in the References header (F-
>>    3.2.1) of a new article, whether in the course of generating a
>>    followup or for some other reason (e.g. the later parts of a
>>    multipart posting such as a FAQ, or the later parts of a
>>    message/partial as suggested in [RFC 2046]).

>1. This text appears under "duties of a followup agent."  You have not
>shown text in USEPRO that lets non-followups have a references header,
>unless they were generated by a followup agent.

Eh? It is USEFOR that "lets" you write a References header for a
non-followup. The proposed USEFOR text directs you to USEPRO to find how
to do it properly, and the above section is where it tells you that (it is
a subsection of 7.6 because there did not seem a better place to put it).
That section even gives you examples of non-followup situations that might
arise.

>Non-followups cannot be generated by a followup agent, because follow up agent is
>defined as software used for preparing and posting followups.

Indeed. But that section is not restricted to use by followup agents, as
its wording clearly implies.

>2. Since the semantics of the header were deleted from USEFOR, and the
>semantics do not appear in this proposed text for USEPRO, there is
>no guidance on how user agents should detect or treat non-followups
>that have References headers.

I have already expressed my regret at the lack of semantics in USEFOR. It
does seem, however, that our chair is willing to permit further discussion
of that. But not of the definition/detection of followups.

As to treatment of References headers, standards do not normally deal with
user interface issues, apart perhaps from sopme brief mention of
possibilites, such as what is currently written in USEPRO 7.7.

>>         NOTE: There is no provision in this standard for an article to
>>         have more than one parent. The essential property of the
>>         References header, guaranteed by the procedure above and to be
>>         preserved in any future extension, is that no article can ever
>>         precede one of its own parents.

>3. Specifying what something is not is by no means a definition of "The essential
>property" of anything.

That is a NOTE, not a definition. Given the lack of any semantics in
USEFOR, it was the best I could do. An implementor writing a threading
algorithm is entitled to assume that the relevant headers (USEPRO 7.7) have
been constructed "properly". The essential property mentioned there is the
minimum that will enable him to do his job. For sure, it is not enough to
say that the References is "just a list of <msg-id>s, as Frank would have
it.

>5. What should an implementor of a user Agent make of the 7.6
>language regarding the References header?  What can that implementor
>assume about articles created by agents that pre-date the draft?  Or
>is there a flag day?  What can they assume about a multi-part FAQ?

There is no need for any flag day. There is nothing that an implementor
using the References header has to do in consequence of our draft that is
in any way different from what he was doing beforehand.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21271
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Mon, 13 Jun 2005 13:51:55 GMT
Lines: 80
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>In message <[email-masked]>, Charles Lindsey
><[email-masked]> writes

>>7.6.1.  Construction of the References header
>>
>>   If the resulting References header is excessively long it MAY (and if
>>   its total length, whether folded or not, exceeds 998 chacacters it
>>   MUST) then be trimmed, but the first and the last two message
>>   identifiers MUST NOT be removed.

>I've two problems here which would confuse me as an implementor...

>a)   does the overall length of 998 include the 12 characters
>     "References: " or not (and what of the CR LF?).

Yes for the References, NO for the CRLF (which is true also of all other
places where that magic "998" appears.

>     Perhaps this might be best expressed as a requirement that when
>     unfolded into a single line including the keyword and terminating
>     CRLF that it does not exceed 1000 ?

>b)   the ones that can't be removed, is that before or after the message
>     ID has been added ?

>     I assume it must be after... because otherwise you'd have a
>     requirement for at four IDs each of which can be 250 long :)

Correct. It is applied to "the resulting References header" after you have
done all the appending etc from the previous paragraph.

>     It would be helpful to make that crystal clear. I can suggest some
>     words, if that's necessary....

   If the parent did not have a References header, the content of the
   new article's References header MUST be inherited from that of the
   Message-ID header of the parent, otherwise (there already was a
   References header) it MUST be comprised of the parent's References
   header followed by CFWS and the Message-ID header content of the
   parent (subject to trimming as described below).

   If the resulting References header is excessively long it MAY (and if
   its total length, whether folded or not, and including the header
   name but not the final CRLF, exceeds 998 chacacters it MUST) then be
   trimmed, but in such a manner that the first and the last two message
   identifiers always remain.

I am not sure that it really better (and it is certainly longer).

>>The essential property of the
>>        References header, guaranteed by the procedure above and to be
>>        preserved in any future extension, is that no article can ever
>>        precede one of its own parents.

>nope :(   if you're given rubbish (eg an article with existing circular
>references) then this statement suggests that you have an obligation to
>fix it up. I don't think that's a reasonable requirement. I suggest that
>since this is merely meant to be a helpful deduction then it is either
>toned down or removed.

If all References headers are constructed according to the preceding
paragraphs, then circular references cannot arise. If, by some accident,
one does arise, then "garbage in, garbage out" applies as usual.

But that paragpraph is just a NOTE, so it has no normative effect (and I
would prefer it to have been said in USEFOR - see my reply to Forrest).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21268
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Forward compatibility (Re: CFWS in References header)
Date: Mon, 13 Jun 2005 12:08:14 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>In message <[email-masked]>, Charles Lindsey
><[email-masked]> writes

>>The problem is that (most) relaying agents need to examine this header
>>during transit. To upgrade them to accept the folding is not rocket
>>science, but it is only now beginning to get done. So at present such
>>articles would not propagate well. Note that the problem is with relaying
>>agents. I think all current user agents would accept and display them
>>properly.

>That's far from obvious!!   I had to read the code to believe this
>statement about "my" client. As it happens, it looks as if it _is_ well
>structured enough to accept a folded Newsgroups header field.

My expectation was based on a presumption that the code used to display
the Newsgroups header would be no different that that used for other
headers, where UAs already cope happily with folded headers, either
displaying them "as received", of wrapping/refolding them to suit the
particular display device.

I could well believe that some agents might have a problem when generating
a followup, though even then the "naive" approach of just copying the
header as received into the new article would be correct.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21270
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Mon, 13 Jun 2005 13:25:09 GMT
Lines: 47
Sender: [email-masked]

In <[email-masked]> "Ruud H.G. van Tol" <[email-masked]> writes:

>Charles Lindsey schreef:

>>    If the resulting References header is excessively long it MAY (and
>>    if its total length, whether folded or not, exceeds 998 chacacters
>>    it MUST) then be trimmed, but the first and the last two message
>>    identifiers MUST NOT be removed.

>The four remaining message identifiers, with their CFWS, can have
>a combined length of more than 998 characters.

No. Three remaining. That is after the new one has been appended.

>I also read in that sentence that you are free to remove comments
>out of (such long) References (as long as you keep the message-ids).
>So the software might decide to try first to shrink all CFWS to a
>single space.

Although not clear from the present wording (because of the way "content"
has been defined), you can keep the old comments, or throw them away, or
even write news ones. Just so long as you keep the msg-ids as directed. I
will make sure that is clear.

And, of course, we are agreed that such comments are at least SHOULD NOT,
and maybe MUST NOT generate.

>It would seem more practical (to me) to just drop the oldest Message-IDs
>(and their CFWS) from the start of the References field, until there
>is room for the Message-ID to be added, but I guess that has all been
>discussed before and rejected. I think that newer Message-IDs are more
>useful (have more entropy/hold more information) than older ones.

No, the first one must be kept, because that one is the root of the tree
and may be used by some agents to distinguish between different trees.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 13 Jun 2005 08:26:44 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: References semantics (Re: #1008 USEFOR 1.5 References: Final text on "followup")
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> I've not started recording tickets on USEPRO yet; are you raising a new 
> issue on whether semantics of the References: header should be described 
> or not in USEFOR (separate from the definition of "followup", which is 
> closed), or are you wishing to raise an issue on USEPRO?

Charles posted text for discussion.  Some of that text was for USEPRO.

Charles previously posted that there is no way to consider the documents
completely independently, and these sections from two documents must
be discussed together.  I agree.

You wrote that you were entering tickets as you saw issues raised on the list.
I did not think that excluded USEPRO.

Why are you not entering tickets for USEPRO?  The point of a ticket
system is to record issues so that work on them can occur at time other
than when they are raised.  Delaying they entry of tickets is completely wrong.

If you don't want to discuss USEPRO now, and you don't want to enter tickets,
then I think it is appropriate to convince Charles that the documents must
be considered independently, and then instruct him not to post USEPRO text
until you will allow discussion.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1031 backward compatibility and handling incompatibilities
Date:  Mon, 13 Jun 2005 02:01:00 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 39
Sender: [email-masked]

Richard Clayton wrote:
 
> Whilst we continue down that path then it behoves us to use
> the jargon from "mail" in a consistent and user-friendly
> manner.

This "jargon" isn't limited to news, it's just what uers say
everywhere, mail, news, http, etc.  Generally this is obvious
from the context.  When I say "Accept-Language header", then it
is the "Accept-Language header field", what else could it be ?

Consistency is only important when there is a realistic chance
that readers get it wrong.  Talking about "header" when it's
actually only a "header field" could be dangerous.

The shorthand "header line" for "header field line" might be
critical, if a reader confuses "header line" (one line) and
"header field" (all lines).

Other shorthands like "header name" or "header content" or the
specific "From header" etc. are IMHO obvious, the long forms
are very artificial.  There's a time where you say "octet" or
"encoded character repertoire" etc. because it's important, but
there's also a time where "byte" or "charset" are good enough,
and clearer for casual readers not familiar with all details.

> The question is what is best going forward.

Get the technical essentials right, msg-id and newsgroup-name.
  
> If we are still allowed to disrupt the smooth progress of the
> NNTP documents then, in my view, that is what we should do.

You could always ask the authors what they think about it.  In
theory they could modify it in their "48" hours.  From my POV
it's a minor nit, and the outcome does not necessarily affect
what we do in USEFOR.
                      Bye, Frank

Date: Sun, 12 Jun 2005 15:30:17 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: "Forrest J. Cavalier III" <[email-masked]>, [email-masked]
Subject: References semantics (Re: #1008 USEFOR 1.5 References: Final text on "followup")
Sender: [email-masked]

I've not started recording tickets on USEPRO yet; are you raising a new 
issue on whether semantics of the References: header should be described or 
not in USEFOR (separate from the definition of "followup", which is 
closed), or are you wishing to raise an issue on USEPRO?

--On 10. juni 2005 17:14 -0400 "Forrest J. Cavalier III" 
<[email-masked]> wrote:

> 2. Since the semantics of the header were deleted from USEFOR, and the
> semantics do not appear in this proposed text for USEPRO, there is
> no guidance on how user agents should detect or treat non-followups
> that have References headers.

The implication (as I explicitly stated earlier) is that a reading agent 
cannot mechanically tell the difference between a followup and a 
non-followup.
So all the rules have to be stated in terms of handling articles with a 
References header; the term "followup" is only useful in USEFOR when 
describing how to generate one.

                    Harald

Date: Sun, 12 Jun 2005 14:59:57 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Russ Allbery <[email-masked]>, [email-masked]
Subject: Re: Newsgroups folding test
Sender: [email-masked]

--On 10. juni 2005 20:34 -0700 Russ Allbery <[email-masked]> wrote:

> Charles Lindsey <[email-masked]> writes:
>
>> OK, let's try it.
>
> I don't find these exercises useful.  I don't think they really prove what
> you're trying to prove.  Propagation only measures whether or not the vast
> majority of servers reject it; if only a smaller number of servers reject
> the message, it could still be a significant problem in practice but won't
> show on this sort of a test.

Actually I find them useful - but of course one has to be careful to check 
what they really test.

If this test had uncovered significant problems in distribution (query: did 
anyone see it in group man.test, but NOT in group misc.test, where both 
where carried?), newsgroup folding would have been dead in this round; it's 
clear that a standard meant to be useful in present reality has to be a 
"MUST NOT generate".

If it should uncover no significant problem in distribution, we can still 
come to the conclusion that it should be MUST NOT generate, because they 
cause other problems.

My opinion....

                    Harald

Date: Sun, 12 Jun 2005 13:53:46 +0100
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: Newsgroups folding test (was  CFWS in References header)
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes

>I have just submitted an article with the following headers to a server
>using INN 2.4.1. It certainly got stored on that server. I invite you all
>to see whether it arrived on any of the groups it was sent to

the article

>Message-ID: <[email-masked]>

is not currently (almost 2 days later) available on news.demon.co.uk
(well connected UK ISP server), news.mit.edu (MIT's server) or
news.cam.ac.uk (Cambridge University's server)

retention is sufficient for the article to be seen if it managed to
reach these servers by any flood-fill route (direct routes would have
meant that it arrived in seconds)

>It would also be helpful if others who have relaying privileges would try
>the same experiment.

the existing result speaks for itself

"Broken" articles like this don't propagate at all well :(

hence any suggestion that folded newsgroups header lines are current
state of the art (rather than some future possibility) seems premature

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

Date: Sun, 12 Jun 2005 13:36:28 +0100
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes

>Within USEPRO:
>
>7.6.1.  Construction of the References header
>
>   If the resulting References header is excessively long it MAY (and if
>   its total length, whether folded or not, exceeds 998 chacacters it
>   MUST) then be trimmed, but the first and the last two message
>   identifiers MUST NOT be removed.

I've two problems here which would confuse me as an implementor...

a)   does the overall length of 998 include the 12 characters
     "References: " or not (and what of the CR LF?).

     Perhaps this might be best expressed as a requirement that when
     unfolded into a single line including the keyword and terminating
     CRLF that it does not exceed 1000 ?

b)   the ones that can't be removed, is that before or after the message
     ID has been added ?

     I assume it must be after... because otherwise you'd have a
     requirement for at four IDs each of which can be 250 long :)

     It would be helpful to make that crystal clear. I can suggest some
     words, if that's necessary....

>        NOTE: There is no provision in this standard for an article to
>        have more than one parent. 

happy with this

>The essential property of the
>        References header, guaranteed by the procedure above and to be
>        preserved in any future extension, is that no article can ever
>        precede one of its own parents.

nope :(   if you're given rubbish (eg an article with existing circular
references) then this statement suggests that you have an obligation to
fix it up. I don't think that's a reasonable requirement. I suggest that
since this is merely meant to be a helpful deduction then it is either
toned down or removed.

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

Date: Sun, 12 Jun 2005 13:16:58 +0100
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: Forward compatibility (Re: CFWS in References header)
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes

>IMO, Newsgroups folding would be very useful.

cross-posting of articles is seldom a wise thing to do; it doesn't seem
to be wise to encourage it any more than at present :-(

One of the advantages of not having any folding is that we do not get
articles spammed to more than a dozen or so newsgroups.

With folding, we will see articles being sent to hundreds of newsgroups
at a time on a regular basis ... if that's not an issue for server
software, then I don't suppose its any worse for the readers than the
current spam situation.

However, I'd like to see some comment as to whether dealing with a
single article that needs to be inserted into hundreds or thousands of
groups at a time is likely to be affect performance, fragility, etc

> Newsgroups headers in excess
>of 80 characters are very common.

yes

> They display badly in many user agents;
>e.g. there is just a '!' to indicate "rest of line truncated", or they are
>are wrapped arbitrarily in the middle of a newsgroup (or between
>newsgroups if you are lucky). 

mayhap

>Even where they are displayed "properly"
>folded, the user is given the impression that it would be OK to generate
>new articles like that, and copy/paste from what is displayed to a new
>article might or might not result in a valid header.

really ?  could you give an example of where this has happened recently?

The way in which most user software works, you don't get to copy and
paste in truncated lists of newsgroups ...  and most people just press
the reply button and their software does the right thing with the
unmangled copy of the line (rather than the displayed version)

>Moreover, this is the only header, within News or Mail, that cannot be
>folded, so it sticks out like a sore thumb.
>
>The problem is that (most) relaying agents need to examine this header
>during transit. To upgrade them to accept the folding is not rocket
>science, but it is only now beginning to get done. So at present such
>articles would not propagate well. Note that the problem is with relaying
>agents. I think all current user agents would accept and display them
>properly.

That's far from obvious!!   I had to read the code to believe this
statement about "my" client. As it happens, it looks as if it _is_ well
structured enough to accept a folded Newsgroups header field.

I have limited faith that other code would be as solidly written :(

>Therefore, I wish to see as much incentive as possible to get relaying
>agents upgraded. MUST accept in our draft is a start. SHOULD NOT generate
>would give more incentive than MUST NOT generate, and something even
>weaker (as in the article-* drafts) would give more incentive again.

"my" client, Turnpike, generates folded References: header fields and so
we now have ten years plus experience of this issue.

It has been quite common to see our References: mangled by other
software -- that assumed that just the first physical line needed to be
considered. When this made a mess of threading (or worse) then their
users complained and the authors of this other software then complained
to us, entirely incorrectly, that the standards had been ignored and we
had done the wrong thing :(

I too would like to see better handling of incoming material, but years
of experience suggest that this is a long haul :(  ALSO, the aim here
ought to be stated explicitly rather than buried in some subtle usage of
SHOULDs or whatever. Some implementors are, I'm afraid, dim :(

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

Date: Sun, 12 Jun 2005 12:52:34 +0100
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1031 backward compatibility and handling incompatibilities
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes

>I think he is upset because we use the term "header" rather than "header
>field" and "header name" rather than "field name".
>
>I would point out that our drafts have always used this terminology (and
>have carefully documented that usage).

Although I have sympathy for the view that this is continuing the
tradition of how "news" documentation is written, I think that's a
mistake :(

Considerable leverage is gained by defining "news" as a special case of
"mail" (albeit, I sometimes wonder if -- when the minutiae is debated at
such length -- the gain is entirely worth the pain).

Whilst we continue down that path then it behoves us to use the jargon
from "mail" in a consistent and user-friendly manner.

Although there may be mileage in saying "In this standard we use frooble
to mean 'some very long phrase that is hard to keep repeating'" I don't
think that it does anything other than create confusion to use a
substitute for quite short phrases AND what's more, to use a word
"header" that so closely resembles what is being substituted.

As a data point, I'm currently in the process of finally editing my PhD
thesis (yes, they let you do PhDs even at my advanced age!) and realised
that I'd been very sloppy with my own use of the word "header". It took
less than an hour to work through the document and fix every mention to
the correct 2822 jargon.  I really do not think that it reads any worse
now that I've done it, nor is it harder to understand for a neophyte.

>RFC 1036 used the term "header" (though not consistently, since it
>sometimes used "header line" - a quick look revealed no occurrence of
>"field", though I have not grepped to confirm it).
>
>Son-of-1036 used the term "header".
>
>The NNTP draft, which has just completed its IETF Last Call period, uses the
>term "header".

of these three points, this is the _only_ one that would concern me (we
are, I trust, hoping to reach a situation where only historians consult
the other two documents, so who cares what they say?) ... so I checked
the documents recently exiting last call, which appear to be:

http://www.ietf.org/internet-drafts/draft-ietf-nntpext-authinfo-09.txt

        which contains the single phrase "This may be accomplished, for
        example, by inserting headers in the posted articles ..."

        ie: trivial change to "header fields"

http://www.ietf.org/internet-drafts/draft-ietf-nntpext-tls-nntp-06.txt

        which doesn't contain "header" at all

http://www.ietf.org/internet-drafts/draft-ietf-nntpext-streaming-05.txt

        this uses "headers" twice, both of which can easily be changed
        to "header fields"

http://www.ietf.org/internet-drafts/draft-ietf-nntpext-base-27.txt

        this uses "header" extensively in 3.6 (relating to folding) and
        would require a moderate amount of work :-(  though in my view
        that work is mechanical and should not give rise to debate.

>I move that we make no change to this long established practice.

You can argue that both alternatives are "long established".

The question is what is best going forward. And there the choice appears
to be between creating significant consistency in the set of documents
that future implementors consult; and in doing some mechanical editing
to documents that are not yet set in stone (albeit some are already a
long way towards that state).

If we are still allowed to disrupt the smooth progress of the NNTP
documents then, in my view, that is what we should do.

- -- 
richard @ highwayman . com                       "Nothing seems the same
                          Still you never see the change from day to day
                                And no-one notices the customs slip away"

Date: Sun, 12 Jun 2005 00:08:55 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: John Stanley <[email-masked]>, [email-masked]
Subject: Re: Newsgroup names #1021 (was: #1008 USEFOR 1.5 References: Final text on "followup")
Sender: [email-masked]

--On fredag, juni 10, 2005 11:12:25 -0700 John Stanley <[email-masked]> 
wrote:

>
>
> Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:
>
>> This was an issue that Charles raised off-list, I believe.
>
> This is the IETF working group tasked with news. There is no "off-list"
> issue, either it comes here or it isn't.
>
>> Good point - if filing tickets in response to issues raised off-list, we
>> need to make sure the WG is notified.....
>
> You are certainly kidding, aren't you? You are actually putting more
> emphasis on this trouble ticket system than on the working group? Gee,
> MAYBE we ought to mention trouble tickets on the working group that is
> supposed to be dealing with them?

John, how much does it take to make you see that the door you're trying to 
bash in is already open?
The fact that this issue made it onto the ticket system without the list 
being notified is an error; it is the list that is the WG.

>
>> The following comment was added by someone logged in as "ietf":
>
> That's how you told people to access the tickets. Is it odd that they
> do it the way you told them? And that they way you told them is
> essentially anonymous?
>
> Since I pointed out this problem in private and never got any resolution
> to it, I'll now bring it up in public.
>
> Why is this "trouble ticket" system using HTTPS with bogus certificates?
> This prevents participation by those who refuse to accept bogus certs. Or
> perhaps that was the goal?

Because this system is run as a voluntary gift to the IETF by Randy Bush, 
and he thinks the entire idea of trusting "root certificates" just because 
they got shipped with your browser is ludicrous.
I can see his point.

> There is no reason for https. IETF discussions are supposed to be public.
>  Whatever you post there is supposed to be available to anyone who wants
> to see it. There is no security issue: Harald published the
> username/password pair we are supposed to use to access the site. Why
> https?

Take it up with Randy. I wish you luck.

> But more important, WHY does the trouble ticket system exist at all? If
> Harald wants to keep a tickler file of issues he thinks needs to be
> resolved, that's fine. If he wants to publish this list, fine.
>
> When that list takes on the task of discussing the issues, not so fine.
> If Charles's issue is worth discussing, it is worth discussing here. This
> "trouble ticket"  system is a violation of, if not the letter, the spirit
> of IETF working groups, not only in theory, but now in practice. There
> should never be a question of "should we tell the WG about this issue
> we're discussing on this trouble ticket system?" Never. It should never be
> possible to have that question come up.
>
> Since it has now been demonstrated that the question has to be asked, it
> is time to shut down the trouble ticket system and bring discussion back
> home. Holding discussions in two places, one of which has limited access,
> is wrong.

John, I'll be impolite for once.
This group has demonstrated conclusively that just having discussions on 
the list does not lead to the group finishing.
A trouble ticket system is one means of helping a group close.
Your behaviour is part of the reason the group is not finishing.

> And then I run across this gem in "Friday - Ticket status":
>
>> 1002 USEFOR 6. References - Updates needed
>> No discussion; ticket seems clear.
>
> References has been discussed at length IN THIS LIST, but now that there
> is no discussion in the "trouble ticket" system, the answer is clear.

You didn't get the point that "6." referred to section 6 in the document, 
did you?

> The only thing that is clear is that this trouble ticket system is being
> used to bypass the standard IETF working group process. Either that, or
> it  creates the marvelous illusion that it is being used to bypass the
> process, and either situation is unacceptable.

Feel free to leave, John.

Date: Sat, 11 Jun 2005 12:58:47 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Newsgroup names #1021
Sender: [email-masked]

On Sat, 11 Jun 2005, Russ Allbery wrote:
> > if net.AAAAAARGH is permitted, is it different from net.aaaaaargh, or are
> > they the same?
> 
> Pretty much every piece of news software out there will treat them as
> different.

The "MUST not" in son-of-1036 was motivated by exactly that.  Given that
existing software overwhelmingly does case-sensitive comparisons, the
viable choices are to say that net.AAAAAARGH and net.aaaaaargh are
different names (aaargh!) or to forbid net.AAAAAARGH altogether (with the
usual "a cooperating subnet can do whatever it wants provided it keeps the
curtains closed" escape hatch).

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: security considerations
Date:  Sat, 11 Jun 2005 18:35:10 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 36
Sender: [email-masked]

Bruce Lilly wrote:

>>> multipart/mixed (; boundary=foo ; xyz=");bOuNdArY*=''next%20part(")

>> that addresses the RfC 2231 issue.  Of course it's abuse,
>> 2231 clearly says "don't do this if it's unnecessary", and
>> for a boundary it is unnecessary.
> [...]

> In this case since the boundary delimiter is English text it
> could have been language-tagged ('en'next%20part).

If an implementation supports RfC 2231 it won't display / spell
check / output / ... the MIME boundary value.  So I'd still say
that it's abuse to try these tricks with a boundary.

We could roll our own "updates 2046" to nail it, but for USEFOR
a short "security consideration" is good enough:

| MIME security considerations are discussed in [RFC2046].  Note that
| applying some [RFC2231] extensions for parameters like multi-line
| paramters on a boundary parameter as defined in [RFC2046] might be
| abused to bypass
-                  simple algorithms trying to analyze MIME parts.
+                  naive methods of handling parameters.  Two examples:
+
+ Content-Type:
+  multipart/mixed (; boundary=foo ; xyz=");bOuNdArY*=''next%20part(")
+ Content-Type: multipart/digest;
+  boundary (not=me) = ("yes ;-) simple (foo;bar") ; x-foo = xyzzy
+

With that provision I'm willing to accept a "MUST support RfC 2231".

                                Bye, Frank

To: [email-masked]
Subject: Re: Newsgroup names #1021
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sat, 11 Jun 2005 09:23:04 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> which leads to another interesting question:
> if net.AAAAAARGH is permitted, is it different from net.aaaaaargh, or are
> they the same?

> I HOPE the answer is that "obviously, they are the same", but I had to
> ask...

Pretty much every piece of news software out there will treat them as
different.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1031 Re: standard terminology and text
Date:  Sat, 11 Jun 2005 18:01:48 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 28
Sender: [email-masked]

Bruce Lilly wrote:
 
> 2110 is obsoleted

Okay, I have now updated e.g. <http://purl.net/net/rfc/2110>
to show the RfC status (+ abstract + keywords) first instead
of directly jumping to the txt files.

If anybody used this PURL redirection (except from me ;-) to
get RfCs directly he just met Murphy.  Do you want a copy of
any flames ?

> the obsoleting RFC has a multitude of problems (which we 
> here are not chartered to solve -- byt we can and should
> fix the problems in our deliverables).

Nonsense like "header" vs. "header field" is irrelevant while
simple questions like "what is a newsgroup name" are unclear.

At least we have already solved "what is a msg-id" minus the
detail of the production names.  BTW, I hope you supoprt the...

       msg-id = "<" id-local "@" id-domain ">"

...style instead of <id-right> which could be almost anything.
 
                     Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date:  Sat, 11 Jun 2005 17:18:33 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 20
Sender: [email-masked]

Ruud H.G. van Tol wrote:
 
>> If the resulting References header is excessively long
..........^^^^^^^^^
>> the first and the last two message identifiers MUST NOT be
>> removed.
 
> The four remaining message identifiers, with their CFWS, can
> have a combined length of more than 998 characters.

There are only three, the text talks about the result _after_
adding the parent msg-id.
  
> I also read in that sentence that you are free to remove
> comments out of (such long) References (as long as you keep 
> the message-ids). So the software might decide to try first
> to shrink all CFWS to a single space.

Good idea, maybe we should say so explicitly.  Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Newsgroup names #1021
Date:  Sat, 11 Jun 2005 17:09:45 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 53
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
 
> which leads to another interesting question:
> if net.AAAAAARGH is permitted, is it different from
> net.aaaaaargh, or are they the same?
 
> I HOPE the answer is that "obviously, they are the
> same", but I had to ask...

Yes, I'm very much for discussing _relevant_ questions like
"what's a newsgroup name ?" or "what's a msg-id ?" instead of
wasting time with "what's a FYI 18 header ?".

s-o-1036 says:

| Newsgroup names MUST not contain uppercase letters

(disclaimer about encoded MIME words for I18N removed)

NNTP draft 27 offers some wild fantasies about UTF-8 in its
<wildmat-exact>, if that's related to reality then it's in a
parallel universe far away from my parallel universe:

 [10,2]
| Although this specification allows UTF-8 for newsgroup
| names, they SHOULD be restricted to US-ASCII until a
| successor to RFC 1036 [RFC1036] standardises another
| approach

Thanks but no thanks.  "Stringprep" is black magic while we
seriously consider to support any EST / EDT / UT timestamps.

 [10.3]
| Until such specifications are published, implementations
| SHOULD match newsgroup names octet-by-octet.  It is
| anticipated that any approved scheme will be applied
| "at the edges" and therefore octet-by-octet comparison
| will continue to apply to most, if not all, uses of
| newsgroup names in NNTP.

| In the meantime, any implementation experimenting with
| UTF-8 newsgroup names is strongly cautioned that a future
| specification may require that those names be canonicalized
| when used with NNTP in a way that is not compatible with
| their experiments.

Did I miss an "intended status: experimental" in nntp-27 ?

Anyway, apparently we can just do "the right thing", and in
the light of this draft I'd say "lowercase a..z0..9+-_".  No
uppercase, same idea as IDN.
                            Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1031 Re: standard terminology and text
Date:  Sat, 11 Jun 2005 16:01:32 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

Bruce Lilly wrote:

> 1. RFC 2822 isn't just "mail".

It's a proposed standard.  I think we have already seen
that there are some issues with its relation to MIME, and
its idea of NO-WS-CTL in the header is plain wrong.

> 2. the standard definition of "header" appears in FYI 18

Yes, just below "hacker".  Please use RfC numbers, it took
me two attempts (1938, 1983) to check that you are really
talking about 1983.

RfC 1983 is obviously obsolete, US-idiosyncratic, and in
parts as irrelevant as RfC 1855.  It's a simple glossary,
it's no "standard terminology", unless you dare say so.

Among other obsolete crap it defines "uuencode", that won't
force us to remove MIME from USEFOR, or to sanction UUE.

                        TTFN, Frank

P.S., TTFN is defined in FYI 18.

Date: Sat, 11 Jun 2005 14:40:09 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: "Ruud H.G. van Tol" <[email-masked]>, [email-masked]
Subject: Re: Newsgroup names #1021
Sender: [email-masked]

--On l�rdag, juni 11, 2005 11:25:06 +0200 "Ruud H.G. van Tol" 
<[email-masked]> wrote:

>
> Russ Allbery schreef:
>> Harald Tveit Alvestrand:
>
>>> 1). uppercase letters must be prohibited in newsgroup names
>>
>> It's not completely clear to me whether this is the right thing to do,
>> since there are some uses of mixed-case newsgroup names in the wild
>> already and they don't actually break anything beyond making things
>> more confusing.  The ftp.isc.org list refuses to add them, but the
>> protocol actually has historically supported them.
>
> I have messages in my test suite that have the same newsgroup name in
> different casings in the Newsgroups: header. This was to test a new
> bot that fights excessive crossposting.

which leads to another interesting question:
if net.AAAAAARGH is permitted, is it different from net.aaaaaargh, or are 
they the same?

I HOPE the answer is that "obviously, they are the same", but I had to 
ask...

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Subject: Re: Newsgroup names #1021
Date: Sat, 11 Jun 2005 11:25:06 +0200
Organization: Chaos rules.
Sender: [email-masked]

Russ Allbery schreef:
> Harald Tveit Alvestrand:

>> 1). uppercase letters must be prohibited in newsgroup names
>
> It's not completely clear to me whether this is the right thing to do,
> since there are some uses of mixed-case newsgroup names in the wild
> already and they don't actually break anything beyond making things
> more confusing.  The ftp.isc.org list refuses to add them, but the
> protocol actually has historically supported them.

I have messages in my test suite that have the same newsgroup name in
different casings in the Newsgroups: header. This was to test a new
bot that fights excessive crossposting.

-- 
Grtz, Ruud

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Sat, 11 Jun 2005 09:35:54 +0200
Organization: Chaos rules.
Sender: [email-masked]

Charles Lindsey schreef:

>    If the resulting References header is excessively long it MAY (and
>    if its total length, whether folded or not, exceeds 998 chacacters
>    it MUST) then be trimmed, but the first and the last two message
>    identifiers MUST NOT be removed.

The four remaining message identifiers, with their CFWS, can have
a combined length of more than 998 characters.

I also read in that sentence that you are free to remove comments
out of (such long) References (as long as you keep the message-ids).
So the software might decide to try first to shrink all CFWS to a
single space.

I would just take the word 'two' out of it. Or maybe change it into
'few'.

It would seem more practical (to me) to just drop the oldest Message-IDs
(and their CFWS) from the start of the References field, until there
is room for the Message-ID to be added, but I guess that has all been
discussed before and rejected. I think that newer Message-IDs are more
useful (have more entropy/hold more information) than older ones.

-- 
Grtz, Ruud

Date: Sat, 11 Jun 2005 01:30:08 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1031 Re: standard terminology and text
Sender: [email-masked]

On Fri, 10 Jun 2005, Bruce Lilly wrote:
> > We should be consistent:  either use the terminology that is common in
> > news, or use the RFC 2822 terminology from mail.
> 
> 1. RFC 2822 isn't just "mail". This has been discussed before.

It has indeed, and while a few people think RFC 2822 isn't "just mail",
others note that RFC 2822 explicitly says that it applies "within the
framework of 'electronic mail' messages".  It tries to be more widely
applicable, but hasn't entirely succeeded yet. 

In any case, there is no question that RFC 2822 derives from mail work and
that its terminology is that of mail, which is the specific issue being
talked about.  Please try to stay on topic. 

> 2. the standard definition of "header" appears in FYI 18, which is
>    certainly not jst "mail"

FYI 18 (aka RFC 1983) gives two definitions for "header", one for the
header of a packet and one for the header of an "electronic mail message". 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
Subject: Re: CFWS in References header
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 10 Jun 2005 20:38:45 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Bruce Lilly <[email-masked]> writes:

>> There is a huge difference between documenting specific issues and
>> making specific recommendations vs. leaving matters to the opinions of
>> non-experts.

> I am impressed by your classification of all news-server implementors as
> "non-experts".

I have no difficulty whatsoever classifying most news server implementors
as non-experts.  One thing that's become abundantly clear over the years
is that most news implementors in general do not bother to read the
standards.  I'm not sure how to fix this, although getting a new, more
modern standard out the door would certainly help.

>> Something similar could certainly be used where there are issues
>> corresponding to what Mark calls "baby programmer errors", e.g.
>> inability to handle msg-ids with a legal 255-octet domain name.  There
>> is no good reason to be stuck with such an absurdity forever;
>> implementers can and should be encouraged to fix the problems "with all
>> due speed".

Sorry, I'm just completely unenthused about trying to fix this
non-problem.  It offends people's sense of cleanliness to limit the length
of anything.  I understand, but that's not a reason to change the
standard.  Interoperability is more important than that sort of
aesthetics.  There is no earthly reason why anything needs to generate
251-octet message IDs.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: CFWS in References header
Date: Fri, 10 Jun 2005 23:34:53 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 10 2005 14:44, Charles Lindsey wrote:

> I am impressed by your classification of all news-server implementors as
> "non-experts".

I have not done so and I do not appreciate your false assertion.

To: [email-masked]
Subject: Re: Newsgroups folding test
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 10 Jun 2005 20:34:57 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> OK, let's try it.

I don't find these exercises useful.  I don't think they really prove what
you're trying to prove.  Propagation only measures whether or not the vast
majority of servers reject it; if only a smaller number of servers reject
the message, it could still be a significant problem in practice but won't
show on this sort of a test.

Furthermore, most transit servers do nearly no syntax validation of the
article whatsoever, or only the most minimal amount possible, so this sort
of test proves that all the new syntax we want to add works fine, also
proves that all the new syntax we don't want to add works fine, and
additionally proves that it's just fine to use "MEZT" as your time zone.
:)  You're lucky transit servers check even Message-ID and Date.  So
proving that things can propagate really is pretty meaningless.

The real test is how many software packages might have trouble when trying
to read the message, trying to construct followups, trying to post a
similar message back into a newsgroup, trying to archive that message,
trying to gate it into mail, etc.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1031 backward compatibility and handling incompatibilities
Date: Fri, 10 Jun 2005 23:33:42 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 10 2005 14:22, Charles Lindsey wrote:

> I think he is upset because we use the term "header" rather than "header
> field" and "header name" rather than "field name".
> 
> I would point out that our drafts have always used this terminology (and
> have carefully documented that usage).

Sample quotes from draft-ietf-usefor-usefor-00.txt:

   User agents SHOULD support on receipt and MAY generate MIME extension
   header fields, 

   The "Message-ID:" field contains a single unique message identifier.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Henry Spencer <[email-masked]>
Subject: Re: #1031 Re: standard terminology and text
Date: Fri, 10 Jun 2005 23:28:32 -0400
Cc: Usefor Mailing List <[email-masked]>
Sender: [email-masked]

On Fri June 10 2005 12:50, Henry Spencer wrote:

> We should be consistent:  either use the terminology that is common in
> news, or use the RFC 2822 terminology from mail.

1. RFC 2822 isn't just "mail". This has been discussed before.
2. the standard definition of "header" appears in FYI 18, which is
   certainly not jst "mail"

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: security considerations
Date: Fri, 10 Jun 2005 23:26:01 -0400
Cc: [email-masked]
Sender: [email-masked]

> > Or Content-Type: 
> >     multipart/mixed (; boundary=foo ; xyz=");bOuNdArY*=''next%20part(")
> 
> Yes, that addresses the RfC 2231 issue.  Of course it's abuse,
> 2231 clearly says "don't do this if it's unnecessary", and for
> a boundary it is unnecessary.
[...]

In this case since the boundary delimiter is English text it could have
been language-tagged ('en'next%20part).

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: #1031 Re: standard terminology and text
Date: Fri, 10 Jun 2005 23:21:03 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 10 2005 13:53, Frank Ellermann wrote:

> This <quote Bruce> "hacker slang" <unquote> is also defined
> in RfC 2110:

2110 is obsoleted, and the obsoleting RFC has a multitude of problems
(which we here are not chartered to solve -- byt we can and should
fix the problems in our deliverables).

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Newsgroups folding test
Date:  Sat, 11 Jun 2005 05:03:29 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 12
Sender: [email-masked]

Charles Lindsey wrote:

> From:  "Charles Lindsey" <[email-masked]>
> Subject: Testing folded newsgroups.
> Message-ID: <[email-masked]>

I don't see it on news.clara.net (about 0300Z), it arrived on
Google <http://purl.net/net/msgid/[email-masked]>
- but there you don't see the Path :-(

                          Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:21223
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Newsgroups folding test (was  CFWS in References header)
Date: Fri, 10 Jun 2005 19:11:45 GMT
Lines: 43
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> My understanding was that INN was now accepting folded Newsgroups
>> headers when relaying.

>Ah, yes, it's done that since 2.4.

OK, let's try it.

I have just submitted an article with the following headers to a server
using INN 2.4.1. It certainly got stored on that server. I invite you all
to see whether it arrived on any of the groups it was sent to, and
especially indications of how long it took and the route it took would be
interesting, since propagation using only INN 2.4+ servers may be rather
slow. If you do not see it in the named groups, it might be worth looking
to see if your server has filed it under "junk".

Newsgroups: man.test,
   uk.test,
   misc.test,
   de.test 
Path: chl
From:  "Charles Lindsey" <[email-masked]>
Subject: Testing folded newsgroups.
Message-ID: <[email-masked]>
Date: Fri, 10 Jun 2005 18:32:18 GMT
Lines: 14

It would also be helpful if others who have relaying privileges would try
the same experiment.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21222
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1030 Re: backward compatibility and handling incompatibilities (was #1022 Summary: Should dot be permitted in phrases?)
Date: Fri, 10 Jun 2005 18:53:52 GMT
Lines: 31
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>I have created ticket #1030 with this text.

>> I suggest we conform as closely as possible to our primary normative
>> references and published extensions, and that any deviations are
>> clearly indicated via the use of Implementation notes of the general
>> form suggested here by Mark Crispin:
>> � � http://www.imc.org/ietf-usefor/2003/Feb/0493.html
>> and that use of BCP 14 imperatives in conjunction with any such
>> deviations in fact conform to the usage requirements in BCP 14.

Although possibly useful in some situations, such a blanket solution is
unlikely to be helpful. Each case needs to be considered on its merits. If
you have specific examples to suggest, then we can do so.

But from the examples of Mark Crispin's Implementation notes that I have
seen, they tend to be of the form "This is how Mark Crispin views this
problem, and he does not wish to be confused by the facts adduced by those
who have actual experience of it".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21221
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: CFWS in References header
Date: Fri, 10 Jun 2005 18:44:59 GMT
Lines: 53
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Thu June 9 2005 13:04, Charles Lindsey wrote:

>> If that is correct, I would have no particular problem with it. But it is,
>> in essence, what the article-* drafts were trying to say (they were
>> certainly not saying "do whatever you please", any more than Mark was
>> saying that).

>There is a huge difference between documenting specific issues and
>making specific recommendations vs. leaving matters to the opinions
>of non-experts.

I am impressed by your classification of all news-server implementors as
"non-experts".

>Something similar could certainly be used where there are issues
>corresponding to what Mark calls "baby programmer errors", e.g.
>inability to handle msg-ids with a legal 255-octet domain name.  There
>is no good reason to be stuck with such an absurdity forever; implementers
>can and should be encouraged to fix the problems "with all due speed".

The restriction of msg-ids t0 250 characters is not a problem to be fixed
"with all due speed". It was extensively discussed by the NNTP working
group and it was discussed in this WG also.

It is now hard coded into the NNTP-draft which has just completed its IETF
Last Call (I note that you did not object to it).

It is a Done Deal.

>No assumption necessary; comments have always been legal in RFC 724/733/822
>structured fields -- any supposed "UA" that doesn't handle them properly is
>not conformant.  Some software is crappy; fact of life.  Absent other
>considerations, that's no reason to degrade specifications to the level of
>crappiness of the lowest common denominator.

Taking note of the lowest common denominator is, in practice, the way
things often work in the Real World. It is more politely expressed as
"being liberal".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2: What zones should be on the MUST accept list?
From: Graham Drabble <[email-masked]>
Date: Sat, 11 Jun 2005 00:48:28 +0100
Organization: Home
Sender: [email-masked]

On 10 Jun 2005 Russ Allbery <[email-masked]> wrote in
news:[email-masked]: 

> 
> Graham Drabble <[email-masked]> writes:

>> I came accross a whole bunch of articles that had the form:
> 
>> Tue, 26 Apr 2005 21:33:30 +Z
> 
>> This was due to a badly configured moderation bot so it was
>> affecting all the posts into the group.
> 
> They were so close.  Without the +, that would have actually been
> valid. 

Apparantly they were missing a '%' and had the wrong case Z.

The point that I was trying to make is that any software that is 
actually going to work on Usenet needs to be able to cope with pretty 
much anything in there. Trying to define what all the possible things 
that could be used is going to be futile so we are probably best off 
keeping the list short and saying something about how to handle the 
crap.

My suggestion is to have all letters as SHOULD NOT generate (there's no 
need for them, the numeric format works just as well if not better) and 
then don't explicitly say how to treat them. Wa can then say that any 
time with an unparceable offset (either because they use GMT, UT, EDT 
etc or because they've mangled it in some new way) should be treated as 
+0000. This means that GMT (which appears to be the most common) still 
gets parsed the same way and junk is handled consistently. 

-- 
Graham Drabble
http://users.ox.ac.uk/~sjoh1646/

To: [email-masked]
Subject: Re: Forward compatibility (Re: CFWS in References header)
From: Graham Drabble <[email-masked]>
Date: Sat, 11 Jun 2005 00:29:20 +0100
Organization: Home
Sender: [email-masked]

On 10 Jun 2005 "Charles Lindsey" <[email-masked]> wrote in
news:[email-masked]: 

> The problem is that (most) relaying agents need to examine this
> header during transit. To upgrade them to accept the folding is
> not rocket science, but it is only now beginning to get done. So
> at present such articles would not propagate well. 

How much of a problem is this going to cause big sites? I imagine the 
extra overhead of unfolding the header could be significant but have no 
data to go on.

-- 
Graham Drabble
http://users.ox.ac.uk/~sjoh1646/

Date: Fri, 10 Jun 2005 17:37:12 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> 
> 
> 
> --On fredag, juni 10, 2005 12:43:16 -0400 "Forrest J. Cavalier III" 
> <[email-masked]> wrote:
> 
>>> The Editors are well aware of many matters of form, boilerplate, 
>>> division
>>> of texts between USEFOR and USEPOR, and other nitpicks, etc. remain 
>>> to be
>>> done. But it is considered more important at the moment to get the
>>> technical issues correct, so that at least each of USEFOR, USEPRO and
>>> USEAGE can be made technically consistent.
>>>
>>
>> When can we expect to see them in the ticket system?  There is no sense
>> using an issues tracking system for only some of the important issues.
>>
>> 80% of the benefit comes only by ensuring that it accurately represents
>> all open issues.
> 
> 
> I create tickets as I see specific issues raised on the mailing list.
> There's no sense in raising a ticket to say that "organization" should 
> really be spelled "organisation" (or vice versa).

Being only 9 weeks out from a deadline in a group that works this slowly,
means there is no reason for delay in finding and fixing even the little
stuff.  (Or do you admit the 9 week deadline is a deathmarch anyway?)

Yes, creating a ticket can be more work than fixing a problem, but if
you STILL don't want to fix the problem now, then you have to create a
ticket.  When you use a Quality Control system only half way, you get
all the costs, and nearly none of the benefits.  Don't bother.

If you don't want lots of little tickets, can I can suggest to create
"Meta" tickets for things you consider "nuisance", and keep tacking
on everything you can think of as amendments.  It would be less work
than having multiple people raise the same issue, and have to respond
3 times with "Oh, that's not in the ticket system, but trust us,
we'll get to it.")

If you don't have everything in the ticket system, there is no way to
use it to get an idea of the work remaining.  If someone had everything in
the ticket system, and saw how much work on average to close each
issue when you use one-man one-vote consensus, you'd realize that 9 weeks
is a deathmarch.  Or it isn't.  How can you tell?

Further, are you certain that these things you want to postpone to cleanup
are merely nuisance?

1. If there are any documents misclassified as normative/informative,
or there are wrong cross-references, that isn't a nuisance problem.
Both change how you read the document, and can't be that hard to
get right.

2. You are asking volunteers to review this work.  They will stop when
they see problems like that, because they know it isn't ready for review.

(Also, I agree with John's comments about the mickey-mouse self-signed
SSL cert.  Doesn't everyone get all the phishing attempts that I
get. I am very careful about accepting CAs.)

I also agree with John's comments that the ticket system must not be
used for discussion.  It's just an accounting system.

Date: Fri, 10 Jun 2005 17:14:53 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

Charles Lindsey wrote:

> 7.6.1.  Construction of the References header
> 
>    The following procedure is to be used whenever some previous article
>    (the "parent") is to be referred to in the References header (F-
>    3.2.1) of a new article, whether in the course of generating a
>    followup or for some other reason (e.g. the later parts of a
>    multipart posting such as a FAQ, or the later parts of a
>    message/partial as suggested in [RFC 2046]).

1. This text appears under "duties of a followup agent."  You have not
shown text in USEPRO that lets non-followups have a references header,
unless they were generated by a followup agent.

Non-followups cannot be generated by a followup agent, because follow up agent is
defined as software used for preparing and posting followups.

I raised this issue at the time of the poll, and was told that the
definition of followup was getting set, and my concerns would be
addressed within the revisions to USEPRO.

I find that my concerns were not addressed at all.

2. Since the semantics of the header were deleted from USEFOR, and the
semantics do not appear in this proposed text for USEPRO, there is
no guidance on how user agents should detect or treat non-followups
that have References headers.

>         NOTE: There is no provision in this standard for an article to
>         have more than one parent. The essential property of the
>         References header, guaranteed by the procedure above and to be
>         preserved in any future extension, is that no article can ever
>         precede one of its own parents.

3. Specifying what something is not is by no means a definition of "The essential
property" of anything.

4. RFC2119 language is not called for and is not required.  (E.g. if there
were a specification of the semantics References header, all the "MUST"
formulations are not needed.  The draft should specify that the appearance
of a references header means "blah blah" and then the implementors will
know how to do the right thing.  Specify MUST constructions and it becomes
ambiguous...

5. What should an implementor of a user Agent make of the 7.6
language regarding the References header?  What can that implementor
assume about articles created by agents that pre-date the draft?  Or
is there a flag day?  What can they assume about a multi-part FAQ?

Date: Fri, 10 Jun 2005 17:04:40 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Newsgroup names #1021
Sender: [email-masked]

On Fri, 10 Jun 2005, Russ Allbery wrote:
> The problem is that those articles probably shouldn't propagate in a group
> named "junk" and junk will often be full of articles that weren't posted
> to that group because of what junk is for.  So it shouldn't be treated the
> same as a normal newsgroup.

However, this is *largely* an internal issue within a news server, not
something that appears "on the wire".  It's reasonable for us to try to
stay out of the way of such internal conventions, but exactly how they
work is not our problem.

Reserving a few name components, and discouraging single-component names,
is not an unreasonable thing to do as part of USEFOR, but trying to do it
in the grammar is a mistake.  Even the most grammar-intensive RFCs have
accompanying text which adds restrictions and constraints.

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 10 Jun 2005 14:02:53 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:
> "Forrest J. Cavalier III" <[email-masked]> wrote:

>> When can we expect to see them in the ticket system?  There is no sense
>> using an issues tracking system for only some of the important issues.
>> 
>> 80% of the benefit comes only by ensuring that it accurately represents
>> all open issues.

> I create tickets as I see specific issues raised on the mailing list.
> There's no sense in raising a ticket to say that "organization" should
> really be spelled "organisation" (or vice versa).

Agreed.  Let's not go overboard here.  It's pointless to spend much time
trying to fix typos and the like at this stage; we should do one pass once
the draft is in reasonably final shape.

To: [email-masked]
Subject: Re: Newsgroup names #1021
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 10 Jun 2005 13:55:00 -0700
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:
> Russ Allbery wrote:

>> In practice, people post to control and junk directly, but this often
>> doesn't really do what one wants

> On GMaNe posting to "junk" is my way to check a problem, if it appears
> immediately the server is fine, and mail2news is in trouble - I do this
> only if a test in gmane.test failed.

The problem is that those articles probably shouldn't propagate in a group
named "junk" and junk will often be full of articles that weren't posted
to that group because of what junk is for.  So it shouldn't be treated the
same as a normal newsgroup.

>> Creating groups by that name with control messages is a bad idea and
>> shouldn't happen.

> Yes, but these details are IMHO a case for USEPRO.  Trying to exclude
> single-component names in the USEFOR ABNF isn't the best way to address
> it.

I think I agree.

>> hard to work around if you want to retain the standard spool structure

> Adding one special character like "#" to the article numbers would avoid
> all potential problems.

That's not retaining the standard spool structure, and doing that will
break software that expects the standard spool structure.  As soon as you
deviate from the standard spool structure there are innumerable ways of
fixing the problem, but that's not the point.

>> alt.2600 is the most famous example -- it gets away with this even on
>> older news servers since there's no newsgroup named "alt".

> Now that's bad, if our syntax says "not only digits" it will be used
> elsewhere, e.g. in my "minimalistic" variant of the two news-nntp-uri
> drafts.

Right, that's why it's an arguable case.

>> We're reserving it for the typical IETF "use in example" reasons.  That
>> will be new with our draft.

> Was #1021 your ticket ?  RT only says that Alexey added it.

No.

But example had been in our draft for eons, I thought.  If not, my bad
memory; we've certainly talked about it lots before.

Date: Fri, 10 Jun 2005 22:53:13 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: "Forrest J. Cavalier III" <[email-masked]>, [email-masked]
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
Sender: [email-masked]

--On fredag, juni 10, 2005 12:43:16 -0400 "Forrest J. Cavalier III" 
<[email-masked]> wrote:

>> The Editors are well aware of many matters of form, boilerplate, division
>> of texts between USEFOR and USEPOR, and other nitpicks, etc. remain to be
>> done. But it is considered more important at the moment to get the
>> technical issues correct, so that at least each of USEFOR, USEPRO and
>> USEAGE can be made technically consistent.
>>
>
> When can we expect to see them in the ticket system?  There is no sense
> using an issues tracking system for only some of the important issues.
>
> 80% of the benefit comes only by ensuring that it accurately represents
> all open issues.

I create tickets as I see specific issues raised on the mailing list.
There's no sense in raising a ticket to say that "organization" should 
really be spelled "organisation" (or vice versa).

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Guaranteed MUSTard (was: #1008 USEFOR 1.5 References: Final text on "followup")
Date:  Fri, 10 Jun 2005 22:39:19 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 68
Sender: [email-masked]

Charles Lindsey wrote:

 {USEFOR]
> references  = "References:" SP [CFWS] msg-id *(CFWS msg-id )
>               [CFWS] CRLF

ACK.  I'd prefer a msg-id-list version for aesthetical reasons,
same idea as for <newsgroup-list> or <path-list>:

  references  = "References:" SP msg-id-list CRLF
  msg-id-list = [CFWS] *(msg-id CFWS) msg-id [CFWS]

In that order the concept of at least one last msg-id is clear,
reflecting the construction algorithm as good as possible.

 [USEPRO]
> The followup MUST (in accordance with the definition of that
> term) have a References header referring to its precursor,
> constructed in accordance with section 7.6.1 below.

>   NOTE: That "MUST" is to be contrasted with the weaker
>   recomendation using "SHOULD" applied, in [RFC 2822], to the
>   generation of "replies" in email. Moreover, in Netnews,
>   there is no expectation of any In-Reply-To header in a
>   followup.

I'd replace this convoluted MUSTard by a simple statement:

  The followup has a References header referring to its
  precursor, constructed in accordance with section 7.6.1
  below.

> in the course of generating a followup or for some other
> reason (e.g. the later parts of a multipart posting such
> as a FAQ, or the later parts of a message/partial as
> suggested in [RFC 2046])

I'd remove these irrelevant examples where other algorithms
might also work, the most important "other reason" is a reply,
not limited to Followup-To: poster.  So that would be:

   in the course of generating a followup or for some other
   reason (e.g. a mail reply).

> NOTE: There is no provision in this standard for an article
> to have more than one parent.

Then don't talk about it, or add the In-Reply-To blurb here:

  NOTE: There is no provision in this standard for an article
  to have more than one parent.  In Netnews, there is no
  expectation of any In-Reply-To header in a followup

> The essential property of the References header, guaranteed
> by the procedure above and to be preserved in any future
> extension, is that no article can ever precede one of its
> own parents.

There is no "guarantee", this is bad advice for implementors.

My UA cannot handle self-referencing articles, and Harald has
reported another case, where this "guarantee" simply crashed.

The essential property of the References is a msg-id-list, and
it's unnecesary to repeat it in USEPRO.

                             Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Newsgroup names #1021
Date:  Fri, 10 Jun 2005 21:37:52 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 68
Sender: [email-masked]

Russ Allbery wrote:

> In practice, people post to control and junk directly,
> but this often doesn't really do what one wants

On GMaNe posting to "junk" is my way to check a problem, if
it appears immediately the server is fine, and mail2news is
in trouble - I do this only if a test in gmane.test failed.

> Creating groups by that name with control messages is a
> bad idea and shouldn't happen.

Yes, but these details are IMHO a case for USEPRO.  Trying
to exclude single-component names in the USEFOR ABNF isn't
the best way to address it.

> not really a protocol issue; it's more a USEAGE issue.

Also fine, so it doesn't affect USEFOR.

 [component name 1*DIGIT]   
> necessary to retain backward compatibility with any news
> server that uses the traditional directory structure for
> the news spool, since the group example.foo.600 and article
> 600 in example.foo both want to use the same file name.

Okay, then we need some ldh-string magic to exclude 1*DIGIT.
We could use the ambiguous grammar style as for <id-quote>:

   c-name = *c-text c-okay *c-text
   c-text = DIGIT / c-okay
   c-okay = ALPHA / "-" / "_"

> hard to work around if you want to retain the standard spool
> structure

Adding one special character like "#" to the article numbers
would avoid all potential problems.  Actually I don't care how
we solve it, but don't mention "14" or I'd scream... <g>

> alt.2600 is the most famous example -- it gets away with
> this even on older news servers since there's no newsgroup
> named "alt".

Now that's bad, if our syntax says "not only digits" it will
be used elsewhere, e.g. in my "minimalistic" variant of the
two news-nntp-uri drafts.

>>  From a user's POV control.all behaves like r/o NGs.

> On what server are you seeing control.all?

Sorry, I meant "stuff below control", e.g. control.cancel.de
 
> .all is a wildcard in C News and is equivalent to .*

Yes, that's exactly what I meant.

>> I've never heard of "example" being reserved.
 
> We're reserving it for the typical IETF "use in example"
> reasons.  That will be new with our draft.

Was #1021 your ticket ?  RT only says that Alexey added it.

If that's a new idea it's fine - I forgot the URL with the
unofficial "TLH registry", so I hope you checked this.  Bye

Date: Fri, 10 Jun 2005 21:21:06 +0200
From: Eivind Tagseth <[email-masked]>
To: [email-masked]
Subject: Re: Admin: Please watch your language
Mail-Followup-To: Eivind Tagseth <[email-masked]>, [email-masked]
Sender: [email-masked]

* Harald Tveit Alvestrand <[email-masked]> [2005-06-10 15:09:45 +0200]:

> Your language is insulting to the editors

I failed to see any insulting language in that message.  I failed to see
anything wrong with it at all...

Eivind

Date: Fri, 10 Jun 2005 12:03:16 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Admin: Please watch your language
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>Your language is insulting ...
>If you continue in this vein, I will remove your posting rights to this 
>list.

I found Bruce's summary of the situation to be remarkably correct and a 
lot less insulting that some of the other participants who continue 
unwarned.

To: [email-masked]
Subject: Re: Forward compatibility (Re: CFWS in References header)
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 10 Jun 2005 12:02:46 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> IMO, Newsgroups folding would be very useful. Newsgroups headers in
> excess of 80 characters are very common. They display badly in many user
> agents; e.g. there is just a '!' to indicate "rest of line truncated",
> or they are are wrapped arbitrarily in the middle of a newsgroup (or
> between newsgroups if you are lucky).

Surely this is a quality of implementation issue.  It's trivial to wrap
the newsgroups header for display; it's not like it's hard to parse.  My
user agent does this without any difficulty.

> Even where they are displayed "properly" folded, the user is given the
> impression that it would be OK to generate new articles like that, and
> copy/paste from what is displayed to a new article might or might not
> result in a valid header.

Surely this too is a quality of implementation issue; my user agent
removes folding and whitespace from the header when posting.

> Moreover, this is the only header, within News or Mail, that cannot be
> folded, so it sticks out like a sore thumb.

Er, no.  Message-ID cannot be folded, Control cannot be folded, Date will
probably break in practice if folded, Followup-To similarly cannot be
folded, and that's just off the top of my head.  And I'm still concerned
that folding Path might not actually work in practice, although it does
work with the software I personally have checked.

> Note that the problem is with relaying agents. I think all current user
> agents would accept and display them properly.

I'm pretty sure you're wrong.

> Therefore, I wish to see as much incentive as possible to get relaying
> agents upgraded. MUST accept in our draft is a start. SHOULD NOT
> generate would give more incentive than MUST NOT generate, and something
> even weaker (as in the article-* drafts) would give more incentive
> again.

If a new RFC differs from existing practice by too much, more requirements
that differ from existing practice cease to be motivation to change
software and become motivation to just ignore the document.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: security considerations
Date:  Fri, 10 Jun 2005 20:51:32 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 48
Sender: [email-masked]

Bruce Lilly wrote:
 
>> The security problem is the simple / naive scanner supposed
>> to catch this malware _before_ it meets any software from
>> Redmond.
 
> Yes, because it botched MIME parsing.

In a simple script looking at the first 50 lines of the body I
decided to use a heuristics instead of proper parsing for the
question "is there (maybe) a B64 part starting with TV or UE ?"

In that case it would delete the rest of the mail, that's not
completely uncritical, and "popstop.cmd" is a published script.
 
> The parsing issue applies in general, and broken "parsing"
> leads to problems.

Or to heuristics.  The problem with RfC 2046 is that it doesn't
mention these issues.

>> maybe an example:
>> boundary (not=me) = ("yes ;-) simple (foo;bar") ; charset=utf-8

Bad example, a multipart has no charset, corrected version at
<http://article.gmane.org/gmane.ietf.usenet.format:28712:raw>

> Or Content-Type: 
>     multipart/mixed (; boundary=foo ; xyz=");bOuNdArY*=''next%20part(")

Yes, that addresses the RfC 2231 issue.  Of course it's abuse,
2231 clearly says "don't do this if it's unnecessary", and for
a boundary it is unnecessary.  But they forgot a very critical
MUST NOT in 2046 / 2231.  And with both examples it should be
clear that it's not only a 2231 problem.

The complete line is too long (81), the body (67) is okay, we
could fold it in the "security considerations":

Content-Type: 
 multipart/mixed (; boundary=foo ; xyz=");bOuNdArY*=''next%20part(")
Content-Type: multipart/mixed;
 boundary (not=me) = ("yes ;-) simple (foo;bar") ; x-foo = xyzzy

Only the puzzle, no solution, I want readers to think about it.

                    Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:21204
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Forward compatibility (Re: CFWS in References header)
Date: Fri, 10 Jun 2005 16:42:41 GMT
Lines: 46
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>IMHO, and speaking for myself-as-participant only:

>Realistically, either newsgroups folding is useful or it is not.
>If it is not, there is no incentive to implement it no matter what the 
>standard says.
>If it is useful, there is incentive to implement it.
>If it is very useful, someone might even be bothered to write an 1-page RFC 
>saying "Newsgroup folding is now permitted".

IMO, Newsgroups folding would be very useful. Newsgroups headers in excess
of 80 characters are very common. They display badly in many user agents;
e.g. there is just a '!' to indicate "rest of line truncated", or they are
are wrapped arbitrarily in the middle of a newsgroup (or between
newsgroups if you are lucky). Even where they are displayed "properly"
folded, the user is given the impression that it would be OK to generate
new articles like that, and copy/paste from what is displayed to a new
article might or might not result in a valid header.

Moreover, this is the only header, within News or Mail, that cannot be
folded, so it sticks out like a sore thumb.

The problem is that (most) relaying agents need to examine this header
during transit. To upgrade them to accept the folding is not rocket
science, but it is only now beginning to get done. So at present such
articles would not propagate well. Note that the problem is with relaying
agents. I think all current user agents would accept and display them
properly.

Therefore, I wish to see as much incentive as possible to get relaying
agents upgraded. MUST accept in our draft is a start. SHOULD NOT generate
would give more incentive than MUST NOT generate, and something even
weaker (as in the article-* drafts) would give more incentive again.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21205
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Date: Fri, 10 Jun 2005 17:13:05 GMT
Lines: 124
Sender: [email-masked]

In <00DF49EE2D14EBC47C06773D@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>In reviewing the thread, the text that seems to come closest to capturing 
>the (VERY) rough consensus of the group is:

> A "followup" is an article containing a response to the contents of
> an earlier article, its "precursor". Every followup includes a
> References header identifying that precursor (but note that non-followup
> articles may also use a References header).

>I have also noted that there is no discernible consensus for the text 
>Charles suggested addition to the top of the References section (attempting 
>to explain what References is used for); it will not be added as part of 
>this decision.

I regret that there will now be no introductory text to the References
header (whether mine, or some other variant).

I also regret that there will be no clear statement of the semantics of
this header, other than what can be gleaned from the process for
constructing it, as set out in USEFOR (Forrest was anxious to have such
semantics included). I would have suggested a text such as the following:

|  The list is composed of message identifiers of ancestors of the
|  current article, sorted so that no article precedes any of its own
|  ancestors. It SHOULD include both the oldest ancestor and the
|  immediate parent of the current article, even if some of the
|  intermediate ones are omitted.

>Unless there are technical issues that have not been raised before, this is 
>the instruction from the WG chairs to the editor of the USEFOR document, 
>and the subject of the definition of "followup" is now closed.

In that case, I believe the following represents the definitive result of
our deliberations:

Definitions (for inclusion in USEFOR or USEPRO as appropriate).

   A "followup" is an article containing a response to the contents of an
   earlier article, its "precursor". Every followup includes a References
   header identifying that precursor (but note that non-followup articles
   may also use a References header).    

   A "followup agent" is a combination of reading agent and posting
   agent that aids in the preparation and posting of a followup.

Within USEFOR:

3.2.1  References 

   The References header is the same as that specified in Section 3.6.4
   of [RFC2822] with the added restrictions detailed in Section 2.2 and
   those listed below:

   o  The updated <msg-id> construct defined in Section 3.1.3 MUST
      be used.

   references      =  "References:" SP [CFWS] msg-id *(CFWS msg-id )
                      [CFWS] CRLF

|  The process of generating a References header is set out in [USEPRO].

Within USEPRO:

7.6.  Duties of a Followup Agent

.......

   4. The followup MUST (in accordance with the definition of that term)
      have a References header referring to its precursor, constructed
      in accordance with section 7.6.1 below.

        NOTE: That "MUST" is to be contrasted with the weaker
        recomendation using "SHOULD" applied, in [RFC 2822], to the
        generation of "replies" in email. Moreover, in Netnews, there is
        no expectation of any In-Reply-To header in a followup.
 
7.6.1.  Construction of the References header

   The following procedure is to be used whenever some previous article
   (the "parent") is to be referred to in the References header (F-
   3.2.1) of a new article, whether in the course of generating a
   followup or for some other reason (e.g. the later parts of a
   multipart posting such as a FAQ, or the later parts of a
   message/partial as suggested in [RFC 2046]).

   If the parent did not have a References header, the content of the
   new article's References header MUST be inherited from that of the
   Message-ID header of the parent, otherwise (there already was a
   References header) it MUST be comprised of the parent's References
   header (subject to trimming as described below) followed by CFWS and
   the Message-ID header content of the parent.

   If the resulting References header is excessively long it MAY (and if
   its total length, whether folded or not, exceeds 998 chacacters it
   MUST) then be trimmed, but the first and the last two message
   identifiers MUST NOT be removed.

        NOTE: There is no provision in this standard for an article to
        have more than one parent. The essential property of the
        References header, guaranteed by the procedure above and to be
        preserved in any future extension, is that no article can ever
        precede one of its own parents.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21203
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
Date: Fri, 10 Jun 2005 16:22:50 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>--On torsdag, juni 09, 2005 16:14:05 +0000 Charles Lindsey 
><[email-masked]> wrote:

>> I think everyone agrees MUST accept.
>>
>> The issue is SHOULD NOT vs MUST NOT for generate.

>Charles, may I remind you again:

>the current wording (2.1) is MAY accept.

>What wording change do you propose?

In relation to the particular matter of <obs-phrase>, I wish to see MUST
accept.

I could live with either of SHOULD NOT or MUST NOT for generate.

My preference is that we should use SHOULD NOT/MUST NOT consistently for
the three cases of <obs-phrase>, <obs-zone> (or whatever part of it we
adopt) and Newsgroups folding, if it is possible to do so (it might not
be, because their priorities are somewhat different).

If you press me for an explicit choice, then it would be for SHOULD NOT in
all three cases.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21206
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1031 backward compatibility and handling incompatibilities
Date: Fri, 10 Jun 2005 18:22:48 GMT
Lines: 46
Sender: [email-masked]

In <[email-masked]> Henry Spencer <[email-masked]> writes:

I hope I have attached the proper ticket number to this reply.

>On Thu, 9 Jun 2005, Bruce Lilly wrote:
>> I suggest we conform as closely as possible to our primary normative
>> references and published extensions...
>> The following sections need to be changed because of that - details
>> to be discussed when the basic change is accepted...

>Personally, I find it impossible to evaluate this suggestion without
>more detail.  The suggestion as phrased is nearly meaningless, given
>that there is no clear indication of what is to be changed or why.

I think he is upset because we use the term "header" rather than "header
field" and "header name" rather than "field name".

I would point out that our drafts have always used this terminology (and
have carefully documented that usage).

RFC 1036 used the term "header" (though not consistently, since it
sometimes used "header line" - a quick look revealed no occurrence of
"field", though I have not grepped to confirm it).

Son-of-1036 used the term "header".

The NNTP draft, which has just completed its IETF Last Call period, uses the
term "header".

I move that we make no change to this long established practice.

NOTE that there is a technical problem with the term "header content" as
defined in USEFOR. I have suggested to Ken that he changes it to "header
body" throughout USEFOR, and I will define a term "semantic content"
(abbrevieted to "content") to cover the way that term is used in USEPRO.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Fri, 10 Jun 2005 11:12:25 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Newsgroup names #1021 (was: #1008 USEFOR 1.5 References: Final text on "followup")
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>This was an issue that Charles raised off-list, I believe.

This is the IETF working group tasked with news. There is no "off-list"
issue, either it comes here or it isn't.

>Good point - if filing tickets in response to issues raised off-list, we 
>need to make sure the WG is notified.....

You are certainly kidding, aren't you? You are actually putting more 
emphasis on this trouble ticket system than on the working group? Gee,
MAYBE we ought to mention trouble tickets on the working group that is 
supposed to be dealing with them? 

>The following comment was added by someone logged in as "ietf":

That's how you told people to access the tickets. Is it odd that they
do it the way you told them? And that they way you told them is 
essentially anonymous?

Since I pointed out this problem in private and never got any resolution 
to it, I'll now bring it up in public. 

Why is this "trouble ticket" system using HTTPS with bogus certificates? 
This prevents participation by those who refuse to accept bogus certs. Or 
perhaps that was the goal?

There is no reason for https. IETF discussions are supposed to be public.  
Whatever you post there is supposed to be available to anyone who wants to
see it. There is no security issue: Harald published the username/password
pair we are supposed to use to access the site. Why https?

But more important, WHY does the trouble ticket system exist at all? If
Harald wants to keep a tickler file of issues he thinks needs to be
resolved, that's fine. If he wants to publish this list, fine. 

When that list takes on the task of discussing the issues, not so fine.  
If Charles's issue is worth discussing, it is worth discussing here. This
"trouble ticket"  system is a violation of, if not the letter, the spirit
of IETF working groups, not only in theory, but now in practice. There
should never be a question of "should we tell the WG about this issue
we're discussing on this trouble ticket system?" Never. It should never be
possible to have that question come up.

Since it has now been demonstrated that the question has to be asked, it
is time to shut down the trouble ticket system and bring discussion back
home. Holding discussions in two places, one of which has limited access,
is wrong. 

And then I run across this gem in "Friday - Ticket status":

>1002 USEFOR 6. References - Updates needed
> No discussion; ticket seems clear.

References has been discussed at length IN THIS LIST, but now that there
is no discussion in the "trouble ticket" system, the answer is clear.

The only thing that is clear is that this trouble ticket system is being 
used to bypass the standard IETF working group process. Either that, or it 
creates the marvelous illusion that it is being used to bypass the 
process, and either situation is unacceptable.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1031 Re: standard terminology and text
Date:  Fri, 10 Jun 2005 19:53:02 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> we're left with whether "header" means one RFC 2822 header
> field (as in the current draft) or the entire part of the
> article before the blank line (as in RFC 2822).

This <quote Bruce> "hacker slang" <unquote> is also defined
in RfC 2110:

|  Header    Field in a message or content heading specifying
|            the value of one attribute.

|  Heading   Part of a message or content before the first
|            CRLFCRLF, containing formatted fields with
|            attributes of the message or content.

In real life nobody says "header field", only we sometimes do,
because we are arrogant pseudo-scientific twits.  URI vs. URL
is another telltale.
                        Bye, Frank

Date: Fri, 10 Jun 2005 13:22:39 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1031 Re: standard terminology and text
Sender: [email-masked]

Henry Spencer wrote:

> I favor keeping the terminology that's in common use in news software and
> news documentation.  The divergence from RFC 2822 is regrettable but it's
> too late to fix it.  Changing to RFC 2822 terminology will increase the
> readability of our document for people coming from the mail world, but
> will reduce it for people already accustomed to the news world.  I submit
> that for this document, the latter are more important.

"More important" based on what criteria?

If RFC2822 were an informative reference, inconsistency might be understandable.

But since it is a base reference, there is NO WAY to read the USEFOR documents
without understanding RFC2822 first.

So that means that EVERYONE is "coming from email world."

Google searches indicate
    14,300 references for the term "RFC1036"
    45,300 for "RFC2822"
   997,000 for "RFC822"

Counting google references isn't a precise measure of anything.  But it
says something.

To: [email-masked]
Subject: Re: Newsgroup names #1021
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 10 Jun 2005 10:05:15 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> This was an issue that Charles raised off-list, I believe.  Good point -
> if filing tickets in response to issues raised off-list, we need to make
> sure the WG is notified.....

> The text of the ticket is the following:
> ------------------------------
> I suggest to update the Newsgroup ABNF as follows:
> 1). uppercase letters must be prohibited in newsgroup names

It's not completely clear to me whether this is the right thing to do,
since there are some uses of mixed-case newsgroup names in the wild
already and they don't actually break anything beyond making things more
confusing.  The ftp.isc.org list refuses to add them, but the protocol
actually has historically supported them.

Sorry, I should have mentioned that earlier.

> 2). single component newsgroups must be prohibited (this also
> addresses conflict with the special value "poster")

I think they should be allowed; "poster" and "junk" just have to be
reserved.  (Getting the wording right for "junk" without going into what
"junk" is used for is going to be a bit tricky.)

> 3). prohibiting all digit components can be done in ABNF, but it would
> be ugly. I suggest to address this with an ABNF comment.
> 4). prohibiting components "ctl"/"all" and newsgroups starting
> with "control", "example" or "to" can also be done in ABNF, but that
> would be even uglier, as there is no way to exclude something in ABNF
> (unless I am mistaken). So I suggest to add another comment(s).

These both sound good to me.

>> 2). single component newsgroups must be prohibited (this
>> also addresses conflict with the special value "poster")

>  But not with pseudo-groups like "junk" or "control", and
> the real NG "spamcop" <news://news.spamcop.net/spamcop>

It depends on how one prohibits them, basically.  Definitely, news readers
have to be able to read control and junk, so it has to be able to appear
in the Xref header.  In practice, people post to control and junk
directly, but this often doesn't really do what one wants and I'm not sure
it's a good idea to allow in the standard because the behavior is somewhat
undefined.  Crossposting to those groups is generally fairly harmless,
just confusing.  Creating groups by that name with control messages is a
bad idea and shouldn't happen.

Other single-component newsgroup names work fine at all levels of the
protocol.  They just don't work with the most common control.ctl patterns
and are frowned on because they don't fit into a hierarchy as cleanly as,
say, spamcop.general or the like.  That's not really a protocol issue;
it's more a USEAGE issue.

>> 3). prohibiting all digit components can be done in ABNF,
>> but it would be ugly.

>  Ugly if necessary is acceptable, but why is it necessary ?

It's necessary to retain backward compatibility with any news server that
uses the traditional directory structure for the news spool, since the
group example.foo.600 and article 600 in example.foo both want to use the
same file name.  Many older news servers will essentially crash if one has
created example.foo.600 and article 600 arrives in example.foo.

Now, one could argue that this is just bad data structures and it's up to
the server to work around this rather than putting this into the protocol,
and I have a lot of sympathy for that argument.  It's just hard to work
around if you want to retain the standard spool structure because normally
the part of the server that numbers articles and the part of the server
that stores them don't have a two-way conversation, and there's a fair bit
of existing software out there that just isn't ever going to cope.

This has been one of the Usenet newsgroup naming rules for many years,
although it's honored in the breech in a few places.  (alt.2600 is the
most famous example -- it gets away with this even on older news servers
since there's no newsgroup named "alt".)

>> 4). prohibiting components "ctl"/"all" and newsgroups starting with
>> "control", "example" or "to" can also be done in ABNF

>  From a user's POV control.all behaves like r/o NGs.

On what server are you seeing control.all?

.all is a wildcard in C News and is equivalent to .*; it's used in feed
pattern specifications.

> On one server I know I "see" control.cancel.de (etc.) for cancels in
> various TLHs.

control.* should be reserved for the local implementation, not reserved in
the sense that it never appears or cannot be read.  Basically, the only
thing you should be forbidden to do is send control messages about it, or
possibly post to it (and I'm very iffy on the latter).

> I've never heard of "example" being reserved.

We're reserving it for the typical IETF "use in example" reasons.  That
will be new with our draft.

> And pseudo-TLH "to" is some mail-by-news trick, from a user's POV like
> an NG.

The to.* newsgroups have extremely special handling with older news
software and were used for point-to-point UUCP feeds and similar sorts of
tricks.  They are special-cased in numerous places in INN and I believe
also in C News, and using them for any normal purposes will make you sad.
This is another case where the client probably gets to read them, and in
this case even post to them, but it should be made clear that they're
special and may do strange things.  (We should probably expand at some
point on just what the strange things are that they're intended to be used
for.)

Date: Fri, 10 Jun 2005 12:50:29 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1031 Re: standard terminology and text
Sender: [email-masked]

On Fri, 10 Jun 2005, Harald Tveit Alvestrand wrote:
>    This specification uses the terms "header", "header name", and
>    "header content" which are synonymous with the [RFC2822] terms
>    "header field", "field name", and "field body" respectively.
> 
> "header name" occurs once.
> "header content" occurs three times.
> Is there anyone who objects to changing those to "field name" and "field 
> body"?

We should be consistent:  either use the terminology that is common in
news, or use the RFC 2822 terminology from mail.  Changing *some* of the
terms will gain us nothing but confusion.  The fact that "header name" and
"header content" see little use doesn't mean they can be changed without
penalty; what it means is that keeping them consistent with any decision
about "header" is not a problem.  They should be changed if, and *only*
if, "header" is changed. 

> If that's OK, we're left with whether "header" means one RFC 2822 header 
> field (as in the current draft) or the entire part of the article before 
> the blank line (as in RFC 2822).

I favor keeping the terminology that's in common use in news software and
news documentation.  The divergence from RFC 2822 is regrettable but it's
too late to fix it.  Changing to RFC 2822 terminology will increase the
readability of our document for people coming from the mail world, but
will reduce it for people already accustomed to the news world.  I submit
that for this document, the latter are more important.

                                                          Henry Spencer
                                                       [email-masked]

Date: Fri, 10 Jun 2005 12:43:16 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
Sender: [email-masked]

Charles Lindsey wrote:

> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> 
>>On Thu June 9 2005 09:40, Harald Tveit Alvestrand wrote:
> 
> 
>>>Then make multiple suggestions, or say "I suggest we make this change, and 
>>>the following sections need to be changed because of that - details to be 
>>>discussed when the basic change is accepted".
> 
> 
>>I suggest that no later than the publication of the next draft revision,
>>the editors carefully spell-check, proofread, and edit for consistency
>>the draft (e.g. section cross-references are correct, references listed
>>in the references sections are actually referenced, informative/normative
>>references are categorized correctly, etc.).
> 
> 
> The Editors are well aware of many matters of form, boilerplate, division
> of texts between USEFOR and USEPOR, and other nitpicks, etc. remain to be
> done. But it is considered more important at the moment to get the
> technical issues correct, so that at least each of USEFOR, USEPRO and
> USEAGE can be made technically consistent.
> 

When can we expect to see them in the ticket system?  There is no sense
using an issues tracking system for only some of the important issues.

80% of the benefit comes only by ensuring that it accurately represents
all open issues.

To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2: What zones should be on the MUST accept list?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 10 Jun 2005 09:38:14 -0700
Sender: [email-masked]

Graham Drabble <[email-masked]> writes:
> On 09 Jun 2005 Russ Allbery <[email-masked]> wrote:

>> Hopefully this will give people a new appreciation for just how
>> much bizarre junk is out there on the wire.

> I came accross a whole bunch of articles that had the form:

> Tue, 26 Apr 2005 21:33:30 +Z

> This was due to a badly configured moderation bot so it was affecting 
> all the posts into the group.

They were so close.  Without the +, that would have actually been valid.

> FWIW this doesn't seem to have affected propogation and my newsreader 
> (XNews) gets the date right and treats +Z as +0000. My stat gathering 
> tool which uses perl's Date::Parse broke.

INN treats all time zones it can't parse as the local time zone, rather
than +0000, just because it did that historically and when I wanted to
change it to +0000, again, people objected.

To: [email-masked]
Xref: clerew local.usefor:21194
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
Date: Fri, 10 Jun 2005 16:11:52 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Thu June 9 2005 09:40, Harald Tveit Alvestrand wrote:

>> Then make multiple suggestions, or say "I suggest we make this change, and 
>> the following sections need to be changed because of that - details to be 
>> discussed when the basic change is accepted".

>I suggest that no later than the publication of the next draft revision,
>the editors carefully spell-check, proofread, and edit for consistency
>the draft (e.g. section cross-references are correct, references listed
>in the references sections are actually referenced, informative/normative
>references are categorized correctly, etc.).

The Editors are well aware of many matters of form, boilerplate, division
of texts between USEFOR and USEPOR, and other nitpicks, etc. remain to be
done. But it is considered more important at the moment to get the
technical issues correct, so that at least each of USEFOR, USEPRO and
USEAGE can be made technically consistent.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Fri, 10 Jun 2005 16:16:05 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Friday - Ticket status
Sender: [email-masked]

I have the following open USEFOR tickets:

1002 USEFOR 6. References - Updates needed
  No discussion; ticket seems clear.

1003 USEFOR 3.1.3 - Cleanup ABNF for msg-id
  No discussion

1004 USEFOR 3.2.14 - Can we deprecate X- headers?
  A few notes, no clear resolution

1021 Newsgroups header ABNF and description needs cleanup
  No discussion

1022 USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
  Discussion sidetracked; no clear resolution

1028 USEFOR 3.1.2 Date: What zones should be on the MUST accept list?
  My impression is that only GMT should be a MUST accept, not UT.

1029 USEFOR 3.2.2 References: Should comments be a MUST NOT?
  My impression is that they are MUST NOT generate at present; people
  are sensitive to exact formulations here.

1030 USEFOR general: Backwards compatibility and handling incompatibilities
  No discussion, no text proposal

1031 USEFOR general: Standard terminology and text
  Under discussion

1032 USEFOR general: Document changes from RFC 1036
  Principle accepted. Method under discussion.

I also have a list of issues sent to me privately, which I have not seen on 
the list yet. I'll ask the source of these issues to raise the ones that 
are not strictly editorial on the list.

                     Harald

Date: Fri, 10 Jun 2005 16:00:31 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: #1031 Re: standard terminology and text
Sender: [email-masked]

After looing through draft-ietf-usefor-usefor-04, I've found that of the 
"redefined" terms:

   This specification uses the terms "header", "header name", and
   "header content" which are synonymous with the [RFC2822] terms
   "header field", "field name", and "field body" respectively.

"header name" occurs once.
"header content" occurs three times.

Is there anyone who objects to changing those to "field name" and "field 
body"?

If that's OK, we're left with whether "header" means one RFC 2822 header 
field (as in the current draft) or the entire part of the article before 
the blank line (as in RFC 2822).

(I'm somewhat unsure what a "header line" means - it isn't listed in the 
definitions, and is not used in 2822. I'm assuming that it's a line (as 
defined in RFC 2822) that occurs in the header (as defined in RFC 2822))

                            Harald

Date: Fri, 10 Jun 2005 09:24:14 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: Admin: Please watch your language
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> 
> Bruce,
> 
> this is a warning. On list and for the record. You were already warned 
> privately about insulting participants.
> 
> Your language is insulting to the editors, the chairs and the members of 
> the working group who do not agree with you - as well as being just 
> vague enough that it is not possible to figure out what exact text you 
> are objecting to, and what you want it to say instead.
> 
> If you continue in this vein, I will remove your posting rights to this 
> list.

Bruce's "language" was insulting only if you consider criticism insult.

You are welcome to remove the posting rights of anyone in order
to finish the document the way you think best.  If you remove Bruce's
posting rights because you don't like what he says, then I see little
point in my participating further.

Date: Fri, 10 Jun 2005 09:14:41 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1031 Re: standard terminology and text (was #1022 Summary: Should dot be permitted in phrases?)
Sender: [email-masked]

> Now, you seem to have a specific objection to:
> 
>   This specification uses the terms "header", "header name", and
>   "header content" which are synonymous with the [RFC2822] terms
>   "header field", "field name", and "field body" respectively.
> 

> Are you making a proposal for a specific text change to delete this 
> paragraph and change the remaining text to use the RFC 2822 synonyms? 
> That's an actionable item; "some of the changes are blatantly obvious" 
> is not.
> 
> (the word "header" occurs approx. 137 times in usefor-04. I think it's 
> makework to do the search-and-replace, but if I've interpreted you 
> correctly, you think differently. Please argue your case.)

In the future, hundreds of people will read these documents.  They
will be expected to be familiar with RFC2822.

Are you saying it is better to force every single one of those people
to learn new, pointless definitions just to understand Usefor, than have
one editor do 5 minutes of search and replace of "makework" now?

Is it OK to pretend that we are going to finish before RFC2822 gets
done?  They finished first, so we get to re-use what they have done,
not fight against it.

Date: Fri, 10 Jun 2005 15:09:45 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Bruce Lilly <[email-masked]>
Cc: Alexey Melnikov <[email-masked]>, [email-masked]
Subject: Admin: Please watch your language
Sender: [email-masked]

Bruce,

this is a warning. On list and for the record. You were already warned 
privately about insulting participants.

Your language is insulting to the editors, the chairs and the members of 
the working group who do not agree with you - as well as being just vague 
enough that it is not possible to figure out what exact text you are 
objecting to, and what you want it to say instead.

If you continue in this vein, I will remove your posting rights to this 
list.

                      Harald

--On fredag, juni 10, 2005 09:00:52 -0400 Bruce Lilly <[email-masked]> 
wrote:

> On Fri June 10 2005 02:30, Harald Tveit Alvestrand wrote:
>
>> After 8 years on the IESG reading drafts, I found nothing special about
>> draft-ietf-usefor-usefor-04 that would warrant the "don't resemble any
>> other IETF work-product" claim. So it seems you have different criteria
>> than mine.
>
> I can't recall seeing a draft other than the one under discussion and
> its predecessors that has anything like
> "When we say 'white' we mean black, and when we say 'black' we mean red,
> and when we say 'red' we mean green.  We won't tell you what we really
> mean when we say 'green' -- sometimes it might mean green and sometimes it
> might mean something else."
> Which is basically what that draft says (except it's not nearly as honest
> about the last part).

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1031 Re: standard terminology and text (was #1022 Summary: Should dot be permitted in phrases?)
Date: Fri, 10 Jun 2005 09:00:52 -0400
Cc: "Forrest J. Cavalier III" <[email-masked]>, [email-masked]
Sender: [email-masked]

On Fri June 10 2005 02:30, Harald Tveit Alvestrand wrote:

> After 8 years on the IESG reading drafts, I found nothing special about 
> draft-ietf-usefor-usefor-04 that would warrant the "don't resemble any 
> other IETF work-product" claim. So it seems you have different criteria 
> than mine.

I can't recall seeing a draft other than the one under discussion and
its predecessors that has anything like
"When we say 'white' we mean black, and when we say 'black' we mean red,
and when we say 'red' we mean green.  We won't tell you what we really
mean when we say 'green' -- sometimes it might mean green and sometimes it
might mean something else."
Which is basically what that draft says (except it's not nearly as honest
about the last part).
 
> > Why is it acceptable to require that Bruce rewrite the sections before
> > entering tickets for every one of those sections?  Bruce isn't editor.
[...]
> > If the chair and editor can't see the problems in the text now, then what
> > criteria will be used to evaluate any rewritten versions?  Just curious.
> 
> I wonder about that too.

That's a strange remark.  I think it's reasonable to state what the
criteria for evaluating comments and determining consensus are before
asking people to jump through ever higher hoops.  For reference, a past
WG Chair had the following to say, and if I recall correctly, Alexey
has agreed:
  On the topic of voting: Especially on this topic, I think voting 
  would be stupid. The decision is about rough consensus. One screaming 
  person does not indicate that there is no rough consensus, but one or 
  two well-reasoned arguments against a screaming huge crowd does. And 
  a huge number of "I'd prefer X, but I couldn't care less" votes 
  versus 2 or 3 well-argued "X will spell doom for the Internet, and Y 
  will save it" votes *is* rough consensus for Y over X. So voting 
  generally doesn't help me decide one way or the other that there is 
  rough consensus.
and
  As I said in the last message, voting (especially on topics where 
  there are only a few voices who are making well-reasoned arguments) 
  is often a lousy way to judge consensus because you can have a huge 
  number of people vote for a position they don't feel strongly about. 
  In the case of this particular issue, I think the chair taking a 
  rough stab at what appears to be consensus and waiting for screams of 
  horror is going to be much more productive than voting.

We've seen a recently declared "(VERY) rough consensus", and you have
referred to your May 27 message which repeats snippets of comments,
few of which can be said to constitute "well-reasoned arguments".  It's
difficult to guess what criteria in fact were used to conclude the
"(VERY) rough consensus", and there have been "screams of horror".

I don't think it's unreasonable to ask if the criteria we have been
using for the past couple of years have changed, and if so, what are
the new criteria.  Ideally we would have had that discussion up front,
but better late than never...
 
> I don't know of any simpler way than to get specific text change proposals 
> on the table, debate those on the open mailing list, and try to decide what 
> the consensus of the WG is.

That's only part of it.  It ought to be followed by
o ensure that the Document Editor(s) faithfully record the WG consensus
o ensure that once recorded in the document based on WG consensus, the
  text is not arbitrarily changed without good cause, WG discussion, and
  WG consensus [may I suggest an explicit note in the draft text, to be
  commented out during RFC publication, indicating the date on which
  consensus was declared as a cautionary note against unintentional
  revision during editing]

If it isn't, there's little point in proceeding; we've seen the movie
before (indeed, we've been watching reruns for 8 years) and we know how
it turns out.
 
> Painful, yes. Demands text proposals, yes.

There's no point in specific *text* proposals (as distinct from a specific
principle, with text to be worked out later) if they're rejected out of hand
by Editorial fiat or if they're rewritten to change the meaning (or, for that
matter, if the text is nitpicked to death ignoring the principles).  All have
happened, and that is why people are reluctant to suggest specific text.

Now if the co-chairs are giving assurances that that won't continue to
happen ("Come on in; the water's fine"), one or two people might cautiously
dip a toe in the pool, and in the fullness of time somebody might actually
dive in if and when it really seems worthwhile.  Whether or not it's a bit
late for that is another matter.

> Now, you seem to have a specific objection to:
> 
>    This specification uses the terms "header", "header name", and
>    "header content" which are synonymous with the [RFC2822] terms
>    "header field", "field name", and "field body" respectively.
> 
> Are you making a proposal for a specific text change to delete this 
> paragraph and change the remaining text to use the RFC 2822 synonyms? 
> That's an actionable item;

With s/RFC 2822 synonyms/standard FYI 18 & RFC 2822 terms/ that sounds
like a great idea.

So, count that as one actionable item.  Following some clarifications
regarding methods of determining consensus and assurances that WG
consensus will be respected, there may be others.

> (the word "header" occurs approx. 137 times in usefor-04.

It wouldn't if the mix of standard and non-standard terminology that was
in -00 had been corrected when that was suggested instead of being changed
to exclusively non-standard terminology by Editorial fiat...

> I think it's  
> makework to do the search-and-replace, but if I've interpreted you 
> correctly, you think differently. Please argue your case.)

See
http://www.imc.org/ietf-usefor/2004/Jul/0077.html
for a case that clear, consistent, and standard terminology matters.  The
goal of a technical specification is to enable implementers to produce
interoperable implementations, and the likelihood of that is diminished
if (any of)
a) the specification is ambiguous
b) the specification is self-inconsistent
c) implementers are forced to remember "in this document 'white' means
   black and 'black' means red, and 'red' means green, but I'm told 'it
   requires complete understanding and implementation of the normative
   references', in which 'white' means white, 'black' means black, and
   'red' means red"
d) there are conflicting definitions of "foo" (for one or more values
   of foo) between the document and its normative reference, with no
   clear indication of which definition has priority or why there is
   a difference
e) ABNF is self-inconsistent or ambiguous (or unchecked contrary to the
   requirement in ID-Guidelines) or conflicts with normative text
f) operational requirements impose the need for an efficient
   implementation, which is precluded by grammar or some normative
   requirement
g) the specification says "This document uses a cite by reference
   methodology, rather than repeating the contents of other standards,
   which could otherwise result in subtle differences and
   interoperability challenges", but the specification in fact repeats
   content of other specifications, introducing subtle (and not-so-subtle)
   differences and interoperability challenges.

Date: Fri, 10 Jun 2005 14:49:07 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: Newsgroup names #1021 (was: #1008 USEFOR 1.5 References: Final text on "followup")
Sender: [email-masked]

This was an issue that Charles raised off-list, I believe.
Good point - if filing tickets in response to issues raised off-list, we 
need to make sure the WG is notified.....

The text of the ticket is the following:
------------------------------
I suggest to update the Newsgroup ABNF as follows:
1). uppercase letters must be prohibited in newsgroup names
2). single component newsgroups must be prohibited (this also
addresses conflict with the special value "poster")
3). prohibiting all digit components can be done in ABNF, but it would
be ugly. I suggest to address this with an ABNF comment.
4). prohibiting components "ctl"/"all" and newsgroups starting
with "control", "example" or "to" can also be done in ABNF, but that
would be even uglier, as there is no way to exclude something in ABNF
(unless I am mistaken). So I suggest to add another comment(s).
-------------------------------------------
The following comment was added by someone logged in as "ietf":

[[email-masked] - Wed Jun 01 19:18:53 2005]:

 > 2). single component newsgroups must be prohibited (this
> also addresses conflict with the special value "poster")

 But not with pseudo-groups like "junk" or "control", and
the real NG "spamcop" <news://news.spamcop.net/spamcop>

 > 3). prohibiting all digit components can be done in ABNF,
> but it would be ugly.

 Ugly if necessary is acceptable, but why is it necessary ?

 > 4). prohibiting components "ctl"/"all" and newsgroups
> starting with "control", "example" or "to" can also be
> done in ABNF

 From a user's POV control.all behaves like r/o NGs. On
one server I know I "see" control.cancel.de (etc.) for
cancels in various TLHs. I've never heard of "example"
being reserved. And pseudo-TLH "to" is some mail-by-news
trick, from a user's POV like an NG.

 (But actually I'm only testing this RT beast, bye, Frank)

To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2: What zones should be on the MUST accept list?
From: Graham Drabble <[email-masked]>
Date: Fri, 10 Jun 2005 11:29:22 +0100
Organization: Home
Sender: [email-masked]

On 09 Jun 2005 Russ Allbery <[email-masked]> wrote in
news:[email-masked]: 

> Hopefully this will give people a new appreciation for just how
> much bizarre junk is out there on the wire.

I came accross a whole bunch of articles that had the form:

Tue, 26 Apr 2005 21:33:30 +Z

This was due to a badly configured moderation bot so it was affecting 
all the posts into the group.

FWIW this doesn't seem to have affected propogation and my newsreader 
(XNews) gets the date right and treats +Z as +0000. My stat gathering 
tool which uses perl's Date::Parse broke.

-- 
Graham Drabble
http://users.ox.ac.uk/~sjoh1646/

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: security considerations (was: Injection-Info issues)
Date: Fri, 10 Jun 2005 05:58:59 -0400
Cc: [email-masked]
Sender: [email-masked]

> >  naive methods of handling parameters, i.e. those which do not
> >  take errata, RFC 2231, and more errata into account are likely
> >  to botch some legal MIME messages.
> 
> But that's not the security issue.  If something botches a MIME
> message you normally get some more or less unreadable text.

As opposed to the MIME structure specified.

> The security issue is that an UA might get it right, and while
> it's at it it installs and starts the mail worm, or opens the
> GIF bomb, or whatever else these clever UAs might do.

That itself (w/o specific user input) would be a MIME violation.
 
> The security problem is the simple / naive scanner supposed to
> catch this malware _before_ it meets any software from Redmond.

Yes, because it botched MIME parsing.
 
> > To keep Charles on his toes, one might also remark about the
> > need to correctly handle all relevant base message format
> > (RFC 2822) constructs, and specifically mention that "find
> > the first semicolon" is not a viable approach.
> 
> Are you still talking about the boundary ?

"bOuNdArY" is just one attribute name. The parsing issue applies in
general, and broken "parsing" leads to problems.
 
> Or are you talking about 2822 CFWS between "=" and boundary,
> where this CFWS includes a semicolon ?

Not only comments, though that's one place where a semicolon
might appear.  Parsing has to use the specified syntax, and that's
not "find the first semicolon".

> <sigh>  Yeah, a warning 
> about comments before or after the boundary is an excellent
> idea.  Or maybe an example:
> 
> boundary (not=me) = ("yes ;-) simple (foo;bar") ; charset=utf-8

Or
 Content-Type: multipart/mixed (; boundary=foo ; xyz=");bOuNdArY*=''next%20part(")

Date: Fri, 10 Jun 2005 08:30:40 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: "Forrest J. Cavalier III" <[email-masked]>, [email-masked]
Subject: Re: #1031 Re: standard terminology and text (was #1022 Summary: Should dot be permitted in phrases?)
Sender: [email-masked]

--On torsdag, juni 09, 2005 14:31:49 -0400 "Forrest J. Cavalier III" 
<[email-masked]> wrote:

>
> Harald Tveit Alvestrand wrote:
>
>> I am looking forward to your specific suggestions on changes based on
>> these documents.
>>
>> This still does NOT qualify as suggesting a text change.
>>
>
> Huh?
>
> Bruce did a huge favor in reviewing and pointing out why the drafts
> don't resemble any other IETF work-product.  That review was a lot of work
> already.  Bruce provided a list of section numbers.

Forrest,

please tell me in which way (picking some random, and short, documents) 
draft-ietf-dnsop-serverid-04 and draft-ietf-sip-sctp-06 resemble each other 
more than either resemble draft-ietf-usefor-usefor-04.

After 8 years on the IESG reading drafts, I found nothing special about 
draft-ietf-usefor-usefor-04 that would warrant the "don't resemble any 
other IETF work-product" claim. So it seems you have different criteria 
than mine.

> Why is it acceptable to require that Bruce rewrite the sections before
> entering tickets for every one of those sections?  Bruce isn't editor.
>
> Many of the issues have been raised before.  Some of the changes are
> blatantly obvious, like the pointless redefinition of RFC2822 terms
> in USEFOR-04 section 2.
>
> If the chair and editor can't see the problems in the text now, then what
> criteria will be used to evaluate any rewritten versions?  Just curious.

I wonder about that too.

Furthermore, since Bruce has raised a lot of objections to a lot of drafts 
over the time I've observed him, and has seen some of those objections 
rejected, it's clear that the IETF can decide to approve something that 
Bruce does not like.

I don't know of any simpler way than to get specific text change proposals 
on the table, debate those on the open mailing list, and try to decide what 
the consensus of the WG is.

Painful, yes. Demands text proposals, yes.
Now, you seem to have a specific objection to:

   This specification uses the terms "header", "header name", and
   "header content" which are synonymous with the [RFC2822] terms
   "header field", "field name", and "field body" respectively.

Are you making a proposal for a specific text change to delete this 
paragraph and change the remaining text to use the RFC 2822 synonyms? 
That's an actionable item; "some of the changes are blatantly obvious" is 
not.

(the word "header" occurs approx. 137 times in usefor-04. I think it's 
makework to do the search-and-replace, but if I've interpreted you 
correctly, you think differently. Please argue your case.)

                      Harald

To: [email-masked]
Subject: Re: IPv6 literals in Path (Re: INN implementation clarifications)
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 09 Jun 2005 23:10:45 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> silly (?) question:

> is there a failure mode in any known deployed software in which Path:
> bang![dead::beef]!not-for-mail breaks things worse than Path:
> bang!dead::beef!not-for-mail?

> Russ' description seems to indicate that INN would cause both paths to
> be treated identically..... and to create trouble for hosts "dead" and
> "beef"....

Not that I'm aware of; I think they're equivalent.  Either software is
only going to use ! as a delimiter or software is going to use the
"standard" method of considering basically everything to be a delimiter.

Note that hosts dead and beef shouldn't happen in normal operations, since
Path entries should be FQDNs unless they're in the UUCP maps (and really,
these days, that latter excuse is getting less and less relevant).
However, of course, people do use random junk as their path identity from
time to time.

Date: Fri, 10 Jun 2005 08:02:34 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: IPv6 literals in Path (Re: INN implementation clarifications)
Sender: [email-masked]

--On fredag, juni 10, 2005 03:28:58 +0200 Frank Ellermann 
<[email-masked]> wrote:

> Russ Allbery wrote:
>
>> INN considers [a-zA-Z0-9._-] to be valid hostname characters
> [...]
>
>>     Path: f-oo.com%  bar_bar.com*10.10.10.10![::45af:89]=not-for-mail
>> is treated by INN as completely identical to:
>>     Path: f-oo.com!bar_bar.com!10.10.10.10!45af!89!not-for-mail
>
> I see a hack for our IPv6 literals, replace colon by underscore
> - but that's only a very bad dream, right ?

The colon is giving everyone bad dreams, some of which are RFCs :-(

RFC 3986 (URI syntax) put square brackets around IPv6 addresses to 
disambiguate them from hostnames; at the time, a number of solutions were 
suggested, including representing the IPv6 address as 32 hex digits without 
punctuation and percent-escaping the colons.

In the end, the people doing that syntax decided to use the colon and let 
the parsers fail where they may.

silly (?) question:

is there a failure mode in any known deployed software in which Path: 
bang![dead::beef]!not-for-mail breaks things worse than Path: 
bang!dead::beef!not-for-mail?

Russ' description seems to indicate that INN would cause both paths to be 
treated identically..... and to create trouble for hosts "dead" and 
"beef"....

                         Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: security considerations
Date:  Fri, 10 Jun 2005 06:46:54 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 5
Sender: [email-masked]

Frank Ellermann wrote:

No, I didn't write nothing.  The raw article is here:
http://article.gmane.org/gmane.ietf.usenet.format:28712:raw

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  security considerations (was: Injection-Info issues)
Date:  Fri, 10 Jun 2005 05:33:54 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

This is a multi-part message in MIME format.

--simple

Bruce Lilly wrote:

> specifically mention that "find the first semicolon" is not a
> viable approach.

Who fixes my UA ?  It's too old for these boundary tricks... :-(

--simple--

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  security considerations (was: Injection-Info issues)
Date:  Fri, 10 Jun 2005 05:24:33 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 39
Sender: [email-masked]

Bruce Lilly wrote:

>>>| MIME security considerations are discussed in [RFC2046].
>>>| Note that applying some [RFC2231] extensions for parameters
>>>| like multi-line paramters on a boundary parameter as defined
>>>| in [RFC2046] might be abused to bypass simple algorithms
>>>| trying to analyze MIME parts.
[...]
>  naive methods of handling parameters, i.e. those which do not
>  take errata, RFC 2231, and more errata into account are likely
>  to botch some legal MIME messages.

But that's not the security issue.  If something botches a MIME
message you normally get some more or less unreadable text.

The security issue is that an UA might get it right, and while
it's at it it installs and starts the mail worm, or opens the
GIF bomb, or whatever else these clever UAs might do.

The security problem is the simple / naive scanner supposed to
catch this malware _before_ it meets any software from Redmond.

> To keep Charles on his toes, one might also remark about the
> need to correctly handle all relevant base message format
> (RFC 2822) constructs, and specifically mention that "find
> the first semicolon" is not a viable approach.

Are you still talking about the boundary ?  In that case it's
more a 2045 / 2046 / 2231 issue, and semicolon is no bchar.

Or are you talking about 2822 CFWS between "=" and boundary,
where this CFWS includes a semicolon ? <sigh>  Yeah, a warning
about comments before or after the boundary is an excellent
idea.  Or maybe an example:

boundary (not=me) = ("yes ;-) simple (foo;bar") ; charset=utf-8

                              Bye, Frank

To: [email-masked]
Subject: Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 09 Jun 2005 19:51:02 -0700
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> How about taking TZ as is ?  On my box I'd get this creature:

> SET TZ=CET-1DST,3,-1,0,7200,10,-1,0,10800,3600

> The CET would survive your table, but the "DST" is completely
> bogus, I use it because IIRC MS C 6.0 wanted 3 letters.  Can't
> check it at the moment, the C library sources are on an almost
> dead disk (disabled until I find the place for the last "read"
> attempt).

> Don't worry, AFAIK nothing on my box uses "DST" for anything,
> but in theory a very strange UA could try it.  And in theory I
> could live in Eastern Australia and then, who knows.

> The obs- timezone crap is as you said above all crap ;-)  Bye.

TZ has not, for many years now and perhaps ever, been bidirectionally
equivalent to the time zone abbreviation.  Some time zone abbreviations
are *also* valid TZ settings, but for instance the most common TZ setting
in my part of the US (prior to switching to the Olson scheme) would be
PST8PDT, which has never been valid as a zone abbreviation.

I doubt any software would use it that way.

What software *does* do is use the time zone abbreviation generated by the
system time zone libraries; most time zones do have assigned (often
ambiguous) abbreviations.  Heck, INN does this.  It just does it in a
*comment*, which is the right way to do it.  The comment is valuable
information (well, insofar as any human-readable time zone information is
valuable at least), since it lets you distinguish between, say, PDT and
MST without knowing the daylight saving date.

That abbreviation is unrelated to the TZ string, however.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
Date:  Fri, 10 Jun 2005 04:31:50 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 23
Sender: [email-masked]

Russ Allbery wrote:

>> Guess what individual Australian users are likely to do.
 
> This is not the sort of thing that's likely to be a
> user-configurable option in a news reader.

How about taking TZ as is ?  On my box I'd get this creature:

SET TZ=CET-1DST,3,-1,0,7200,10,-1,0,10800,3600

The CET would survive your table, but the "DST" is completely
bogus, I use it because IIRC MS C 6.0 wanted 3 letters.  Can't
check it at the moment, the C library sources are on an almost
dead disk (disabled until I find the place for the last "read"
attempt).

Don't worry, AFAIK nothing on my box uses "DST" for anything,
but in theory a very strange UA could try it.  And in theory I
could live in Eastern Australia and then, who knows.

The obs- timezone crap is as you said above all crap ;-)  Bye.

To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2: What zones should be on the MUST accept list?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 09 Jun 2005 19:21:04 -0700
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:
> Russ Allbery wrote:

>> Hopefully this will give people a new appreciation for just
>> how much bizarre junk is out there on the wire.

> Thanks, nice statistics.  So nobody uses Z etc. ?

I've never seen a use of the military time zones in the wild, although
it's possible my script might have missed some uses if there was no space
before the time zone name (which is allowed by RFC 822; RFC 822 doesn't
require any spaces at all in the string, last time I checked).  But I
doubt it.

>>          MEZ:      56 (00.0%)

> Ugh.  But at least no MESZ or "MEZT".

At least in this batch of stuff!  :)

Incidentaly, I think I checked at one point and the LOCAL and UNKNOWN time
zones were from articles from the 1990s that were in groups that never
expired.  I'm not sure, though.

> For USEFOR your results mean "no GMT hits 21%", that would be
> bad.  Henry already said that he won't insist on UT.  So the
> real problem is EDT + EST (0.9%).

I think it would be insane to allow anything other than what's in RFC 822.
I'm certainly not arguing for adding BST, JST, etc.  I'm mostly pointing
out two things:  EDT/EST/PDT/PST/etc. are both fully compliant with RFC
1036 and used in practice by significant numbers of articles
(*particularly* if you want to do things with old news spools, which
people do often want to do), and real-life news software accepts a ton of
junk that's never been in any standard because of common use of getdate as
mentioned in RFC 1036.  getdate supports an unfortunate number of textual
time zone strings.

The Date specification in RFC 1036 is unfortunately something of a mess.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  LTRU questions (was: #1022 Summary: Should dot be permitted in phrases?)
Date:  Fri, 10 Jun 2005 04:03:02 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 40
Sender: [email-masked]

Bruce Lilly wrote:

> ISO 3166 is used normatively in section 3.2.6 but is listed
> as informative.

ISO 3166:1988 3rd ed. :-(  Maybe we can use the region codes
in the new LTRU registry, it's probably ready for take-off
before USEFOR.  I'll try to watch this point.  In any case of
course "normative", but no debates about CS 1988 please.

While I'm at it here's another LTRU issue:  The "worst" encoded
word for 3066bis could be something like:

Subject:
 =?pc-multilingual-850+euro*lng-exa-exb-exc-Latn-831-variant1-variant2?Q?a?=
....5...10....5...20....5...30....5...40....5...50....5...60....5...70....5.

That's ridiculous, but "legal" under 2047 / 2231 / 3066bis.

Now let's assume that it's the first character of a word where
the language tag makes sense.  And it's the first character of
the Subject.  And it hits a mail2news gateway.  Any attempt to
split one letter fails.

There are some rules in 3066bis allowing to strip "-variant2":

Subject: =?pc-multilingual-850+euro*lng-exa-exb-exc-Latn-831-variant1?Q?a?=
....5...10....5...20....5...30....5...40....5...50....5...60....5...70....5.

Good enough, but do we expect this clever trick from gateways ?

Or should we just offer a way to force compatibility with our
rules by adding a "dummy" word in the 1st line of the Subject ?

Subject: =?utf-8?Q?=EF=BF=BE?=
 =?pc-multilingual-850+euro*lng-exa-exb-exc-Latn-831-variant1-variant2?Q?a?=
....5...10....5...20....5...30....5...40....5...50....5...60....5...70....5.

Or what should the poor mail2news gateway do ?  Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: INN implementation clarifications
Date:  Fri, 10 Jun 2005 03:28:58 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

Russ Allbery wrote:

> INN considers [a-zA-Z0-9._-] to be valid hostname characters
[...]

>     Path: f-oo.com%  bar_bar.com*10.10.10.10![::45af:89]=not-for-mail
> is treated by INN as completely identical to:
>     Path: f-oo.com!bar_bar.com!10.10.10.10!45af!89!not-for-mail

I see a hack for our IPv6 literals, replace colon by underscore
- but that's only a very bad dream, right ?

                         Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1028 USEFOR 3.1.2: What zones should be on the MUST accept list?
Date:  Fri, 10 Jun 2005 03:09:50 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Russ Allbery wrote:

> Hopefully this will give people a new appreciation for just
> how much bizarre junk is out there on the wire.

Thanks, nice statistics.  So nobody uses Z etc. ?

>          MEZ:      56 (00.0%)

Ugh.  But at least no MESZ or "MEZT".

> I write better code when I expect to ever use it again.  :)

I won't understand it unless it's gawk ;-)  If you're sure
that you didn't miss the military zones I simply believe it.

For USEFOR your results mean "no GMT hits 21%", that would be
bad.  Henry already said that he won't insist on UT.  So the
real problem is EDT + EST (0.9%).
                                 Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Newsgroup names #1021 (was: #1008 USEFOR 1.5 References: Final text on "followup")
Date:  Fri, 10 Jun 2005 02:50:41 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 15
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> the subject of the definition of "followup" is now closed.

Amen.

> Any new messages on the subject of the definition of
> "followup" that do not raise new technical issues will be
> treated as off-topic postings.

OB on topic:  what's the purpose of ticket #1021 ?  Where does
it come from ?  https://rt.psg.com/Ticket/Display.html?id=1021

                Bye, Frank

To: [email-masked]
Subject: Re: INN implementation clarifications
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 09 Jun 2005 14:59:52 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:
> On Thu June 9 2005 16:01, Russ Allbery wrote:

>>> My understanding was that INN was now accepting folded Newsgroups
>>> headers when relaying.

>> Ah, yes, it's done that since 2.4.

> Q1: for relaying only, or for relaying and injection?

Only for relaying.

> Q2: any estimate of the number of pre-2.4 installations still active?

Certainly at least hundreds.  My guess would be that something in the
range of 50-60% of the INN installations out there have upgraded at this
point, based on the questions that I get.

>> It doesn't accept colons, btw.

> Q3: In the Newsgroups field?

Right.  Sorry, that was rather unimportant; I ended up getting confused
between this and the Path header.  Of course it doesn't accept colons in
Newsgroups.  :)

> Q4: What about colon in Path; does "doesn't accept" mean:
>    a) rejects articles
>    b) interprets colon as a delimiter
>    c) does not interpret colon as a delimiter

INN considers [a-zA-Z0-9._-] to be valid hostname characters in the Path
header and all other characters are treated as delimiters.  So in other
words, the following path header:

    Path: f-oo.com%  bar_bar.com*10.10.10.10![::45af:89]=not-for-mail

is treated by INN as completely identical to:

    Path: f-oo.com!bar_bar.com!10.10.10.10!45af!89!not-for-mail

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: CFWS in References header
Date: Thu, 9 Jun 2005 17:25:16 -0400
Cc: [email-masked]
Sender: [email-masked]

On Thu June 9 2005 13:04, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >On Wed June 8 2005 09:52, Charles Lindsey wrote:
> 
> >> So the position we took was that people should start generating them when,
> >> in their opinion, they would work well enough [...].

> AIUI, Mark Crispin's idea is that you give guidance to implementors to the
> effect that such and such behaviour, although permitted, will cause such
> and such problems on the present network, and that implementors should
> consider this carefully before deciding whether to provide it.

There were several specific examples, none of which used anything
resembling "in their opinion".
   
> If that is correct, I would have no particular problem with it. But it is,
> in essence, what the article-* drafts were trying to say (they were
> certainly not saying "do whatever you please", any more than Mark was
> saying that).

There is a huge difference between documenting specific issues and
making specific recommendations vs. leaving matters to the opinions
of non-experts.

Another example of how issues can be handled is the following text
taken from the rfc2440bis draft:

        While this attack is somewhat obscure, requiring a special set
        of circumstances to create it, it is nonetheless quite serious
        as it permits someone to trick a user to decrypt a message.
        Consequently, it is important that:

         1. Implementers treat MDC errors and decompression failures as
            security problems.

         2. Implementers implement Modification Detection with all due
            speed and encourage its spread.

         3. Users migrate to implementations that support Modification
            Detection with all due speed.

Something similar could certainly be used where there are issues
corresponding to what Mark calls "baby programmer errors", e.g.
inability to handle msg-ids with a legal 255-octet domain name.  There
is no good reason to be stuck with such an absurdity forever; implementers
can and should be encouraged to fix the problems "with all due speed".
 
> The problems are indeed with UAs, and it is true that such comments are
> most unlikely to appear in Real Life (so getting agents to accept them
> comes fairly low in implementors' priority orders).
> 
> However, it would be unwise to assume that existing mail UAs "already
> necessarily handle comments".

No assumption necessary; comments have always been legal in RFC 724/733/822
structured fields -- any supposed "UA" that doesn't handle them properly is
not conformant.  Some software is crappy; fact of life.  Absent other
considerations, that's no reason to degrade specifications to the level of
crappiness of the lowest common denominator.

> For sure, it is known for a fact that some agents (news-only ones maybe)
> do NOT currently handle these comments correctly.

Nobody is going to force them to claim compliance with our specification.
As RFCs 850 and 1036 refer to 822 and make no exception regarding comments
in References fields, such agents were not 850- or 1036-conforming.

Date: Thu, 09 Jun 2005 23:25:10 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
Sender: [email-masked]

--On torsdag, juni 09, 2005 16:14:05 +0000 Charles Lindsey 
<[email-masked]> wrote:

>> To clarify, my strong preference is "exactly as RFC 2822".
>> I'm willing to entertain other options, so long as there is an
>> accompanying well-reasoned argument for a deviation from 2822.
>
> I think everyone agrees MUST accept.
>
> The issue is SHOULD NOT vs MUST NOT for generate.

Charles, may I remind you again:

the current wording (2.1) is MAY accept.

What wording change do you propose?

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Russ Allbery <[email-masked]>
Subject: INN implementation clarifications
Date: Thu, 9 Jun 2005 16:50:31 -0400
Cc: [email-masked]
Sender: [email-masked]

On Thu June 9 2005 16:01, Russ Allbery wrote:
> 
> Charles Lindsey <[email-masked]> writes:
> > Russ Allbery <[email-masked]> writes:
> 
> >> I think I've mentioned this before, but maybe not.  INN still does not
> >> accept folded newsgroup headers, since that would mean accepting an
> >> article that's just going to get rejected by the next news server and
> >> fail silently.  Be conservative in what one generates, etc.
> 
> > My understanding was that INN was now accepting folded Newsgroups
> > headers when relaying.
> 
> Ah, yes, it's done that since 2.4.

Q1: for relaying only, or for relaying and injection?

Q2: any estimate of the number of pre-2.4 installations still active?

> It doesn't accept colons, btw.

Q3: In the Newsgroups field?

Q4: What about colon in Path; does "doesn't accept" mean:
   a) rejects articles
   b) interprets colon as a delimiter
   c) does not interpret colon as a delimiter

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: Date-time yet again
Date: Thu, 9 Jun 2005 16:42:48 -0400
Cc: [email-masked]
Sender: [email-masked]

On Thu June 9 2005 14:37, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >On Wed June 8 2005 10:42, Charles Lindsey wrote:
> 
> >> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> >> 
> >> >Precisely how would "PDT" "break the protocols" in a way that "UT"
> >> >would not?
> >> 
> >> I am not aware what other zones use "PDT", but [...]
> 
> >Charles, stop trying to evade the issue.
> 
> Whether or not "PDT" has clones is beside the point

Then its rather telling that you keep changing the subject to that when
you are asked to substantiate your claims, viz.:

  You MUST NOT generate "EST" or any of the other things in <obs-zone>
  because it would actually break the protocols
and that it would
  REQUIRE all News software to be upgraded to accept constructes that
  had Never Ever been seen in Netnews

The simple fact is that the specific alphabetic zones listed in RFC
2822 parse grammar also appeared in RFC 822 (cited as the definitive
reference for date-time by RFCs 850 and 1036) and are in fact handled
by news software past (including B news) and present.  There is nothing
that "would actually break the protocols" as you claimed; obviously,
since those abbreviations have been in use for decades without
breakage (illegal cruft not in 822 is another matter).

For the purpose of specifying the date-time field component semantics
and syntax, I see absolutely no reason to deviate from RFC 2822 and
several very good reasons to defer the matter to 2822.

For protocol purposes, we can address handling of obs-zone by injection
and transport agents in the protocol document, which is a separate matter.

To: [email-masked]
Subject: Re: #1031 Re: standard terminology and text
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 09 Jun 2005 13:08:46 -0700
Sender: [email-masked]

Forrest J Cavalier <[email-masked]> writes:

> Huh?

> Bruce did a huge favor in reviewing and pointing out why the drafts
> don't resemble any other IETF work-product.  That review was a lot of
> work already.  Bruce provided a list of section numbers.

> Why is it acceptable to require that Bruce rewrite the sections before
> entering tickets for every one of those sections?  Bruce isn't editor.

In those messages, he wasn't sufficiently specific about what he wanted
changed for me to have any idea what he was talking about without applying
knowledge from previous discussions (sometimes years back).  He doesn't
have to rewrite the sections, but he should at least list specifically
what terms he wants changed and outline what textual changes he wants
made.

> Many of the issues have been raised before.

Yes, I agree with this, but to get the information into the ticket it
needs to be summarized without assuming people have read all the previous
messages.

> Some of the changes are blatantly obvious, like the pointless
> redefinition of RFC2822 terms in USEFOR-04 section 2.

When they're blatantly obvious, they're very easy to list.

> If the chair and editor can't see the problems in the text now, then
> what criteria will be used to evaluate any rewritten versions?

I think you're putting words into the chair's mouth.  We're not talking
about his personal ability to see the desired changes; we're talking about
a procedure that we were asked to follow in making changes.  Proposals
need to be specific enough that everyone knows exactly what is being
proposed to change.

To: [email-masked]
Subject: Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 09 Jun 2005 13:05:34 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Yes, I agree that particular scenario is unlikely, though it is quite
> easy to hack the TZ tables in Unix to make EST mean whatever you like,
> and I have no doubt that many Australian sysadmins will have done so.

I find this incredibly unlikely.  Why on earth would they do that rather
than just configuring the system to be in Australia/Sydney?

> 1. It is also easy to hack the news software (indeed, the CNews sources
> already contain the necesary line of code - and for Easter Island and
> East Brazil too - just proteced by an "#if 0 .... #endif"). I worry that
> some Australian newsadmin will have succumbed to the temptation.

Also extremely unlikely; the comment says explicitly why not to do that.

> 2. Individual users may have configured their user agents with such
> timezones (indeed, I suspect that is how the non-GMT examples arose in
> your test). Guess what individual Australian users are likely to do.

This is not the sort of thing that's likely to be a user-configurable
option in a news reader.  I've never seen a news reader that could be
configured that way.  The bogus time zones are probably due to bad choices
in writing the code, likely code that was written in the US and therefore
always used RFC 822-approved time zones in that context being run
elsewhere in the world.

I think your objections are on grounds that are very unlikely, and that
none of them warrant a diversion from RFC 2822.  EST has a well-defined
meaning as a time zone, for better or for worse, and news software has
been giving it that meaning for at least fifteen years.  I really do not
see how this could suddenly become some sort of huge interoperability
problem.

To: [email-masked]
Subject: Re: CFWS in References header
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 09 Jun 2005 13:01:54 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> I think I've mentioned this before, but maybe not.  INN still does not
>> accept folded newsgroup headers, since that would mean accepting an
>> article that's just going to get rejected by the next news server and
>> fail silently.  Be conservative in what one generates, etc.

> My understanding was that INN was now accepting folded Newsgroups
> headers when relaying.

Ah, yes, it's done that since 2.4.

It doesn't accept colons, btw.

To: [email-masked]
Xref: clerew local.usefor:21158
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Date-time yet again
Date: Thu, 9 Jun 2005 18:37:54 GMT
Lines: 98
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Wed June 8 2005 10:42, Charles Lindsey wrote:

>> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
>> 
>> >Precisely how would "PDT" "break the protocols" in a way that "UT"
>> >would not?
>> 
>> I am not aware what other zones use "PDT", but [...]

>Charles, stop trying to evade the issue.

Whether or not "PDT" has clones is beside the point, unless you intend to
single out and permit the "safe" zones and deprecate the others. Which
would be most unwise IMO.

Here, for your general amusement, are the clones I have picked out of the
Cnews sources. I strongly suspect this list is but the tip of the real
iceberg :-( .

/*   text           token   lexval */
    "adt",          DTZ,    PACK(-180),     /* Atlantic DT */
    "adt",          DTZ,    PACK(-240),     /* Acre DT */
    "adt",          DTZ,    PACK(0),        /* Azores DT */
    "ast",          TZ,     PACK(-240),     /* Atlantic ST (Canada) */
    "ast",          TZ,     PACK(-300),     /* Acre ST */
    "ast",          TZ,     PACK(-60),      /* Azores ST */
    "bst",          DTZ,    PACK(60),       /* British Summer Time */
    "bst",          TZ,     PACK(-180),     /* Brazil ST */
    "cdt",          DTZ,    PACK(-180),     /* Chile DT */
    "cdt",          DTZ,    PACK(-240),     /* Cuba DT */
    "cdt",          DTZ,    PACK(-300),     /* Central DT */
    "cdt",          DTZ,    PACK(540),      /* China DT */
    "cst",          TZ,     PACK(-240),     /* Chile ST */
    "cst",          TZ,     PACK(-300),     /* Cuba ST */
    "cst",          TZ,     PACK(-360),     /* Central ST */
    "cst",          TZ,     PACK(480),      /* China ST */
    "edt",          DTZ,    PACK(-120),     /* East Brazil DT */
    "edt",          DTZ,    PACK(-240),     /* Eastern DT */
    "edt",          DTZ,    PACK(-300),     /* Easter Island DT */
    "edt",          DTZ,    PACK(660),      /* Australian Eastern DT */
    "est",          TZ,     PACK(-180),     /* East Brazil ST */
    "est",          TZ,     PACK(-300),     /* Eastern ST */
    "est",          TZ,     PACK(-360),     /* Easter Island ST */
    "est",          TZ,     PACK(600),      /* Australian Eastern ST */
    "fst",          DTZ,    PACK(120),      /* French Summer Time */
    "fst",          TZ,     PACK(-120),     /* Fernando de Noronha ST */
    "ist",          TZ,     PACK(120),      /* Israel */
    "ist",          TZ,     PACK(330),      /* Indian ST */
    "sst",          DTZ,    PACK(120),      /* Swedish Summer Time */
    "sst",          TZ,     PACK(-660),     /* Samoa ST */
    "sst",          TZ,     PACK(480),      /* Singapore ST */
    "wet",          TZ,     PACK(0),        /* Western Europe */
    "wet",          TZ,     PACK(60),       /* Western European Time */
    "wst",          TZ,     PACK(-240),     /* Western Brazil ST */
    "wst",          TZ,     PACK(480),      /* West Australian ST */

>> We decided that 
>> obs-syntax in general was a Bad Thing, and it would be an imposition to
>> demand that news agents should accept it all regardless (a point you
>> yourself once made, as you will doubtless recall).

>I do not recall any such thing

No, you wouldn't. It must have been decided in 1997 or 1998, and has not
been challenged since, apart from the two exceptions for obs-zone and
obs-phrase which were also discussed agreed (in 2001 and 2003 respectively).

That is how the decisions made by the WG stand at the moment. By all means
propose further changes in this area, but do not pretend that those former
decisions were never made.

>> >> IIRC, it was you who asked for <obs-zone> to be brought within our
>> >> standard.
>> 
>> >References, please.
>> 
>> www.imc.org/ietf-usefor/2003/Feb/0221.html

>That message has nothing whatsoever to do with obs-zone (the term is
>nowhere mentioned in that message, nor is the subject matter).

Yes, I was confusing the <obs-zone> discussion with the <obs-phrase>
discussion. My apologies for that.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21156
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: CFWS in References header
Date: Thu, 9 Jun 2005 17:04:27 GMT
Lines: 68
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Wed June 8 2005 09:52, Charles Lindsey wrote:

>> So the position we took was that people should start generating them when,
>> in their opinion, they would work well enough [...].

>I suspect that in a Standards Track document "do whatever you please"
>isn't going to pass muster.  Less so, I suspect, than a flag day.
>  
>> So there is the gamut of possibilities. They need to be considered
>> together.

>No, there are other possibilities, including, but not limited to:
>o provide a roadmap (X at Proposed, Y at Draft, Z at Full Standard).
>  Harald can probably give some guidance as to how well that would be
>  received by the IESG, but failing something like that, there is a
>  good chance that each such change would require a reset to Proposed
>o Implementation notes as suggested by Mark Crispin:
>    http://www.imc.org/ietf-usefor/2003/Feb/0486.html
>    http://www.imc.org/ietf-usefor/2003/Feb/0493.html
>    http://www.imc.org/ietf-usefor/2003/Feb/0554.html

AIUI, Mark Crispin's idea is that you give guidance to implementors to the
effect that such and such behaviour, although permitted, will cause such
and such problems on the present network, and that implementors should
consider this carefully before deciding whether to provide it.

If that is correct, I would have no particular problem with it. But it is,
in essence, what the article-* drafts were trying to say (they were
certainly not saying "do whatever you please", any more than Mark was
saying that).

>Specifically regarding the References field and comments, what is the
>scope of the issue?  Transport agents are unaffected.  Any
>considerations appear to be confined to UAs, and as the field plays
>imperatives), "[RFC2822]" (for implementations that wish to claim
>conformance) may suffice [note that a) comments have always been
>permitted per RFC 822 as adopted by 850/1036, b) many UAs are combined
>mail/news UAs and already necessarily handle comments (unless they're
>broken UAs), and c) message-generating UAs generally do not insert
>comments in References fields, so the only likely source is a human
>manually editing a field].  It doesn't seem worth spending much time
>on -- not beyond an implementation note at most.

The problems are indeed with UAs, and it is true that such comments are
most unlikely to appear in Real Life (so getting agents to accept them
comes fairly low in implementors' priority orders).

However, it would be unwise to assume that existing mail UAs "already
necessarily handle comments". Have you actually tried it? Sadly, broken
agents are a fact of life, however much King Canute might wish them to go
away.

For sure, it is known for a fact that some agents (news-only ones maybe)
do NOT currently handle these comments correctly.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21157
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
Date: Thu, 9 Jun 2005 18:04:10 GMT
Lines: 39
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> It is just not realistic to suppose that servers in Australia will
>> interpret "EST" according to RFC 2822 (they will probably just hand the
>> zone off to the OS, and for sure that will take an Australian view of
>> time zones).

>I have never seen any news software that does this, and furthermore, given
>that there is no OS facility short of changing TZ for the process that
>could make head or tails of "EST" on Unix at least, I disbelieve in the
>existence of software that takes this approach.

Yes, I agree that particular scenario is unlikely, though it is quite easy
to hack the TZ tables in Unix to make EST mean whatever you like, and I
have no doubt that many Australian sysadmins will have done so.

But there are two more realistic scenarios.

1. It is also easy to hack the news software (indeed, the CNews sources
already contain the necesary line of code - and for Easter Island and East
Brazil too - just proteced by an "#if 0 .... #endif"). I worry that some
Australian newsadmin will have succumbed to the temptation.

2. Individual users may have configured their user agents with such
timezones (indeed, I suspect that is how the non-GMT examples arose in
your test). Guess what individual Australian users are likely to do.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21154
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1028 USEFOR 3.1.2: What zones should be on the MUST accept list?
Date: Thu, 9 Jun 2005 16:32:53 GMT
Lines: 45
Sender: [email-masked]

In <[email-masked]> Henry Spencer <[email-masked]> writes:

>On Wed, 8 Jun 2005, Harald Tveit Alvestrand wrote:
>> These numbers could argue for skipping UT and/or including EDT (and 
>> presumabily EST, since this measurement was taken in summer).

>"UT" was my addition in son-of-1036, and since it seems to have achieved
>essentially zero market penetration :-), I think we can forget about it. 

>We should *NOT* bless EST or EDT, still in significant use though they may
>be, because they are definitely ambiguous.  (I think it most unlikely that
>software which puts in a local timezone name is first checking to make
>sure that the definition agrees with the RFC 2822 obs-zone definition.)

I concur.

GMT is a special case because (a) it is not ambiguous and (b) it occurs in
20% of current articles (presumably because some widely used software is
enforcing it - I know that CNews is one of the culprits in that regard).

So I think we MUST accept GMT, though I would happily drop UT.

As for the rest, I do not think we should cherry pick.

Either we accept everything in the RFC 2822 <obs-zone> (with some risk of
misuse in Australia and other places), or we accept none of it beyond
GMT. My preference is the latter, followed by the former (OK, you might
omit the military ones).

As for generation, we currently say SHOULD NOT. I would have no probnlem
if someone wants to promoite that to MUST NOT.

BTW, I discovered this morning that "EST" is also used in Eastern Brazil
and Easter Island :-) .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21153
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
Date: Thu, 9 Jun 2005 16:14:05 GMT
Lines: 37
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Wed June 8 2005 07:13, Harald Tveit Alvestrand wrote:

>> - Bruce Lilly argues against a deviation (I think).

>To clarify, my strong preference is "exactly as RFC 2822".
>I'm willing to entertain other options, so long as there is an
>accompanying well-reasoned argument for a deviation from 2822.

I think everyone agrees MUST accept.

The issue is SHOULD NOT vs MUST NOT for generate.

The argument for MUST NOT is that it follows RFC 2822 exactly.

The argument for SHOULD NOT is that we may be choosing SHOULD NOT for the
References and Newsgroups folding cases, in which case there is some merit
in being consistent across all three.

And I am assuming that we are discussing only <obs-phrase> (aka
<extended-phrase>) here.

I would be totally opposed to opening the floodgates to the obs- syntax
as a whole. Just the two cases we are discussing - <obs-phrase> and
<obs-zone>.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21155
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: CFWS in References header
Date: Thu, 9 Jun 2005 16:39:08 GMT
Lines: 23
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>I think I've mentioned this before, but maybe not.  INN still does not
>accept folded newsgroup headers, since that would mean accepting an
>article that's just going to get rejected by the next news server and fail
>silently.  Be conservative in what one generates, etc.

My understanding was that INN was now accepting folded Newsgroups headers
when relaying.

I agree that it would be foolish, at this time, to accept them when
injecting.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Thu, 9 Jun 2005 12:37:41 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

Henry Spencer <henry@xxxxxxxxxxxxx>:

>However, the mere fact that someone says the consensus is wrong doesn't
>make it so.

If you are going to pretend that the only argument I've ever made about 
this is "I say it is wrong", then I will happily go through all the 
details again. Is that what you want, or would you like to change your
statement and admit that I've provided much more than "I say"?

>No, only that RFC 2822 -- a mail standard -- doesn't entirely apply to
>news, 

When our standard says it defers to RFC2822, it defers to RFC2822. It
doesn't matter if RFC2822 "doesn't entirely apply", it applies in those
sections where we have already said it applies. References is one such
section. 

If you wish to submit proposed text so that we do NOT defer to RFC2822 for
References, fine. If you wish to discuss the text I've already provided
for USEFOR to bring it in line with RFC2822, that's fine too. But claiming
that RFC2822 doesn't apply to news after we've said that it does is
dishonest.

>Defining the format and semantics of news articles is our job, and it is
>appropriate for us to do it.

Within the context of the IETF, which includes RFC2119. If we don't want
RFC2119 to apply, then why do we explicitely say it does? If you would
like to propose text to correct this compliance with RFC2119, please do.

>Correct, it's not.  The ability to detect intended relationships between
>articles is important -- that's what References is for 

That's what YOU say it is for. Unfortunately, none of the RFCs agree with 
you. So, I guess it's ok for "Henry says" to be a closing argument. Double 
standard (pun intended)?

Date: Thu, 09 Jun 2005 14:31:49 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1031 Re: standard terminology and text (was #1022 Summary: Should dot be permitted in phrases?)
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> I am looking forward to your specific suggestions on changes based on 
> these documents.
> 
> This still does NOT qualify as suggesting a text change.
> 

Huh?

Bruce did a huge favor in reviewing and pointing out why the drafts
don't resemble any other IETF work-product.  That review was a lot of work
already.  Bruce provided a list of section numbers.

Why is it acceptable to require that Bruce rewrite the sections before
entering tickets for every one of those sections?  Bruce isn't editor.

Many of the issues have been raised before.  Some of the changes are
blatantly obvious, like the pointless redefinition of RFC2822 terms
in USEFOR-04 section 2.

If the chair and editor can't see the problems in the text now, then what
criteria will be used to evaluate any rewritten versions?  Just curious.

Date: Thu, 9 Jun 2005 14:07:19 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

On Thu, 9 Jun 2005, John Stanley wrote:
> >...the text that seems to come closest to capturing the (VERY) rough 
> >consensus of the group is:
> 
> This is a technical standard. A consensus to be wrong is still wrong.

However, the mere fact that someone says the consensus is wrong doesn't
make it so.

> Just to make it clear, the official position of this working group is that 
> RFC2822 doesn't say what it does...

No, only that RFC 2822 -- a mail standard -- doesn't entirely apply to
news, or won't until 2822 is revised to be less mail-centric.  (It's
better than it was, but still needs work.)

> that mandates based on "we want" are appropriate despite RFC2119...

Defining the format and semantics of news articles is our job, and it is
appropriate for us to do it.

> and that the ability to detect followups is not important.

Correct, it's not.  The ability to detect intended relationships between
articles is important -- that's what References is for -- but deciding
whether those relationships came about because an article is a followup
to another is not.

                                                          Henry Spencer
                                                       [email-masked]

Date: Thu, 9 Jun 2005 10:20:49 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>...the text that seems to come closest to capturing the (VERY) rough 
>consensus of the group is:

This is a technical standard. A consensus to be wrong is still wrong.

> Unless there are technical issues that have not been raised before,

The existing one is sufficient. When did we get to vote on changing the
meaning of RFC2822?

> ...the definition of "followup" is now closed.

Just to make it clear, the official position of this working group is that 
RFC2822 doesn't say what it does, that mandates based on "we want" are 
appropriate despite RFC2119, and that the ability to detect followups is
not important.

I'll also point out that your "definition" is really just the first 
sentence. The second sentence is not definition, especially when you note 
that the attribute you are using to define followup is also present in 
non-followups. You might as well leave the second sentence out; it means 
nothing and is the cause of the problem.

Date: Thu, 09 Jun 2005 19:20:30 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1032 documenting new fields (was #1022 Summary: Should dot be permitted in phrases?)
Sender: [email-masked]

I have assigned ticket #1032 to this issue.

However, I think (personal opinion) that a better approach is to add an 
appendix to the document stating how the format differs from the one 
described by RFC 1036.

                             Harald

--On torsdag, juni 09, 2005 11:55:07 -0400 Bruce Lilly <[email-masked]> 
wrote:

> On Thu June 9 2005 09:40, Harald Tveit Alvestrand wrote:
>
>> Then make multiple suggestions, or say "I suggest we make this change,
>> and  the following sections need to be changed because of that - details
>> to be  discussed when the basic change is accepted".
>
> I suggest that the draft text be revised to conform to the Chair's defined
> scope as presented in
>   http://www.imc.org/ietf-usefor/2004/Jun/0370.html
> specifically, that new fields must clearly be marked as such.
>
> The following sections need to be changed because of that - details to be
> discussed when the basic change (to the draft, no change in policy is
> suggested) is reiterated:
>   3.2.11, 3.2.12, 3.2.13
>

Date: Thu, 09 Jun 2005 19:15:50 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1031 Re: standard terminology and text (was #1022 Summary: Should dot be permitted in phrases?)
Sender: [email-masked]

I have assigned ticket #1031 to this suggestion.

FYI, the numbers quoted are:

FYI 18 - RFC 1983 Internet Users' Glossary (1996)
FYI 36 - RFC 2828 Internet Security Glossary (2000)
BCP 14 - RFC 2119 Key words for use in RFCs to Indicate Requirement Levels
BCP 26 - RFC 2434 Guidelines for Writing an IANA Considerations Section
                  in RFCs
BCP 72 - RFC 3552 Guidelines for Writing RFC Text on Security
                  Considerations
BCP 104 - RFC 4084 Terminology for Describing Internet Connectivity

I am looking forward to your specific suggestions on changes based on these 
documents.

This still does NOT qualify as suggesting a text change.

--On torsdag, juni 09, 2005 11:17:46 -0400 Bruce Lilly <[email-masked]> 
wrote:

> On Thu June 9 2005 09:40, Harald Tveit Alvestrand wrote:
>
>> Then make multiple suggestions, or say "I suggest we make this change,
>> and  the following sections need to be changed because of that - details
>> to be  discussed when the basic change is accepted".
>
> I suggest that we use standard terminology and section text as used in
> the primary normative references and as documented in current Internet
> FYI/BCP glossaries and guidelines (notably FYIs 18, 36, BCPs 14, 26, 72,
> 104) and relevant other guidelines and procedures (notably ID-checklist,
> ID-Guidelines, "instructions to RFC Authors").
>
> The following sections of USEFOR version -03 need to be changed because
> of that - details to be discussed when the basic change is accepted:
>   1.4, 1.5, 1.6, 2.1, 2.2, 2.3, 3, 3.1, 3.1.1, 3.1.2, 3.1.3, 3.1,4, 3.1.5,
>   3.1.6, 3.1.7, 3.2, 3.2.1, 3.2.2, 3.2.3, 3.2.4, 3.2.5, 3.2.6, 3.2.7,
>   3.2.8, 3.2.9, 3.2.10, 3.2.11, 3.2.12, 3.2.13, 3.3, 3.3.1, maybe 4,
>   probably 5, An IANA Considerations section is missing but REQUIRED
>   (per ID-Guidelines and "Instructions to RFC Authors"), 6.2, Appendix A.
>
> Note that most of the changes are small and could be performed
> automatically. In other cases, a section needs to be moved (Appendices
> are supposed to appear before references) or added (IANA Considerations).
> Several of the changes are due to requirements which are not currently
> met by the draft.
>

Date: Thu, 09 Jun 2005 19:03:30 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1030 Re: backward compatibility and handling incompatibilities (was #1022 Summary: Should dot be permitted in phrases?)
Sender: [email-masked]

I have created ticket #1030 with this text.

I will attempt to attach any suggestions for specific modifications to the 
mentioned sections on this basis to that ticket; in order to make this 
easier, please keep the subject line with #1030 in it.

I'm still waiting to hear what specific changes to 2.1 Bruce is suggesting.

--On torsdag, juni 09, 2005 11:34:02 -0400 Bruce Lilly <[email-masked]> 
wrote:

> On Thu June 9 2005 09:40, Harald Tveit Alvestrand wrote:
>
>> Then make multiple suggestions, or say "I suggest we make this change,
>> and  the following sections need to be changed because of that - details
>> to be  discussed when the basic change is accepted".
>
> I suggest we conform as closely as possible to our primary normative
> references and published extensions, and that any deviations are
> clearly indicated via the use of Implementation notes of the general
> form suggested here by Mark Crispin:
> � � http://www.imc.org/ietf-usefor/2003/Feb/0493.html
> and that use of BCP 14 imperatives in conjunction with any such
> deviations in fact conform to the usage requirements in BCP 14.
>
> The following sections need to be changed because of that - details
> to be discussed when the basic change is accepted:
>   (probably 1.4), 2.1, 2.2, (small note in 3.6), 3.1.1, 3.1.2,
>   3.1.3, 3.1.4, 3.2, 3.2.1, 3.2.3, 3.2.5
>

Date: Thu, 09 Jun 2005 12:23:53 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: documenting new fields (was #1022 Summary: Should dot be permitted in phrases?)
Sender: [email-masked]

Bruce Lilly wrote:

> On Thu June 9 2005 09:40, Harald Tveit Alvestrand wrote:
> 
> 
>>Then make multiple suggestions, or say "I suggest we make this change, and 
>>the following sections need to be changed because of that - details to be 
>>discussed when the basic change is accepted".
> 
> 
> I suggest that the draft text be revised to conform to the Chair's defined
> scope as presented in
>   http://www.imc.org/ietf-usefor/2004/Jun/0370.html
> specifically, that new fields must clearly be marked as such.
> 
> The following sections need to be changed because of that - details to be
> discussed when the basic change (to the draft, no change in policy is
> suggested) is reiterated:
>   3.2.11
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/Archive.htm

> 3.2.12
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/User-Agent.htm

> 3.2.13
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/Injection-Info.htm

Date: Thu, 09 Jun 2005 12:22:18 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: backward compatibility and handling incompatibilities (was #1022 Summary: Should dot be permitted in phrases?)
Sender: [email-masked]

Bruce Lilly wrote:

> On Thu June 9 2005 09:40, Harald Tveit Alvestrand wrote:
> 
> 
>>Then make multiple suggestions, or say "I suggest we make this change, and 
>>the following sections need to be changed because of that - details to be 
>>discussed when the basic change is accepted".
> 
> 
> I suggest we conform as closely as possible to our primary normative
> references and published extensions, and that any deviations are
> clearly indicated via the use of Implementation notes of the general
> form suggested here by Mark Crispin:
>     http://www.imc.org/ietf-usefor/2003/Feb/0493.html
> and that use of BCP 14 imperatives in conjunction with any such
> deviations in fact conform to the usage requirements in BCP 14.
> 
> The following sections need to be changed because of that - details
> to be discussed when the basic change is accepted:
>   (probably 1.4), 
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/Syntax_Notation.htm

> 2.1
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/Base.htm

> 2.2
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/Headers.htm

> (small note in 3.6)
Bruce, I can find no section with that number.

> 3.1.1
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/From.htm

> 3.1.2,
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/Date.htm

> 3.1.3
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/Message-ID.htm

 > 3.1.4
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/Subject.htm

 > 3.2
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/Optional_Headers.htm

 > 3.2.1
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/References.htm

 > 3.2.3
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/Expires.htm

 > 3.2.5
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/Supersedes.htm

Date: Thu, 09 Jun 2005 12:18:00 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
CC: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: standard terminology and text (was #1022 Summary: Should dot be permitted in phrases?)
Sender: [email-masked]

Bruce Lilly wrote:

> On Thu June 9 2005 09:40, Harald Tveit Alvestrand wrote:
> 
> 
>>Then make multiple suggestions, or say "I suggest we make this change, and 
>>the following sections need to be changed because of that - details to be 
>>discussed when the basic change is accepted".
> 
> 
> I suggest that we use standard terminology and section text as used in
> the primary normative references and as documented in current Internet
> FYI/BCP glossaries and guidelines (notably FYIs 18, 36, BCPs 14, 26, 72,
> 104) and relevant other guidelines and procedures (notably ID-checklist,
> ID-Guidelines, "instructions to RFC Authors").
> 
> The following sections of USEFOR version -03 need to be changed because
> of that - details to be discussed when the basic change is accepted:

Bruce listed all sections of version -03, except for

1.  Introduction
1.1  Basic Concepts
1.2  Scope
1.3  Requirements Notation
2.  Format
6.1  Normative References

Links to separated sections of version 03 are available at:

     http://www.mibsoftware.com/userkt/usefor/usefor-usefor-03/

Each page has a diff to the immediately previous version, and links
to older and newer versions.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: documenting new fields (was #1022 Summary: Should dot be permitted in phrases?)
Date: Thu, 9 Jun 2005 11:55:07 -0400
Cc: Harald Tveit Alvestrand <[email-masked]>
Sender: [email-masked]

On Thu June 9 2005 09:40, Harald Tveit Alvestrand wrote:

> Then make multiple suggestions, or say "I suggest we make this change, and 
> the following sections need to be changed because of that - details to be 
> discussed when the basic change is accepted".

I suggest that the draft text be revised to conform to the Chair's defined
scope as presented in
  http://www.imc.org/ietf-usefor/2004/Jun/0370.html
specifically, that new fields must clearly be marked as such.

The following sections need to be changed because of that - details to be
discussed when the basic change (to the draft, no change in policy is
suggested) is reiterated:
  3.2.11, 3.2.12, 3.2.13

Date: Thu, 9 Jun 2005 11:50:37 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: [email-masked]
Subject: Re: backward compatibility and handling incompatibilities (was #1022 Summary: Should dot be permitted in phrases?)
Sender: [email-masked]

On Thu, 9 Jun 2005, Bruce Lilly wrote:
> I suggest we conform as closely as possible to our primary normative
> references and published extensions...
> The following sections need to be changed because of that - details
> to be discussed when the basic change is accepted...

Personally, I find it impossible to evaluate this suggestion without
more detail.  The suggestion as phrased is nearly meaningless, given
that there is no clear indication of what is to be changed or why.

                                                          Henry Spencer
                                                       [email-masked]

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
Date: Thu, 9 Jun 2005 11:47:02 -0400
Cc: [email-masked]
Sender: [email-masked]

On Thu June 9 2005 09:40, Harald Tveit Alvestrand wrote:

> Then make multiple suggestions, or say "I suggest we make this change, and 
> the following sections need to be changed because of that - details to be 
> discussed when the basic change is accepted".

I suggest that no later than the publication of the next draft revision,
the editors carefully spell-check, proofread, and edit for consistency
the draft (e.g. section cross-references are correct, references listed
in the references sections are actually referenced, informative/normative
references are categorized correctly, etc.).

The following excerpts illustrate some of the issues; these issues should
be corrected by the Editors before being presented to the WG:

1.2: "An best" is not English

ISO 3166 is used normatively in section 3.2.6 but is listed as informative.

The suggestion of course affects the entire draft.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: backward compatibility and handling incompatibilities (was #1022 Summary: Should dot be permitted in phrases?)
Date: Thu, 9 Jun 2005 11:34:02 -0400
Cc: Harald Tveit Alvestrand <[email-masked]>
Sender: [email-masked]

On Thu June 9 2005 09:40, Harald Tveit Alvestrand wrote:

> Then make multiple suggestions, or say "I suggest we make this change, and 
> the following sections need to be changed because of that - details to be 
> discussed when the basic change is accepted".

I suggest we conform as closely as possible to our primary normative
references and published extensions, and that any deviations are
clearly indicated via the use of Implementation notes of the general
form suggested here by Mark Crispin:
� � http://www.imc.org/ietf-usefor/2003/Feb/0493.html
and that use of BCP 14 imperatives in conjunction with any such
deviations in fact conform to the usage requirements in BCP 14.

The following sections need to be changed because of that - details
to be discussed when the basic change is accepted:
  (probably 1.4), 2.1, 2.2, (small note in 3.6), 3.1.1, 3.1.2,
  3.1.3, 3.1.4, 3.2, 3.2.1, 3.2.3, 3.2.5

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: standard terminology and text (was #1022 Summary: Should dot be permitted in phrases?)
Date: Thu, 9 Jun 2005 11:17:46 -0400
Cc: Harald Tveit Alvestrand <[email-masked]>
Sender: [email-masked]

On Thu June 9 2005 09:40, Harald Tveit Alvestrand wrote:

> Then make multiple suggestions, or say "I suggest we make this change, and 
> the following sections need to be changed because of that - details to be 
> discussed when the basic change is accepted".

I suggest that we use standard terminology and section text as used in
the primary normative references and as documented in current Internet
FYI/BCP glossaries and guidelines (notably FYIs 18, 36, BCPs 14, 26, 72,
104) and relevant other guidelines and procedures (notably ID-checklist,
ID-Guidelines, "instructions to RFC Authors").

The following sections of USEFOR version -03 need to be changed because
of that - details to be discussed when the basic change is accepted:
  1.4, 1.5, 1.6, 2.1, 2.2, 2.3, 3, 3.1, 3.1.1, 3.1.2, 3.1.3, 3.1,4, 3.1.5,
  3.1.6, 3.1.7, 3.2, 3.2.1, 3.2.2, 3.2.3, 3.2.4, 3.2.5, 3.2.6, 3.2.7,
  3.2.8, 3.2.9, 3.2.10, 3.2.11, 3.2.12, 3.2.13, 3.3, 3.3.1, maybe 4,
  probably 5, An IANA Considerations section is missing but REQUIRED
  (per ID-Guidelines and "Instructions to RFC Authors"), 6.2, Appendix A.

Note that most of the changes are small and could be performed automatically.
In other cases, a section needs to be moved (Appendices are supposed to
appear before references) or added (IANA Considerations).  Several of the
changes are due to requirements which are not currently met by the draft.

Date: Thu, 09 Jun 2005 16:54:10 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Admin: Posters this week
Sender: [email-masked]

FYI, because some might find it interesting:
Here's the list of posters to the mailing list for the last 7 days.

Searching for msgs since 02-Jun-2005
  1   38  23.31 "Charles Lindsey" <[email-masked]>
  2   27  39.88 Bruce Lilly <[email-masked]>
  3   22  53.37 Harald Tveit Alvestrand <[email-masked]>
  4   22  66.87 Frank Ellermann <[email-masked]>
  5   18  77.91 Russ Allbery <[email-masked]>
  6    9  83.44 "Forrest J. Cavalier III" <[email-masked]>
  7    8  88.34 John Stanley <[email-masked]>
  8    6  92.02 Henry Spencer <[email-masked]>
  9    5  95.09 Seth Breidbart <[email-masked]>
 10    4  97.55 "Ruud H.G. van Tol" <[email-masked]>
 11    2  98.77 "Forrest J. Cavalier III" <[email-masked]>
 12    1  99.39 Richard Clayton <[email-masked]>
 13    1 100.00 Eivind Tagseth <[email-masked]>

I'll provide this as a regular (weekly?) feature.... it's nice to have a 
running idea of who "everyone" is.....

                      Harald

Date: Thu, 09 Jun 2005 16:47:54 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1008 USEFOR 1.5 References: Final text on "followup"
Sender: [email-masked]

In reviewing the thread, the text that seems to come closest to capturing 
the (VERY) rough consensus of the group is:

 A "followup" is an article containing a response to the contents of
 an earlier article, its "precursor". Every followup includes a
 References header identifying that precursor (but note that non-followup
 articles may also use a References header).

I have also noted that there is no discernible consensus for the text 
Charles suggested addition to the top of the References section (attempting 
to explain what References is used for); it will not be added as part of 
this decision.

Unless there are technical issues that have not been raised before, this is 
the instruction from the WG chairs to the editor of the USEFOR document, 
and the subject of the definition of "followup" is now closed.

Any new messages on the subject of the definition of "followup" that do not 
raise new technical issues will be treated as off-topic postings.

                     Harald & Alexey

Date: Thu, 09 Jun 2005 16:14:40 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Forward compatibility (Re: CFWS in References header)
Sender: [email-masked]

--On onsdag, juni 08, 2005 13:52:02 +0000 Charles Lindsey 
<[email-masked]> wrote:

> I think you need to consider a more general issue.
>
> There are various places where our draft mandates behaviour that is not in
> accordance with current practice; in all such cases, it is necessary to
> consider what transitional arrangements need to be put in place.
>
> The issue here is where we are mandating that something MUST be accepted,
> and where the transitional behaviour is of the form "but do not generate
> it yet", for some value of "yet". The cases in this category are:
>
> 1. <comment>s in References.
> 2. Naked '.' in <phrase>s.
> 3. Folding of the Newsgroups header.
> So we either say "MUST NOT generate" for them all, or
> we say "SHOULD NOT generate" plus verbiage, or
> we say that Newsgroups (for example) is urgent, and gets "SHOULD NOT",
> whereas nobody is actually wanting to put <comment>s in References, so
> "MUST NOT" will do there.

After this many years of this WG trying to provide *one* revision, I wonder 
about the likelyhood of there ever being another......

What's more likely to happen is that someone produces a News extension, 
which updates USEFOR saying "this MUST NOT generate is now changed to a 
MAY".
This is standard practice for protocol extensions.

> But it has also been pointed out (Harald will have seen an email from
> Alexey on this) that even "SHOULD NOT" is too strong for Newsgroups
> folding, since it gives no incentive to implement the "MUST accept".
>
> So there is the gamut of possibilities. They need to be considered
> together.

IMHO, and speaking for myself-as-participant only:

Realistically, either newsgroups folding is useful or it is not.
If it is not, there is no incentive to implement it no matter what the 
standard says.
If it is useful, there is incentive to implement it.
If it is very useful, someone might even be bothered to write an 1-page RFC 
saying "Newsgroup folding is now permitted".

But I very much doubt it.

We have better things to do than to encourage the implementation of useless 
features.

                         Harald

Date: Thu, 09 Jun 2005 15:40:34 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
Sender: [email-masked]

--On 9. juni 2005 08:36 -0400 Bruce Lilly <[email-masked]> wrote:

>> (Repeat mantra: When arguing a point, plese specify what section you
>> want  changed, what text you want changes to, and what you want it
>> changed to.)
>
> As in Andrew's comments, many of the issues are general and affect
> multiple sections.

Then make multiple suggestions, or say "I suggest we make this change, and 
the following sections need to be changed because of that - details to be 
discussed when the basic change is accepted".

I can imagine a number of different ways in which section 2.1 of USEFOR 
could be formulated based on your comments. But you're the one who wants it 
changed (or, if I have misunderstood you, you don't want it changed). Until 
you propose a change, I can't tell.

                            Harald

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
Date: Thu, 9 Jun 2005 08:36:40 -0400
Cc: [email-masked]
Sender: [email-masked]

On Thu June 9 2005 01:35, Harald Tveit Alvestrand wrote:

> Is this what you are arguing for when you say "exactly as RFC 2822"?
> Or are you arguing that in the specific case of obs-phrase, USEFOR should 
> say "MUST accept, but MUST NOT generate", just like RFC 2822 says?

In the specific case of obs-phrase, the rules should be the same as
RFC 2822, but that need not be achieved by repeating what 2822 says --
indeed a simple reference to 2822 is preferable.  This particular
provision of 2822 is a parse-only provision made to accommodate
previously illegal but commonly generated syntax errors.  One WG
participant has made a big deal out of that minor point; it is clear
from 2822 text what the rationale is for its inclusion, as it is
clear that the obs- syntax is a parse grammar separate from the
generate grammar.

In general, as I have said many times, we should conform to 2822
(and other relevant RFCs), and the above is a particular instance
of that general principle.  That principle applies to various aspects
of our work including terminology (2822, MIME, FYIs 18 & 36, etc.) ,
message format, and so on.  The reasons for doing so have been discussed
in this WG many times by many participants and include:
 o the principle of least astonishment
 o backward compatibility with widely deployed, Standards Track
   protocols (e.g. IMAP) [I note that our charter specifies "with
   particular attention to backward compatibility"]
 o likelihood of achieving IETF consensus and passing IESG review
A sampling of relevant discussions from the WG archives:
 http://www.imc.org/ietf-usefor/2003/Feb/0209.html
 http://www.imc.org/ietf-usefor/2003/Feb/0374.html
 http://www.imc.org/ietf-usefor/2003/Feb/0493.html
 http://www.imc.org/ietf-usefor/2003/Feb/0827.html
 http://www.imc.org/ietf-usefor/2004/Jun/0370.html
----------- excerpts (not in web archive) -----------------
Subject: Re: Final Last Last Last Call
From: Andrew Gierth <[email-masked]>
Date: 01 Aug 2002 19:27:44 +0100
Message-ID: <[email-masked]>

[...]
Back in November last year I made this proposal:

----
  I propose the following major changes:

  1. Strike all basic syntax and punt it back to the mail standards.
  Remove any changes that would have the effect of allowing UTF-8 in
  headers meaningful to mail. Instead, explicitly provide for the use
  of UTF-8 in Newsgroups and Followup-To, and provide _exactly one_
  mapping to a format compatible with mail, to allow gateways to handle
  it, and to allow servers to reject Newsgroups lines which have been
  munged for mail purposes. As regards UTF-8 in other fields, point out
  that it should be a future direction, that there are currently issues
  with interoperability with mail, but that both standards should be
  aiming to migrate towards UTF-8 (or some such wording).

  2. Strike all statements of policy or default policy, and all
  statements that refer to actions of the administrator rather than the
  software. In a few of the cases it may be appropriate to substitute
  recommendations.

  3. Generally aim to reduce the document's size; it is far too big.

  4. Strike the new path stuff and standardise the existing practice.
  Reliance on '!' turns out to be _extremely_ widespread.
----

At the time, I count only four direct responses to this proposal:
expressions of agreement from Russ Allbery and Ralph Babel, and
disagreement from Henry Spencer and Charles Lindsey. Personally I
regard that as indicating pretty strongly that there is not, in fact,
a genuine consensus behind the current document, and the subsequent
(but months later) adoption of changes that parallel my proposal (1)
(but without the simplifying benefits of referring back to the mail
standards) adds weight to this.

-- 
Andrew.
--------------------------------------------------

> (Repeat mantra: When arguing a point, plese specify what section you want 
> changed, what text you want changes to, and what you want it changed to.)

As in Andrew's comments, many of the issues are general and affect
multiple sections.

Date: Thu, 09 Jun 2005 09:04:18 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

--On 8. juni 2005 19:11 -0400 Bruce Lilly <[email-masked]> wrote:

>> >    A followup MUST have a References-header, and an article that is not
>> >    a followup MUST NOT have a References-header.
>>
>> Bruce,
>>
>> remember the article I posted on May 27?
>
> Yes, I remember; I suspect that there are more than 9 people on the
> mailing list, and I know some (e.g. me) didn't respond.  Some of us
> monkeys are tired of chasing the same weasel around the same mulberry
> bush, year after year.

Yep. That's why I'm trying to nail it to the tree. And make it stay nailed.

FWIW, the mailing list had 49 members a few weeks ago, and 20 people have 
so far responded to my call for participants.

> IIRC, the discussion after your message went
> down the usual "thread by Subject" rathole.
>
> If you like you can put me in the "1)" column with a note that there
> is disagreement with the alternatives for the reasons mentioned in
> recent discussion (those alternatives do not provide an objective
> term, hence not a useful term).

Thanks.

Date: Thu, 09 Jun 2005 07:35:06 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
Sender: [email-masked]

--On onsdag, juni 08, 2005 19:14:01 -0400 Bruce Lilly <[email-masked]> 
wrote:

> On Wed June 8 2005 07:13, Harald Tveit Alvestrand wrote:
>
>> - Bruce Lilly argues against a deviation (I think).
>
> To clarify, my strong preference is "exactly as RFC 2822".
> I'm willing to entertain other options, so long as there is an
> accompanying well-reasoned argument for a deviation from 2822.
>

"exactly as RFC 2822" means changing the "MAY accept" for the obs-syntax in 
section 2.1 to a "MUST accept".

RFC 2822 section 1.3 says:

   Section 4 of this document specifies an "obsolete" syntax.  There are
   references in section 3 to these obsolete syntactic elements.  The
   rules of the obsolete syntax are elements that have appeared in
   earlier revisions of this standard or have previously been widely
   used in Internet messages.  As such, these elements MUST be
   interpreted by parsers of messages in order to be conformant to this
   standard.  However, since items in this syntax have been determined
   to be non-interoperable or to cause significant problems for
   recipients of messages, they MUST NOT be generated by creators of
   conformant messages.

Is this what you are arguing for when you say "exactly as RFC 2822"?
Or are you arguing that in the specific case of obs-phrase, USEFOR should 
say "MUST accept, but MUST NOT generate", just like RFC 2822 says?

(Repeat mantra: When arguing a point, plese specify what section you want 
changed, what text you want changes to, and what you want it changed to.)

                       Harald

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: CFWS in References header
Date: Thu, 9 Jun 2005 00:36:36 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 8 2005 09:52, Charles Lindsey wrote:

> The cases in this category are:
> 
> 1. <comment>s in References.
> 2. Naked '.' in <phrase>s.
> 3. Folding of the Newsgroups header.
> So what is an appropriate transitional arrangement for those three?
[...] 
> Our first reaction was to propose some cutoff date after which they could
> be freely generated, but we were told that the IESG did not like Flag
> Days.
> 
> OTOH, we did not like the idea of "do not generate until the next standard
> comes out" (expressible as "MUST NOT generate") because #3, in particular,
> really needed a more urgent fix than that [...]

It is possible to go from Proposed to Draft in six months.

> So the position we took was that people should start generating them when,
> in their opinion, they would work well enough [...].

I suspect that in a Standards Track document "do whatever you please"
isn't going to pass muster.  Less so, I suspect, than a flag day.
  
> So there is the gamut of possibilities. They need to be considered
> together.

No, there are other possibilities, including, but not limited to:
o provide a roadmap (X at Proposed, Y at Draft, Z at Full Standard).
  Harald can probably give some guidance as to how well that would be
  received by the IESG, but failing something like that, there is a
  good chance that each such change would require a reset to Proposed
o Implementation notes as suggested by Mark Crispin:
    http://www.imc.org/ietf-usefor/2003/Feb/0486.html
    http://www.imc.org/ietf-usefor/2003/Feb/0493.html
    http://www.imc.org/ietf-usefor/2003/Feb/0554.html
o Implementations are of course free to ignore the specification provided
  they don't claim to conform to it
o As already suggested in discussion, specifically w.r.t. dot in a
  phrase, "do nothing" (aside from "[RFC2822]") may be a reasonable
  option

Specifically regarding the References field and comments, what is the
scope of the issue?  Transport agents are unaffected.  Any
considerations appear to be confined to UAs, and as the field plays
no critical role in protocol (certainly not warranting BCP 14
imperatives), "[RFC2822]" (for implementations that wish to claim
conformance) may suffice [note that a) comments have always been
permitted per RFC 822 as adopted by 850/1036, b) many UAs are combined
mail/news UAs and already necessarily handle comments (unless they're
broken UAs), and c) message-generating UAs generally do not insert
comments in References fields, so the only likely source is a human
manually editing a field].  It doesn't seem worth spending much time
on -- not beyond an implementation note at most.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Wed, 8 Jun 2005 23:37:09 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 8 2005 10:00, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >For the purpose of the USEFOR (syntax & semantics) document, it suffices to
> >specify the syntax and semantics (obviously?).  For the purpose of the
> >USEPRO (protocol) document, one could add an implementation note such as:
> 
> >  When modifying a References field, care must be taken to ensure that the
> >  result complies with the syntax specification for the field, specifically
> >  that line length limits are observed and that no msg-id is truncated or
> >  subjected to internal line folding.
> 
> I agree that any mention of pruning is a matter for USEPRO rather than
> USEFOR. What I currently propose to say, at Frank's request (see suggested
> text posted earlier) is that you MAY prune (subject to keeping the first
> and the last two), and that you MUST prune to keep it within 998
> (including folded lines, because of the kind of breakage I described).
> 
> I think it goes without saying that whatever results must be syntactically
> correct, and so there can be no folding within a msg-id.

The specific case you mentioned involved a transport agent that folded a
long line at 990 (N.B. not 998) octets, which happened to be in the middle
of a msg-id.  The suggested text above specifically covers that case.  If
you're not now concerned about that, one wonders why you brought it up...

To: [email-masked]
Subject: Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 08 Jun 2005 17:50:36 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> It is just not realistic to suppose that servers in Australia will
> interpret "EST" according to RFC 2822 (they will probably just hand the
> zone off to the OS, and for sure that will take an Australian view of
> time zones).

I have never seen any news software that does this, and furthermore, given
that there is no OS facility short of changing TZ for the process that
could make head or tails of "EST" on Unix at least, I disbelieve in the
existence of software that takes this approach.

Software uses a hard-coded table listing time zone abbreviations and the
corresonding offsets.  That's the way all news (and e-mail, for that
matter) software I've ever seen has done this.  It's possible that there's
something out there that doesn't do things this way, but I'm still very
dubious that any sort of fallback to some OS facility is involved.

Furthermore, if you set TZ to EST in Australia, you most likely get the
same US zone as you would get anywhere else, since OS manufacturers are
not in the habit of modifying POSIX time zones depending on your locale.
For actual time zone configuration, nearly everyone switched to the Olson
approach a long time ago (which means you use things like
America/New_York, not EST).

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Wed, 8 Jun 2005 20:43:27 -0400
Sender: [email-masked]

On Wed June 8 2005 13:46, John Stanley wrote:
> 
> "Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:
> 
> >And may I also say that the text that Bruce has quoted from the earlier
> >drafts was written *before* someone came on this list to point out the use
> >of References in multi-part FAQs (a practice I had not been aware of, but
> >which seemed eminently sensible).
> 
> It is such a common practice that I cannot imagine that anyone familiar 
> enough with news operations to be participating in rewriting the news 
> standards would not know about it. I certainly did. I don't recall anyone 
> saying "what a great surprise" when this was pointed out. We did say "we 
> ought not prohibit this", and that's why I proposed a few word change to 
> the definition of followup to include existing practice but not exclude
> the ability to detect followups altogether.

Matches my recollection.

> >The *whole* of our discussions, proposed changes, etc. since then have
> >arisen as a direct result of what that someone (I cannot recall who it
> >was) said.
> 
> Removing the MUST/MUST NOT section was not the result of any discussion.

Ditto. 

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Date-time yet again
Date: Wed, 8 Jun 2005 20:40:18 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 8 2005 10:42, Charles Lindsey wrote:

> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >On Mon June 6 2005 12:36, Charles Lindsey wrote:
> 
> >> >> You MUST NOT generate "EST" or any of the other things in <obs-zone>.
> >> 
> >> >Already covered by 2822.
> >> 
> >> Indeed, but worth reiterating because it would actually break the
> >> protocols.
> 
> >Precisely how would "PDT" "break the protocols" in a way that "UT"
> >would not?
> 
> I am not aware what other zones use "PDT", but [...]

Charles, stop trying to evade the issue.  You made a claim with no
supporting evidence or reasoning, and you have been politely asked
to produce such evidence or reasoning.  Instead, you have tried to
evade the issue.  Will you kindly either provide a supporting line
of reasoning for your stated claim about PDT ("any of the other
things in <obs-zone>"), or failing that, retract your claim.

Your changing the subject to avoid addressing the issue is a waste
of time, which is a scarce resource.  Nevertheless, to put this
to bed...
> It is just not realistic to suppose that servers in Australia will
> interpret "EST" according to RFC 2822 (they will probably just hand the
> zone off to the OS, and for sure that will take an Australian view of
> time zones).

No, as Russ has already pointed out to you, the zones specified
in RFCs 561, 724, 733, 822 (as amended by 1123) and 2822 have a
specific protocol-defined meaning in date-time.  They do not, and
never have had, semantics of "just hand the zone off to the OS". As
John os fond of pointing out, software that does such things is just
plain broken and is therefore irrelevant to our standards work.

> >> I could easily be persuaded to promote 
> >> it to MUST NOT if someone asks for it. Are you formally proposing that?
> 
> >I have said about as clearly as possible that I feel that we should
> >conform to RFC 2822 format as closely as possible.  Since 2822 clearly
> >says MUST NOT generate for alphabetic zone indication, yes, that is
> >what I have proposed.  Specifically, I propose no deviations from
> >RFC 2822 regarding date-time format.
> 
> Then you have failed to answer the question I asked, which was in relation
> to "UT" only.

Do you see a special exception anywhere in RFC 2822 for "UT" vis-a-vis
MUST NOT generate?  Exactly which part of the word "yes" do you fail to
grasp?
 
> "GMT" and "UT" are safe, in that _everybody_ understands them
[...]
> "GMT" is a special case, insofar as it is not ambiguous and is widely
> used.

Last time I checked, there were at least a dozen well-known places named
Greenwich.  To use your (fallacious) argument, a (broken) news server
could presume that GMT meant the time at some local "Greenwich".

> >> This WG decided. more years ago than I care to remember, that it was
> >> ludicrous to REQUIRE all News software to be upgraded to accept
> >> constructes that had Never Ever been seen in Netnews.
> 
> >You are mistaken in your implied assertion that handling of the standard
> >alphabetic zones for parsing would would be "constructes [sic] that had
> >Never [sic] Ever [sic] been seen in Netnews".
> 
> Again, you avoid the issue by changing the question.

I have not changed anything, including the typos.

> We decided that 
> obs-syntax in general was a Bad Thing, and it would be an imposition to
> demand that news agents should accept it all regardless (a point you
> yourself once made, as you will doubtless recall).

I do not recall any such thing (*illegal* syntax is another matter,
and a requirement for parsing coupled with a prohibition against
generation in a document produced by a body that has no enforcement
arm cannot constitute a "demand").
 
[quotes from RFCs 850 and 1036 noting that date fields must conform to
RFC 822]
> So what? There may well have been articles with strange zones in the past;

We're not talking about "strange zones", we're discussing (now deprecated)
standard alphabetic zone abbreviations.  The relevant news article format
specification RFCs have always incorporated those -- there are in date-time
no "constructes [sic] that had Never [sic] Ever [sic] been seen in Netnews"
as you claimed.  Your accusation that I am "changing the question" is out
of place -- you are again trying to change the subject instead of either
defending or retracting your earlier claim.  No dice.  Stick to the subject.
Either substantiate your claim that using RFC 2822 date-time verbatim
would "REQUIRE all News software to be upgraded to accept constructes [sic]
that had Never [sic] Ever [sic] been seen in Netnews" or retract that
claim.  You may start with any particular category of "all News software"
and proceed to the others in any order of your choosing.

> >As Russ has noted, there is *substantial* existing usage of alphabetic
> >zone abbreviations.
> 
> No, he has not noted that. Usage of GMT is indeed "substantial", but not
> the others.

Quote from <[email-masked]>:

   There is substantial existing usage of PST/MST/EST too.

You have called me a liar.  I have substantiated my claim.  Are you
also calling Russ a liar?

> >> That decision have never been challenged since then. Why are you raising
> >> it now?
> 
> >Again, this has been discussed before.
> 
> References please.

---------------------------------------------------
Message-ID: <[email-masked]>
Date: Tue, 19 Aug 1997 00:18:48 +0100
From: Ian Davis <[email-masked]>
Resent-Message-ID: <"COQv_2.0.0F3.rWD-p"@ambient>
Resent-From: [email-masked]

Here's my first cut at a draft for the Date header. I have taken two
liberties with the format of the time zone the implications of which
will need examining. Both of these are with a view to bringing the date
format closer to the ISO 8601 date and time standard and also to the
latest drums draft. They are: removing the space between the time and
the timezone and removing UT and GMT and other timezone names. I am
fairly confident that these changes are backwards compatible
[...]
---------------------------------------------------
Date: Tue, 15 May 2001 21:47:44 -0700
From: Brad Templeton <[email-masked]>
Subject: Re: RFC 2822 consequentials
Message-ID: <[email-masked]>

[...]
> Charles Lindsey <[email-masked]> writes:
> 
> > [There are just a few tidyings up in that syntax; nothing significant.
> > Observe that we rely on RFC 2822 for the syntaxt of date-time, excluding
> > their obs-syntax. This means we do not allow 2-digit years (and rightly
> > so) but we also do not allow ancient timezones such as GMT (it all has
> > to be +0100 now).  Do we want to relax that a little?]
[...]

Well, since lots of people are generating timezones like this, it would
have to be a rule to accept them, but not generate them.
---------------------------------------------------
Message-ID: <[email-masked]>
Date: Wed, 11 Sep 2002 02:39:27 -0400
From: Bruce Lilly <[email-masked]>
Subject: More draft 8 issues

[...]
    Also, 2.4.2's description of the 2822 obsolete
    syntax is somewhat puzzling; 2822 clearly states that the obs-
    forms MUST NOT be used in generation of new text messages, but
    MUST be recognized (so as to enable parsing of old messages).
    2.4.2 claims that the draft excludes the obsolete syntax, then
    says that it MAY be recognized, and then there are instances
    where obsolete syntax is *mandated* by the draft (e.g. "UT" and
    "GMT" for zone in 5.1).
[...]
---------------------------------------------------
Subject: Re: draft-kohn-news-article-01a.txt
From: Russ Allbery <[email-masked]>
Date: Mon, 10 Mar 2003 09:42:21 -0800
Message-ID: <[email-masked]>

[...]
> Section 3.3:

> Singling out "GMT" seems odd; other 822 alphabetic zones are also in
> common use.  It would perhaps be both simpler and better to say 2822's
> obs-zone MUST be recognized (but MUST NOT be generated).

I think you're probably right here.  There was extensive discussion of
this a while back in the USEFOR archives, including quite a few real
numbers on the Date headers seen on the wire.
[...]
---------------------------------------------------
That's a sample of discussions of the same date-time, obs-zone
issue spanning six *YEARS* (not including the current discussion).

> >> IIRC, it was you who asked for <obs-zone> to be brought within our
> >> standard.
> 
> >References, please.
> 
> www.imc.org/ietf-usefor/2003/Feb/0221.html

That message has nothing whatsoever to do with obs-zone (the term is
nowhere mentioned in that message, nor is the subject matter).

To: [email-masked]
Subject: Re: #1028 USEFOR 3.1.2: What zones should be on the MUST accept list?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 08 Jun 2005 17:16:00 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:

> That reminds me; the corresponding [ECMP]ST (case-insensitive) versions
> should probably also be counted.  Some places (e.g. the U.S. state of
> Indiana) do not observe daylight savings time, and some software is so
> broken that it uses the wrong zones, or fails to take account of (or is
> configured to fail to take account of) DST.  Indiana, incidentally, uses
> EST year-round.

Okay, okay, fine.  :)  #JUNK# are various date lines that could not be
parsed into a time zone by my simple algorithm (most of them didn't
specify a time zone at all, but some of them had other problems, like not
zero-padding the seconds in the timestamp).

Hopefully this will give people a new appreciation for just how much
bizarre junk is out there on the wire.

      #JUNK#:    1553 (00.0%)
      +-0400:      10 (00.0%)
        +000:       8 (00.0%)
       +0000:  165048 (04.5%)
       +0001:       1 (00.0%)
       +0028:       1 (00.0%)
       +0100:  190268 (05.2%)
       +0102:       1 (00.0%)
       +0200:  563629 (15.3%)
       +0300:   12204 (00.3%)
       +0307:       1 (00.0%)
       +0400:    5279 (00.1%)
       +0430:      21 (00.0%)
       +0500:     477 (00.0%)
       +0530:     881 (00.0%)
       +0545:     421 (00.0%)
       +0600:     675 (00.0%)
       +0630:       1 (00.0%)
       +0700:    1174 (00.0%)
       +0800:   39837 (01.1%)
       +0900:    7291 (00.2%)
  +0900(KST):       1 (00.0%)   # This is legit, just a flaw in my script
       +0930:    2695 (00.1%)
       +0950:      77 (00.0%)
        +100:       2 (00.0%)
       +1000:   29806 (00.8%)
       +1030:       3 (00.0%)
       +1100:      75 (00.0%)
       +1200:    8782 (00.2%)
       +1300:       5 (00.0%)
       +1400:      18 (00.0%)
       -0000:   18257 (00.5%)
       -0001:      25 (00.0%)
       -0002:       1 (00.0%)
       -0100:      64 (00.0%)
       -0200:      63 (00.0%)
       -0230:     461 (00.0%)
       -0300:    4596 (00.1%)
       -0400:  262862 (07.1%)
       -0500:  122173 (03.3%)
       -0600:   29009 (00.8%)
       -0700: 1116599 (30.3%)
       -0756:       1 (00.0%)
       -0800:  269930 (07.3%)
       -0900:      13 (00.0%)
       -1000:     684 (00.0%)
       -1100:       1 (00.0%)
       -1200:      45 (00.0%)
        -500:       6 (00.0%)
        BRST:       2 (00.0%)
         BST:      59 (00.0%)
         CDT:      22 (00.0%)
        CEST:      18 (00.0%)
         CET:       7 (00.0%)
         CST:    4224 (00.1%)
     EASTERN:       1 (00.0%)
         EDT:   31806 (00.9%)
         EET:       1 (00.0%)
         EST:      40 (00.0%)
          GM:      84 (00.0%)
         GMT:  782196 (21.3%)
        GMT):       7 (00.0%)
    GMT+0000:      35 (00.0%)
       GMT+1:      65 (00.0%)
       GMT+2:       1 (00.0%)
       GMT+6:      60 (00.0%)
       GMT-6:       3 (00.0%)
         JST:       3 (00.0%)
       LOCAL:       3 (00.0%)
         MDT:       1 (00.0%)
        MEST:       1 (00.0%)
         MET:       6 (00.0%)
         MEZ:      56 (00.0%)
         MST:       3 (00.0%)
         NZS:       1 (00.0%)
         PDT:     214 (00.0%)
         PST:    2834 (00.1%)
   UNDEFINED:       9 (00.0%)
          UT:      13 (00.0%)
         UTC:    3733 (00.1%)
�X�_�зǮɶ�:       1 (00.0%)

This data was collected with:

find . -name \*.DAT | xargs cut -f 4 \
    | sed -e 's/.*:[0-9][0-9] //' -e 's/ (.*//' \
    | perl -n /tmp/script

where /tmp/script is:

chomp;
if (\!$_ || / /) {
    $_ = "#junk#";
}
$zones{uc $_}++;
$total++;
END {
    for (sort keys %zones) {
        printf "%12s: %7d (%04.1f%%)\n", $_, $zones{$_},
            $zones{$_} / $total * 100
    }
}

I write better code when I expect to ever use it again.  :)

The large numbers of PST compared to PDT is probably an artifact of my
spool retention; I have a bunch of local newsgroups that I keep forever,
I'm in the Pacific time zone, and I'm at a university where almost all of
our traffic is not during daylight saving time.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1028 USEFOR 3.1.2: What zones should be on the MUST accept list?
Date: Wed, 8 Jun 2005 19:20:17 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 8 2005 07:24, Harald Tveit Alvestrand wrote:

> Russ Allbery computed the following numbers:
[...]
> EDT:   30721
> MDT:       1
> CDT:      21
> PDT:     216
> 
> Which is more than I thought.  MDT, CDT, and PDT are a lot less than I
> thought.  Apparently someone does actually use UT, but maybe only one
> person.  :)
> 
> These numbers could argue for skipping UT and/or including EDT (and 
> presumabily EST, since this measurement was taken in summer).

That reminds me; the corresponding [ECMP]ST (case-insensitive) versions
should probably also be counted.  Some places (e.g. the U.S. state of
Indiana) do not observe daylight savings time, and some software is so
broken that it uses the wrong zones, or fails to take account of (or is
configured to fail to take account of) DST.  Indiana, incidentally, uses
EST year-round.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
Date: Wed, 8 Jun 2005 19:14:01 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 8 2005 07:13, Harald Tveit Alvestrand wrote:

> - Bruce Lilly argues against a deviation (I think).

To clarify, my strong preference is "exactly as RFC 2822".
I'm willing to entertain other options, so long as there is an
accompanying well-reasoned argument for a deviation from 2822.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Wed, 8 Jun 2005 19:11:01 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 8 2005 01:59, Harald Tveit Alvestrand wrote:
> 
> --On 7. juni 2005 14:12 -0400 Bruce Lilly <[email-masked]> wrote:
> >
> > The bottom line is that the net result is what our drafts said
> > for quite some time and which enjoyed consensus, viz.
> >
> >    A followup MUST have a References-header, and an article that is not
> >    a followup MUST NOT have a References-header.
> 
> Bruce,
> 
> remember the article I posted on May 27?

Yes, I remember; I suspect that there are more than 9 people on the
mailing list, and I know some (e.g. me) didn't respond.  Some of us
monkeys are tired of chasing the same weasel around the same mulberry
bush, year after year.  IIRC, the discussion after your message went
down the usual "thread by Subject" rathole.

If you like you can put me in the "1)" column with a note that there
is disagreement with the alternatives for the reasons mentioned in
recent discussion (those alternatives do not provide an objective
term, hence not a useful term).

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: Injection-Info issues
Date: Wed, 8 Jun 2005 18:59:17 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 8 2005 02:29, Frank Ellermann wrote:

> >| MIME security considerations are discussed in [RFC2046].
> >| Note that applying some [RFC2231] extensions for parameters
> >| like multi-line paramters on a boundary parameter as defined
> >| in [RFC2046] might be abused to bypass simple algorithms
> >| trying to analyze MIME parts.
[...]
> That was meant as a request, not only as a proposal.  But the
> wording is of course open for debate.

Wording is a problem, given the fact that 2231 (and errata) have
amended 2045 and 2046.  The issue could more accurately be phrased
as
  naive methods of handling parameters, i.e. those which do not take
  errata, RFC 2231, and more errata into account are likely to botch
  some legal MIME messages.

To keep Charles on his toes, one might also remark about the need to
correctly handle all relevant base message format (RFC 2822) constructs,
and specifically mention that "find the first semicolon" is not a
viable approach.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Wed, 8 Jun 2005 18:51:23 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 8 2005 01:53, Harald Tveit Alvestrand wrote:

> Actually RFC 2822 gets away with only giving a construction algorithm.

But N.B. that is given for a particular case:

   When creating a reply to a message,...

which is not the only case:

   ...and the message identifiers of other messages (for example,
   in the case of a reply to a message which was itself a reply).

(N.B. "for example"). [not arguing for anything other than "[RFC2822]",
just cautioning against reading too much into what 2822 says]

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Wed, 8 Jun 2005 18:45:47 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 8 2005 00:26, Frank Ellermann wrote:
> Charles Lindsey wrote:

> >> Whatever it is, this is not always and not necessarily the
> >> case.
> 
> > Yes it is.
> 
> No, it's not.  It's a msg-id-list.

With the (draft) provision(s) for eliding random ids, it is
just a list.

Without that provision (i.e. if all ids are required to be
retained), *AND* with sufficient external information
(gathered from all referenced articles, therefore requiring
access to all such articles), one could theoretically generate
a directed graph.  It might plausibly contain one or more
cycles.

Even with such extreme measures to generate a graph, the ids
comprising a References field body remain a list.

> > If you are not trying to describe at least a DAG (of which
> > trees and linear lists are a particular case),

No. A tree is not necessarily directed (unrooted trees are not,
rooted trees are).  Lists are sets, and do not possess edges,
much less directed edges. See
http://www.csse.monash.edu.au/~lloyd/tildeAlgDS/Graph/

> It has _no_ idea what the old references were.  And that's only
> the normal case.  Under less normal conditions I could put some
> "related Message-IDs" manually in the references, for "retrieval"
> as you said.  It's strictly my business.

Yes, "other messages" (RFC 2822), not "earlier messages" or
"previous messages".

To: [email-masked]
Subject: Re: CFWS in References header
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 08 Jun 2005 12:11:38 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> OTOH, we did not like the idea of "do not generate until the next
> standard comes out" (expressible as "MUST NOT generate") because #3, in
> particular, really needed a more urgent fix than that (the Newsgroups
> header is the only one of all the mail and news headers that cannot be
> folded, and people are still astounded to be told that it still breaks
> existing software - even INN has only fixed it in the last 12 months
> AFAIAA).

I think I've mentioned this before, but maybe not.  INN still does not
accept folded newsgroup headers, since that would mean accepting an
article that's just going to get rejected by the next news server and fail
silently.  Be conservative in what one generates, etc.

To: [email-masked]
Subject: Re: #1022 Summary: Should dot be permitted in phrases?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 08 Jun 2005 12:04:08 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> Of the flurry of messages on this subject line, I found exactly two
> responses that pertained to the subject of the original ticket:

> - Charles would like to somehow elevate obs-phrase (which allows dot) from
> the "MAY accept/MUST NOT generate" implied by the current text to a status
> that says "MUST accept", and either SHOULD NOT or MUST NOT generate.

> - Bruce Lilly argues against a deviation (I think).

> Are these the only two voices on the subject?

Deviating from RFC 2822 seems *really* pointless here.  Although, again,
this is one of those cases where no standard describes the crap that
people currently use in From headers and that news software accepts.  INN
is almost alone in even bothering to look for something that looks like an
e-mail address, and it only does that on posting.  Things like:

    From: No one

are common-place.  So arguing over including or not including the period
seems like a waste of time; unless we're going to bless "use whatever you
want," (which I'm not arguing in favor of) no deviation from RFC 2822 is
going to get us close enough to existing practice to be worth the pain.

To: [email-masked]
Subject: Re: OT: "UT"
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 08 Jun 2005 11:50:18 -0700
Sender: [email-masked]

Seth Breidbart <[email-masked]> writes:
> "Charles Lindsey" <[email-masked]> wrote:

>> But the clear prepondenance of GMT (21% of the total) is
>> notable. But where is the notorious "EST"?

> It's Daylight Saving Time in the US, so EST is EDT (the second most
> common in the list).  (That also says something about the frequency
> of Australia's EST, since it's their winter now.)

I didn't even check for EST.  Since I have a fairly long expire for some
groups (forever for a few), it won't tell you anything certain, though,
even if I did check.

Date: Wed, 8 Jun 2005 10:46:29 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>And may I also say that the text that Bruce has quoted from the earlier
>drafts was written *before* someone came on this list to point out the use
>of References in multi-part FAQs (a practice I had not been aware of, but
>which seemed eminently sensible).

It is such a common practice that I cannot imagine that anyone familiar 
enough with news operations to be participating in rewriting the news 
standards would not know about it. I certainly did. I don't recall anyone 
saying "what a great surprise" when this was pointed out. We did say "we 
ought not prohibit this", and that's why I proposed a few word change to 
the definition of followup to include existing practice but not exclude
the ability to detect followups altogether.

>The *whole* of our discussions, proposed changes, etc. since then have
>arisen as a direct result of what that someone (I cannot recall who it
>was) said.

Removing the MUST/MUST NOT section was not the result of any discussion.

Date: Wed, 8 Jun 2005 13:16:28 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: OT: "UT"
Sender: [email-masked]

"Charles Lindsey" <[email-masked]> wrote:

> But the clear prepondenance of GMT (21% of the total) is
> notable. But where is the notorious "EST"?

It's Daylight Saving Time in the US, so EST is EDT (the second most
common in the list).  (That also says something about the frequency
of Australia's EST, since it's their winter now.)

Seth

Date: Wed, 8 Jun 2005 16:37:35 +0100
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: OT: "UT"
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes

>But the clear prepondenance of GMT (21% of the total) is notable. But
>where is the notorious "EST"?

it's summer in New York, Boston &c... so it's EDT at the moment :)

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

To: [email-masked]
Xref: clerew local.usefor:21123
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Wed, 8 Jun 2005 12:18:06 GMT
Lines: 36
Sender: [email-masked]

In <0464704FF79B826F0500D9DE@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>--On 7. juni 2005 14:12 -0400 Bruce Lilly <[email-masked]> wrote:
>>
>> The bottom line is that the net result is what our drafts said
>> for quite some time and which enjoyed consensus, viz.
>>
>>    A followup MUST have a References-header, and an article that is not
>>    a followup MUST NOT have a References-header.

>remember the article I posted on May 27?

........

>Whatever the basis is for your claim of an earlier consensus, we don't seem 
>to have consensus for your position now.

And may I also say that the text that Bruce has quoted from the earlier
drafts was written *before* someone came on this list to point out the use
of References in multi-part FAQs (a practice I had not been aware of, but
which seemed eminently sensible).

The *whole* of our discussions, proposed changes, etc. since then have
arisen as a direct result of what that someone (I cannot recall who it
was) said.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21125
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: CFWS in References header
Date: Wed, 8 Jun 2005 13:52:02 GMT
Lines: 82
Sender: [email-masked]

In <D31C74C83D426255121F35C6@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>References is defined in 2822, while Xref and Supersedes are defined in 
>USEFOR.
>I'll open a ticket on whether comments in References should be listed as 
>SHOULD NOT generate or MUST NOT generate - saying MUST NOT generate would, 
>I think, make it equivalent with replacing CFWS with FWS.

I think you need to consider a more general issue.

There are various places where our draft mandates behaviour that is not in
accordance with current practice; in all such cases, it is necessary to
consider what transitional arrangements need to be put in place.

The issue here is where we are mandating that something MUST be accepted,
and where the transitional behaviour is of the form "but do not generate
it yet", for some value of "yet". The cases in this category are:

1. <comment>s in References.
2. Naked '.' in <phrase>s.
3. Folding of the Newsgroups header.

(There were also matters of <comment>s in some other headers, but these
are mostly gone now - there remain cases like 
   From: (rhubarb) Joe Doe (rhubarb) <[email-masked]> (Joe Doe)
where the first two <comment>s might confuse some existing newsreaders,
but I could perhaps live with that).

So what is an appropriate transitional arrangement for those three?

Our first reaction was to propose some cutoff date after which they could
be freely generated, but we were told that the IESG did not like Flag
Days.

OTOH, we did not like the idea of "do not generate until the next standard
comes out" (expressible as "MUST NOT generate") because #3, in particular,
really needed a more urgent fix than that (the Newsgroups header is the
only one of all the mail and news headers that cannot be folded, and
people are still astounded to be told that it still breaks existing
software - even INN has only fixed it in the last 12 months AFAIAA).

So the position we took was that people should start generating them when,
in their opinion, they would work well enough that the benefit would
outweigh the risk that articles would not be seen at some sites. That was
expressed by saying "SHOULD NOT generate", plus some verbiage such as:

   ....  Posting agents should be aware that such postings
   may be rejected by overly-critical old-style relaying agents. When a
   sufficient number of relaying agents are in conformance, posting
   agents SHOULD generate such whitespace in the form of <CRLF WSP> so
   as to keep the length of lines in the relevant headers (notably
   Newsgroups and Followup-To) to a reasonable length (such as 79
   characters, which is likely to be displayed satisfactorily by most
   current reading agents).  Before such critical mass occurs, injecting
   agents MAY reformat such headers by removing whitespace inserted by
   the posting agent, but relaying agents MUST NOT do so.

which is a bit wordy, but conveys the essential idea.

So we either say "MUST NOT generate" for them all, or
we say "SHOULD NOT generate" plus verbiage, or
we say that Newsgroups (for example) is urgent, and gets "SHOULD NOT",
whereas nobody is actually wanting to put <comment>s in References, so
"MUST NOT" will do there.

But it has also been pointed out (Harald will have seen an email from
Alexey on this) that even "SHOULD NOT" is too strong for Newsgroups
folding, since it gives no incentive to implement the "MUST accept".

So there is the gamut of possibilities. They need to be considered
together.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21127
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
Date: Wed, 8 Jun 2005 14:42:02 GMT
Lines: 123
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon June 6 2005 12:36, Charles Lindsey wrote:

>> >> You MUST NOT generate "EST" or any of the other things in <obs-zone>.
>> 
>> >Already covered by 2822.
>> 
>> Indeed, but worth reiterating because it would actually break the
>> protocols.

>Precisely how would "PDT" "break the protocols" in a way that "UT"
>would not?

I am not aware what other zones use "PDT", but it is wll known that "EST"
in Australia means something at least 6 hours different from what it means
in America, and I also understand that it means something different again
in Estonia, where it will likely be six hours in error the other way.

It is just not realistic to suppose that servers in Australia will
interpret "EST" according to RFC 2822 (they will probably just hand the
zone off to the OS, and for sure that will take an Australian view of
time zones).

The proper operation of the News protocols requires date-times to be
accurate. They have to be compared, and articles get
accepted/rejected/expired on the basis of those comparisons. If an article
fails to get stored because its date-time is misinterpreted, that is bad.
But if the error is the other way round and it gets stored when it should
have been rejected as stale, then that is far worse because it can result
in looping (example of this were discussed at the time we were introducing
the Injection-Date header some months back).

>> I could easily be persuaded to promote 
>> it to MUST NOT if someone asks for it. Are you formally proposing that?

>I have said about as clearly as possible that I feel that we should
>conform to RFC 2822 format as closely as possible.  Since 2822 clearly
>says MUST NOT generate for alphabetic zone indication, yes, that is
>what I have proposed.  Specifically, I propose no deviations from
>RFC 2822 regarding date-time format.

Then you have failed to answer the question I asked, which was in relation
to "UT" only.

"GMT" and "UT" are safe, in that _everybody_ understands them and
interprets them correctly. But if you follow RFC 2822 and say MUST accept
"EST" and the rest, then you cause actual harm (see above) and you force
some existing injecting agents which (quite correctly) refuse such zones
to be downgraded from a safe behaviour to an unsafe behaviour. We ought to
be moving in the opposite direction to that.

"GMT" is a special case, insofar as it is not ambiguous and is widely
used. So we MUST accept it for the time being.
> 
>> This WG decided. more years ago than I care to remember, that it was
>> ludicrous to REQUIRE all News software to be upgraded to accept
>> constructes that had Never Ever been seen in Netnews.

>You are mistaken in your implied assertion that handling of the standard
>alphabetic zones for parsing would would be "constructes [sic] that had
>Never [sic] Ever [sic] been seen in Netnews".

Again, you avoid the issue by changing the question. We decided that
obs-syntax in general was a Bad Thing, and it would be an imposition to
demand that news agents should accept it all regardless (a point you
yourself once made, as you will doubtless recall).

There may be a few exceptions to that, and they need to be considered on
their merits (but GMT and obs-phrase are the only ones that have been
excepted to date).

  RFC 850:

>   2.1.4  Date  The Date line (formerly   "Posted")   is  the
>   date,  in  a  format  that  must be acceptable both to the
>   ARPANET and to the getdate routine,

>RFC 1036:

>    The "Date" line (formerly "Posted") is the date that the message was
>    originally posted to the network.  Its format must be acceptable
>    both in RFC-822 and to the getdate(3) routine that is provided with
>    the Usenet software.

So what? There may well have been articles with strange zones in the past;
there are even a few even now. But nobody is going to reinject old
articles into the news transport system again (and I doubt any user agent
would refuse to display them anyway). It is the news transport where the
harm arises, and there is nothing gained by prolonging the agony.

>As Russ has noted, there is *substantial* existing usage of alphabetic
>zone abbreviations.

No, he has not noted that. Usage of GMT is indeed "substantial", but not
the others.

>> That decision have never been challenged since then. Why are you raising
>> it now?

>Again, this has been discussed before.

References please.

>> IIRC, it was you who asked for <obs-zone> to be brought within our
>> standard.

>References, please.

www.imc.org/ietf-usefor/2003/Feb/0221.html

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21128
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: OT: "UT"
Date: Wed, 8 Jun 2005 14:47:36 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>I was curious, so here are some actual numbers.

>% cd spool/overview
>% find -name \*.DAT -print > /tmp/files
>% xargs cut -f 4 < /tmp/files | sed 's/ (.*)$//' > /tmp/dates
>% wc -l /tmp/dates
>3632585 /tmp/dates
>% foreach zone ( GMT UT UTC EDT MDT CDT PDT )
>foreach? echo -n "${zone}: "
>foreach? grep $zone'$' /tmp/dates | wc -l
>foreach? end
>GMT:  780813
>UT:       13
>UTC:    3756
>EDT:   30721
>MDT:       1
>CDT:      21
>PDT:     216

>Which is more than I thought.  MDT, CDT, and PDT are a lot less than I
>thought.  Apparently someone does actually use UT, but maybe only one
>person.  :)

But the clear prepondenance of GMT (21% of the total) is notable. But
where is the notorious "EST"?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21124
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Info issues
Date: Wed, 8 Jun 2005 12:27:34 GMT
Lines: 43
Sender: [email-masked]

In <62AD13FE5CBD2F4E22C5DB92@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>--On 6. juni 2005 16:05 -0700 Russ Allbery <[email-masked]> wrote:

>> Authentication identity is
>> very internal to the application and not generally of use or interest to
>> anyone outside that application except for audit purposes.
>>
>>> So I propose a new Injection-Info parameter
>>>     authentication=<value>
>>
>>> Would people be happy with that?
>>
>> Works for me.

>I think Russ made a very good argument that revealing the authentication 
>identity in the Injection-Info is a mistake.

And yet, in the bit you have quoted, he appears to support the inclusion
of that parameter.

The Injection-Info provides a variety of methods of indicating the article
origin, from which injecting agents can choose what suits them.

When this header was first introduced, there were long arguments between
those who wanted to protect the poster's privacy and anonymity, and those
who wanted to prevent malefactors hiding behind their privacy walls.

The upshot was the variety of tools that you see, with the choice being
left open (because standards don't deal with privacy issues). The
discussion of the tensions between the various viewpoints was therefore
moved to USEAGE, where you can still read it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21122
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Wed, 8 Jun 2005 12:11:53 GMT
Lines: 27
Sender: [email-masked]

In <5E8CC1FF1C846ABF3D954EC0@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>--On 7. juni 2005 12:50 +0000 Charles Lindsey <[email-masked]> wrote:

>> What you cannot do is to say nothing at all.

>Actually RFC 2822 gets away with only giving a construction algorithm.

Yes, but Alexey pointed out a couple of weeks back that RFC 2822 is both a
protocol and a format document, with the implication that that
construction algorithm belonged in USEPRO (where it indeed is).

Which leaves the question whether it is sufficient, in USEFOR, to point
you at that text in USEPRO, or whether you also need some explicit
semantics text in USEFOR, most likely worded in terms of parents coming earlier
than children (though maybe not using the words 'parent' and 'child').

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21126
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Wed, 8 Jun 2005 14:00:20 GMT
Lines: 32
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>For the purpose of the USEFOR (syntax & semantics) document, it suffices to
>specify the syntax and semantics (obviously?).  For the purpose of the
>USEPRO (protocol) document, one could add an implementation note such as:

>  When modifying a References field, care must be taken to ensure that the
>  result complies with the syntax specification for the field, specifically
>  that line length limits are observed and that no msg-id is truncated or
>  subjected to internal line folding.

I agree that any mention of pruning is a matter for USEPRO rather than
USEFOR. What I currently propose to say, at Frank's request (see suggested
text posted earlier) is that you MAY prune (subject to keeping the first
and the last two), and that you MUST prune to keep it within 998
(including folded lines, because of the kind of breakage I described).

I think it goes without saying that whatever results must be syntactically
correct, and so there can be no folding within a msg-id.

There is some additional advice on pruning in USEAGE.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21121
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Wed, 8 Jun 2005 12:05:12 GMT
Lines: 31
Sender: [email-masked]

In <6024687CE64910DA7D2748FF@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>I have rarely seen an example of a linear thread.....

I think the use of the metaphor "thread" came about in the following way.

Consider a tree (or a list of a DAG or a graph) with a bead at each node.

Take a piece of cotton thread, and thread it through some/all of those
beads in some manner. That defines a linear ordering on the nodes in the
tree (DAG/graph/whatever).

A useful threading order for a tree of news articles is a depth first
traversal, and newsreaders that present articles in that order (and
sometimes with some indentation to make it even prettier) are hence known
as "threading newsreaders".

But the question immediately facing us is whether or not it is useful to
use "thread" informally in some text that serves to introduce the
References header in USEFOR.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 8 Jun 2005 10:27:23 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1028 USEFOR 3.1.2: What zones should be on the MUST accept list?
Sender: [email-masked]

On Wed, 8 Jun 2005, Harald Tveit Alvestrand wrote:
> These numbers could argue for skipping UT and/or including EDT (and 
> presumabily EST, since this measurement was taken in summer).

"UT" was my addition in son-of-1036, and since it seems to have achieved
essentially zero market penetration :-), I think we can forget about it. 

We should *NOT* bless EST or EDT, still in significant use though they may
be, because they are definitely ambiguous.  (I think it most unlikely that
software which puts in a local timezone name is first checking to make
sure that the definition agrees with the RFC 2822 obs-zone definition.)

                                                          Henry Spencer
                                                       [email-masked]

Date: Wed, 08 Jun 2005 13:32:05 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1029: USEFOR 3.2.2 References: Should comments be a MUST NOT? 
Sender: [email-masked]

New ticket:

 in
 will
 we

Date: Wed, 08 Jun 2005 13:24:11 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1028 USEFOR 3.1.2: What zones should be on the MUST accept list?
Sender: [email-masked]

I've created the following ticket:

USEFOR section 3.1.2 "Date" currently says:

3.1.2  Date

   The Date header is the same as that specified in Sections 3.3 and
   3.6.1 of [RFC2822] with the added restrictions detailed in
   Section 2.2.  However, the use of "UT" and "GMT" as time zones
(part
   of <obs-zone>), although deprecated, is widespread in news articles
   today.  Therefore, agents MUST accept <date-time> constructs which
   include the updated <zone> construct below.

   orig-date       =  "Date:" SP date-time CRLF

   zone            =  (( "+" / "-" ) 4DIGIT) / "UT" / "GMT"

   Note that agents SHOULD NOT generate <date-time> constructs which
   include either "UT" or "GMT" and MUST NOT generate <date-time>
   constructs which include any other zone names defined by
<obs-zone>,
   some of which have ambiguous interpretations and would have adverse
   effects on the Netnews protocols.

   Also note that these requirements apply wherever <date-time> is used,
   including Injection-Date and Expires in Section 3.2.1 and
   Section 3.2.4 respectively.

The question has been raised about how realistic this is.
Russ Allbery computed the following numbers:

I was curious, so here are some actual numbers.

% cd spool/overview
% find -name \*.DAT -print > /tmp/files
% xargs cut -f 4 < /tmp/files | sed 's/ (.*)$//' > /tmp/dates
% wc -l /tmp/dates
3632585 /tmp/dates
% foreach zone ( GMT UT UTC EDT MDT CDT PDT )
foreach? echo -n "${zone}: "
foreach? grep $zone'$' /tmp/dates | wc -l
foreach? end
GMT:  780813
UT:       13
UTC:    3756
EDT:   30721
MDT:       1
CDT:      21
PDT:     216

Which is more than I thought.  MDT, CDT, and PDT are a lot less than I
thought.  Apparently someone does actually use UT, but maybe only one
person.  :)

These numbers could argue for skipping UT and/or including EDT (and 
presumabily EST, since this measurement was taken in summer).

(note - not in ticket - last sentence was added by me, not Russ)

Date: Wed, 08 Jun 2005 13:13:20 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1022 Summary: Should dot be permitted in phrases?
Sender: [email-masked]

Of the flurry of messages on this subject line, I found exactly two 
responses that pertained to the subject of the original ticket:

- Charles would like to somehow elevate obs-phrase (which allows dot) from 
the "MAY accept/MUST NOT generate" implied by the current text to a status 
that says "MUST accept", and either SHOULD NOT or MUST NOT generate.

- Bruce Lilly argues against a deviation (I think).

Are these the only two voices on the subject?

I'm opening another ticket on the issue of UT in timezones.

                          Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Injection-Info issues
Date:  Wed, 08 Jun 2005 09:13:04 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Charles Lindsey wrote:
 
> I need to wait for your "continued later"

For the latest and greatest SPF draft published yesterday see
http://www.ietf.org/internet-drafts/draft-schlitt-spf-classic-02.txt

The weird header is in chapter 7, grmbl, still with <... 2822>
for CRLF.  

Wait a moment, usefor-04 also says that 2822 defines CRLF, now
what is this, easter-eggs for Bruce everywhere ?  The ABNF for
CRLF is in 2234bis, not in 2822.

> "client-ip" presumably expects an <IPv4address> or
> <IPv6address>

All values are either dot-atom or quoted-string, the semantical
details are explained in prose.
                               Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Injection-Info issues
Date:  Wed, 08 Jun 2005 08:29:50 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 39
Sender: [email-masked]

Charles Lindsey wrote:
 
>> Add authentication= with a reference to the new NNTPauth,
>> Russ said that this is a good idea.  Alexey would warn us
>> if NNTPauth is blocked.
 
> OK, that makes 3 people happy with that idea.

"Happy" isn't the correct word, it's more like "I really don't
see how Sam could kill us for it, and I hope that Alexey would
warn us in time".

 [x-attribute]
> the folklore is certainly that they are available for use,
> and would never be standardized/registered as such.
[...]
> we write whatever we think fit. In which case, I most
> certainly think that we should allow x-attributes for
> injecting agents that wish to include further information
> that does not fit into any of the categories provided

Okay, where's the ABNF, key-value-pair as in SPF, or a 2231
variant of it ?

  [security considerations about 2231]
>| MIME security considerations are discussed in [RFC2046].
>| Note that applying some [RFC2231] extensions for parameters
>| like multi-line paramters on a boundary parameter as defined
>| in [RFC2046] might be abused to bypass simple algorithms
>| trying to analyze MIME parts.
 
> Posibly. Does anyone else want to see that written into our
> Security Considerations?

That was meant as a request, not only as a proposal.  But the
wording is of course open for debate.

                          Bye, Frank

Date: Wed, 08 Jun 2005 08:06:51 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

--On 7. juni 2005 13:03 +0000 Charles Lindsey <[email-masked]> wrote:

> But I am not aware of anyone (trolls apart) who has ever tried to
> construct, for example, a References header with a loop in it.

I have been told by someone familiar with News (I think it was by Arnt 
Gulbrandsen, the implementor of the Leafnode newsserver, but it's so many 
years ago that it could easily be someone else) that he's seen at least one 
example of a References: loop.

I believe this was a case of two messages having References referring to 
each other. Had "interesting" effects on some news reading software he was 
writing that expected the References chain to be a DAG.

                       Harald

Date: Wed, 08 Jun 2005 07:53:41 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

--On 7. juni 2005 12:50 +0000 Charles Lindsey <[email-masked]> wrote:

>
> In <[email-masked]> Frank Ellermann
> <[email-masked]> writes:
>
>> It would surely kill USEFOR for good, yes.  No "threads", no
>> nonsense, no aclycic graphs.  A msg-id-list is simply a list
>> of msg-id-s, ready.  Harald's wording (or your version of it)
>> was fine, any additional comma in 3.2.2 is asking for trouble.
>
> No, the <msg-id-list> in the References header is not any random
> collection of <msg-id>s. It is subject to constraints (e.g. that a
> precursor does not come later in the list that any followup to it).
>
> You can express this by referring to the rule for constructing it in
> USEPOR (which is what my currently proposed text does), or you can express
> it by writing some similar rules into USEFOR (which is what Forrest has
> suggested), or you can express it by talking about parents coming earlier
> than children (which is what my earlier suggestion did, though not using
> the words 'parent' and 'child').
>
> What you cannot do is to say nothing at all.

Actually RFC 2822 gets away with only giving a construction algorithm.

Date: Wed, 08 Jun 2005 07:59:06 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked], "Forrest J. Cavalier III" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

--On 7. juni 2005 14:12 -0400 Bruce Lilly <[email-masked]> wrote:
>
> The bottom line is that the net result is what our drafts said
> for quite some time and which enjoyed consensus, viz.
>
>    A followup MUST have a References-header, and an article that is not
>    a followup MUST NOT have a References-header.

Bruce,

remember the article I posted on May 27?

> 1) All "followups" have a References: header.
>    All articles with References: header are "followups".
> - Eivind Tagseth: "Because I view... multi-part..special type of followup"
>
> 2) All "followups" have a References: header.
>    There exist articles that have References:, but are not "followups"
> - Henry Spencer: "Existing practice"
> - Richard Clayton: "captures my perception of useful current practice"
> - Seth Breidbarth: "current reality"
> - Russ Albery: "but would be content with 1)"
> - Frank Ellermann

(I also had 3 people arguing other positions).

Whatever the basis is for your claim of an earlier consensus, we don't seem 
to have consensus for your position now.

                     Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
Date:  Wed, 08 Jun 2005 08:05:56 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 50
Sender: [email-masked]

Russ Allbery wrote:

> people use some of this nonsense in real articles

For sure not MEZT, the worst would be MESZ <shudder />

> Usenet software has interpreted those time zones as
> having those meanings for something like fifteen
> years now, at least, so people expect it.

I can understand the 822 and 1036 timezones, and I'm
not at all surprised that you got a lot of EDT in
your sample, IIRC EST / EDT used to be the default.

The other (undocumented) names are as you said crap.

> Here's the question:  How do you plan on getting
> rid of this sort of thing on Usenet?

For the 822 / 1036 stuff I can live with whatever
you, Bruce, Henry, and Charles say.  And if you say
"delete the 2822 obs-stuff" I'd be happy with it.

> I can tell you as a Usenet software author that
> whenever I try to tighten date parsing, I get
> pushback from my users who want everything that
> currently works to just keep working.

And here I was thinking that I'm the most conservative
"backwards compatibility" extremist of the world using
an ersatz-newsreader built 1996.

> I don't have a good answer for that question other
> than "standards purity,"

Tell them where the source is and where they'd find a
compiler.  Really, MEZT _is_ plain nonsense.  And MEZ
is CET, nobody uses the German abreviations outside of
Germany.  Not in the Internet...  okay, in Usenet you
get all the worst cases, and "deusenet" is Usenet as
it was 1995, but it's still state of the art s-o-1036:

| time     = hh ":" mm [ ":" ss ] space timezone
| timezone = "UT" / "GMT"
|          / ( "+" / "-" ) hh mm [ space "(" zone-name ")" ]

See, no MEZ, MESZ, "MEZT", the worst case is a comment.

                    Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date:  Wed, 08 Jun 2005 07:32:20 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 13
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> I have rarely seen an example of a linear thread.....

Some [FWS] and msg-id threads here were rather linear.
In the latter case there was still a sort of result -
or maybe two almost equivalent results.

The threads about references were in fact loops, they
started from scratch periodically.  No surprising new
arguments for years.
                     Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date:  Wed, 08 Jun 2005 06:49:19 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

Forrest J. Cavalier III wrote:
 
> Frank, it seems like you want to push USEFOR to be basically
> ABNF only, with almost no semantics.

For a "msg-id-list" the semantics is "list of msg-ids", for
a more elaborated answer about Charles' DAG idea (which also
does not fly) see my parallel article.
 
> Why would such a terse document merit its own separate RFC?

There are some interesting and important differences from RfC
2822, one header field is irrelevant in news (In-Reply-To),
many header fields are only relevant in news, not mentioned
in 2822, above all "we" have the one and only msg-id syntax.

In two slightly different variants, but syntactically the same
msg-id, _very_ different from 2822.  There are at the moment
at least three drafts building on "our" fixed msg-id:

The 26th NNTP monster in last call, Charles' news-nntp-uri-00,
my news-nntp-uri-00.  They all say "NO-WS-CTL is not correct,
RfC 2822 got it wrong".
                           Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date:  Wed, 08 Jun 2005 06:26:24 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 87
Sender: [email-masked]

Charles Lindsey wrote:

  [DAG]
>> Is that a "directed acyclic graph" ?

> Yes.

Then it's too general, if you're lucky you can deduce a rooted
tree, and then the References are the way from the root to the
article.

In a DAG the direction "from root to leave" isn't clear, it's
just a tree with arbitrary directions attached to the edges...

Wait, I see, you want multiple trees, all the msg-ids are by
definition direct or indirect "precursors" of the article in
question, and for two msg-ids m and in this list with m < n
you have either m is an (in)direct "precursor" of n, or it is
unrelated.  In any case all listed msg-ids are "precursors" of
the article in question.

Defining "precursor" as "existed before (pray)".  Fine, that
makes sense, but we are in trouble with the "pray".  And it's
not guaranteed, if I pick msg-ids (x, y, z) as my "references",
and x was a reply or followup of z, then it's not your DAG.

And while I do such stupid things no text in the world forces
me to sort these msg-ids as ((y), (z, x)) or ((z, x), y) or
(z, y, x) only because you like to see "z before x".  There's
a serious chance that I don't know this even if I'm willing to
follow your decree.  Maybe I found x, y, z individually.

>> Whatever it is, this is not always and not necessarily the
>> case.

> Yes it is.

No, it's not.  It's a msg-id-list.  Only if you are lucky it's
a way from "the" root to the article in question.  Often you
don't know what it is (= references from an invisible context).

> If you are not trying to describe at least a DAG (of which
> trees and linear lists are a particular case), then you have
> no business putting a References header in your article.
> That is what Forrest is worried about.

My UA doesn't care what Forrester is worried about, because his
definition is not correct.  My UA _normally_ grabs whatever it
sees in the References, a msg-id-list, adds the msg-id of the
parent article, and uses this as references in the followup or
reply.

It has _no_ idea what the old references were.  And that's only
the normal case.  Under less normal conditions I could put some
"related Message-IDs" manually in the references, for "retrieval"
as you said.  It's strictly my business.

And I'd bet that no UA in the world has a crystal ball to check
these references against theoretical trees, they just do what
my UA does:  grab it, add parent Message-ID, ready.  Exactly
this is the correct behaviour.  If you want to say more about
it put it in 1855bis / USEAGE.

> my current USEPRO text is only telling you how to construct a
> tree

It doesn't, it's only a theoretical tree by induction, if the
_complete_ references were constructed this way in each step.
That is not guaranteed, and it's generally neither possible
nor desirable to check it.

What you have in USEPRO is the one correct step to create a
followup given a parent msg-id and its old msg-id-list.  This
old msg-id-list can be anything, it can be some "See-Also".

> I am not aware of anyone (trolls apart) who has ever tried
> to construct, for example, a References header with a loop
> in it.

I tried this here, References containing my own Message-ID:

<http://article.gmane.org/gmane.ietf.usenet.format/27603/raw>
<news://news.gmane.org/[email-masked]>

It's not that I'd "recommend" this practice, it's a bad idea,
but it's still something UAs have to deal with.  Bye, Frank

To: [email-masked]
Subject: Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 07 Jun 2005 20:20:37 -0700
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> That's the ambiguous EST beast.  Better delete this crap, it's
> far beyond ridiculous, it's plain wrong.

Except, see, it's not, because people use some of this nonsense in real
articles and the time zone they mean is, surprisingly enough, those.
Furthermore, Usenet software has interpreted those time zones as having
those meanings for something like fifteen years now, at least, so people
expect it.

Here's the question:  How do you plan on getting rid of this sort of thing
on Usenet?  I can tell you as a Usenet software author that whenever I try
to tighten date parsing, I get pushback from my users who want everything
that currently works to just keep working.  If I require that all times be
given properly zero-padded, for instance, it breaks some random Mac
off-line reader.  I then get asked "why do you have to do that, what
really breaks?"  I don't have a good answer for that question other than
"standards purity," which personally I'm down with, but which isn't a very
persuasive argument.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
Date:  Wed, 08 Jun 2005 05:08:41 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 26
Sender: [email-masked]

Russ Allbery wrote:

> well, all this crap:

> { "CUT",    0 },                 /* Coordinated Universal */

Utter dubious, is that the 822 UT ?

> { "WET",    0 },                 /* Western European */

Utter dubious, where's GMT ?

> { "NDT",  (-2 * 60 + 30) * 60 }, /* Newfoundland Daylight */

Crap, skipping more crap...

> { "MEZ",    1 * 60 * 60 },       /* Middle European */
> { "MEZT",   2 * 60 * 60 },       /* Middle European Summer */

Completely bogus, MESZ is "MittelEurpaeische SommerZeit"
 
> { "EAST",  10 * 60 * 60 },  /* Eastern Australian Standard */

That's the ambiguous EST beast.  Better delete this crap, it's
far beyond ridiculous, it's plain wrong.  What a headache, bye.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Russ Allbery <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Tue, 7 Jun 2005 15:11:18 -0400
Cc: [email-masked]
Sender: [email-masked]

On Tue June 7 2005 14:23, Russ Allbery wrote:
> 
> Bruce Lilly <[email-masked]> writes:

> > We have been directed to base our work on RFC 2822 and MIME, and MIME
> > (specifically RFC 2046 section 5.2.2.2) permits use of a References
> > field to reference parts of a fragmented message. Such a message with a
> > References field is a "followup" per the consensus text quoted above.
> 
> I'm not sure what you're trying to demonstrate by inserting the word
> "consensus" into this sentence.  I would instead argue that RFC 1036 did
> not anticipate uses such as MIME when that text was drafted.

Apologies if it was unclear; let me clarify.  The "consensus text" referred
to is what appeared in draft-ietf-usefor-article versions 06 through 13 at
least (possibly others), viz.:

� �A followup MUST have a References-header, and an article that is not
� �a followup MUST NOT have a References-header.

That text was subject to WG review including internal last calls, and
including after a WG Chair clarified that our work was to be based on
2822 + MIME.  Clearly that text survived at least eight iterations of
the draft, including last calls, and while we were considering MIME.

It also happens to be the same as in Henry's news.txt document modulo
capitalization of "NOT", and is a reasonable interpretation of 1036 (which
does have issues with clarity).  I am not claiming that the consensus
applied to those other documents or to text other than that quoted above.
The text does not appear verbatim as a unit in 1036; that is not the claim.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Russ Allbery <[email-masked]>
Subject: Re: CFWS in References header
Date: Tue, 7 Jun 2005 14:35:53 -0400
Cc: [email-masked]
Sender: [email-masked]

On Tue June 7 2005 13:50, Russ Allbery wrote:

> References is a borderline case.  Adding CFWS to References *will* break
> existing news software, in the sense that the comments will be interpreted
> as message IDs, but the result of the breakage isn't particularly
> dramatic.  The open question is whether that degree of breakage warrants
> divergence or not.  I don't think we ever really reached a conclusion
> everyone was satisfied with, although we've gone around on the topic
> several times.

A good example of where Mark Crispin's suggestion of an implementation
note would be appropriate, e.g.:

  Although comments have always been permitted in the References field
  per RFC 822 syntax as adopted by RFCs 850 and 1036, some implementations
  are known to attempt to interpret comments as msg-ids.  Consequently,
  implementations concerned about such matters should avoid comments
  when generating References fields. Implementers concerned about
  resources used for handling msg-ids should ensure that comments are
  not misinterpreted as msg-ids. 

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Tue, 7 Jun 2005 20:23:03 +0200
Organization: Chaos rules.
Sender: [email-masked]

Charles Lindsey schreef:

> But I am not aware of anyone (trolls apart) who has ever tried to
> construct, for example, a References header with a loop in it.

A have seen the shortest variant a few times: self-referencing.

The last time even because somebody considered it useful.
See news:[email-masked]

http://groups.google.co.uk/groups?selm=[email-masked]&fwc=1
http://groups.google.co.uk/groups?selm=[email-masked]&fwc=1
http://groups.google.co.uk/groups?selm=[email-masked]&fwc=1

The URL with 465 will not return a message, I guess that google filters
such messages.

-- 
Grtz, Ruud

(Bcc: that somebody)

To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 07 Jun 2005 11:23:40 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:

> That text provided a convenient shorthand ("followup" == article with a
> References field) and is an objective fact about a given article which
> can be determined directly from that article without the need to examine
> an arbitrarily large collection of other articles, without having to
> break out a Ouija board to divine whether or not the "subject" changed
> or what the authors' or poster's intentions were, etc.

> Until it was arbitrarily changed w/o prior WG discussion or consensus.

Whether there was consensus before it was changed or not, we just went
through this and there's a fair bit of consensus to leave it the way that
it is now.  That's simply not how the word "followup" is used in software
and in discussion of Usenet, for better or for worse.

> We have been directed to base our work on RFC 2822 and MIME, and MIME
> (specifically RFC 2046 section 5.2.2.2) permits use of a References
> field to reference parts of a fragmented message. Such a message with a
> References field is a "followup" per the consensus text quoted above.

I'm not sure what you're trying to demonstrate by inserting the word
"consensus" into this sentence.  I would instead argue that RFC 1036 did
not anticipate uses such as MIME when that text was drafted.

> RFC 2822 and its predecessors defines use of the References field for
> "message identifiers of other messages" without restriction, and that
> applies to multi-part FAQs.

I agree with you on this, as previously mentioned.

> I move that we restore the text quoted above which was present in our
> drafts prior to the arbitrary change, and which enjoyed WG consensus and
> was subjected to internal last calls.  Anybody care to second the
> motion?

I think this is a really bad argument to use to support the change.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Tue, 7 Jun 2005 14:19:08 -0400
Cc: [email-masked]
Sender: [email-masked]

On Tue June 7 2005 09:47, Charles Lindsey wrote:

> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >The current (RFC 1036) definition of "followup" includes current --
> >indeed long-standing -- practices such as multi-part FAQs and other
> >multi-part articles.
> 
> No, the consensus view here is that RFC 1036 does NOT include multi-part
> FAQS etc in that manner, which is why the issue needs to be clarified.

It was eminently clear in earlier drafts which passed internal last call;
see separate message.

> >o "excessively long lines" is a non-issue due to the ability to fold
> >  physical lines wherever linear whitespace is allowed, as has been the
> >  case with RFC 822 message format and its predecessors even before that
> >  format was adopted by RFC 850.
> >o there is of course no limit to the length of a logical (i.e. logically
> >  unfolded) field.  Never has been.
> 
> That's the theory, but it ain't the practice.
> 
> There was a long running thread on the ietf-822 list, which you may well
> remember. The References header got longer and longer. Different agents
> folded it in different ways (some even took a whole new line for each
> msg-id, which was actually quite nice if you wanted to look through it for
> some particular message). Moreover, some agents when replying took the old
> References list, added the new item, and refolded the whole thing in a
> completely different way. All quite permitted by RFC 2822, of course.
> 
> BUT there was at least one agent (I never discovered which) which unfolded
> the whole line, added the new entry, and then did not refold, thus undoing
> all the good done by the preceding agents. Naturally, the total length
> eventually got beyoind 998. At which point some agent (not sure whether it
> was the same culprit or the next one in line) truncated the header after
> exactly 998 characters (usually in the middle of a <msg-id>) and passed it
> on, to the great confusion of the next agent to process it.

For the purpose of the USEFOR (syntax & semantics) document, it suffices to
specify the syntax and semantics (obviously?).  For the purpose of the
USEPRO (protocol) document, one could add an implementation note such as:

  When modifying a References field, care must be taken to ensure that the
  result complies with the syntax specification for the field, specifically
  that line length limits are observed and that no msg-id is truncated or
  subjected to internal line folding.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Forrest J. Cavalier III" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Tue, 7 Jun 2005 14:12:42 -0400
Cc: [email-masked]
Sender: [email-masked]

On Tue June 7 2005 12:59, Forrest J. Cavalier III wrote:

> Here are links to all the RFC1036 sections which use the term:

Thanks.

The bottom line is that the net result is what our drafts said
for quite some time and which enjoyed consensus, viz.

   A followup MUST have a References-header, and an article that is not
   a followup MUST NOT have a References-header.

Modulo capitalization of "NOT", that text also appeared in Henry's
news.txt.Z.

That text provided a convenient shorthand ("followup" == article with a
References field) and is an objective fact about a given article which
can be determined directly from that article without the need to examine
an arbitrarily large collection of other articles, without having to 
break out a Ouija board to divine whether or not the "subject" changed or
what the authors' or poster's intentions were, etc.

Until it was arbitrarily changed w/o prior WG discussion or consensus.

> I find no wording above which definitely excludes or includes
> multi-parts.

We have been directed to base our work on RFC 2822 and MIME, and
MIME (specifically RFC 2046 section 5.2.2.2) permits use of a
References field to reference parts of a fragmented message. Such
a message with a References field is a "followup" per the consensus
text quoted above.  RFC 2822 and its predecessors defines use of
the References field for "message identifiers of other messages"
without restriction, and that applies to multi-part FAQs.

I move that we restore the text quoted above which was present in
our drafts prior to the arbitrary change, and which enjoyed WG
consensus and was subjected to internal last calls.  Anybody care
to second the motion?

To: [email-masked]
Subject: Re: CFWS in References header
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 07 Jun 2005 10:50:36 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> I was told earlier that the WG had a previous consensus that it would:
> a) not change definitions from RFC 2822 (where References has CFWS)
> b) allow CFWS wherever possible, with a "MUST accept/SHOULD NOT generate"
> clause for certain headers (including References).

I don't think so.

The consensus, as I understood it, was that we would avoid unnecessary
divergences from RFC 2822 while recognizing that some changes to converge
with RFC 2822 would break existing news software and therefore couldn't be
allowed.  In particular, we HAVE to diverge on Message-ID because the RFC
2822 definition is too lax and will break NNTP.

I would not have agreed with the consensus as stated above, for the
Message-ID issue if for no other reason.

References is a borderline case.  Adding CFWS to References *will* break
existing news software, in the sense that the comments will be interpreted
as message IDs, but the result of the breakage isn't particularly
dramatic.  The open question is whether that degree of breakage warrants
divergence or not.  I don't think we ever really reached a conclusion
everyone was satisfied with, although we've gone around on the topic
several times.

To: [email-masked]
Subject: Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 07 Jun 2005 10:45:10 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> However, in the Date header, EST has a protocol-defined meaning.

> Only in an obs- part of RFC 2822 which we have chosen to exclude

Our draft isn't actually published.  Given the best available *published*
protocol information that people currently have to work with, EST has a
protocol-defined meaning and UTC doesn't.

INN's date parser parses the full obs RFC 2822 syntax, plus some other
random junk, primarily missing leading zeros in time components and dates
without any time zone at all, but also including UTC and, well, all this
crap:

    { "CUT",    0 },                 /* Coordinated Universal */
    { "WET",    0 },                 /* Western European */
    { "BST",    1 * 60 * 60 },       /* British Summer */
    { "NDT",  (-2 * 60 + 30) * 60 }, /* Newfoundland Daylight */
    { "NST",  (-3 * 60 + 30) * 60 }, /* Newfoundland Standard */
    { "ADT",   -3 * 60 * 60 },       /* Atlantic Daylight */
    { "AST",   -4 * 60 * 60 },       /* Atlantic Standard */
    { "YDT",   -8 * 60 * 60 },       /* Yukon Daylight */
    { "YST",   -9 * 60 * 60 },       /* Yukon Standard */
    { "AKDT",  -8 * 60 * 60 },       /* Alaska Daylight */
    { "AKST",  -9 * 60 * 60 },       /* Alaska Standard */
    { "HADT",  -9 * 60 * 60 },       /* Hawaii-Aleutian Daylight */
    { "HAST", -10 * 60 * 60 },       /* Hawaii-Aleutian Standard */
    { "HST",  -10 * 60 * 60 },       /* Hawaii Standard */
    { "CES",    2 * 60 * 60 },       /* Central European Summer */
    { "CEST",   2 * 60 * 60 },       /* Central European Summer */
    { "MEZ",    1 * 60 * 60 },       /* Middle European */
    { "MEZT",   2 * 60 * 60 },       /* Middle European Summer */
    { "CET",    1 * 60 * 60 },       /* Central European */
    { "MET",    1 * 60 * 60 },       /* Middle European */
    { "EET",    2 * 60 * 60 },       /* Eastern European */
    { "MSK",    3 * 60 * 60 },       /* Moscow Winter */
    { "MSD",    4 * 60 * 60 },       /* Moscow Summer */
    { "WAST",   8 * 60 * 60 },       /* Western Australian Standard */
    { "WADT",   9 * 60 * 60 },       /* Western Australian Daylight */
    { "HKT",    8 * 60 * 60 },       /* Hong Kong */
    { "CCT",    8 * 60 * 60 },       /* China Coast */
    { "JST",    9 * 60 * 60 },       /* Japan Standard */
    { "KST",    9 * 60 * 60 },       /* Korean Standard */
    { "KDT",    9 * 60 * 60 },       /* Korean Daylight (no change?) */
    { "CAST",  (9 * 60 + 30) * 60 }, /* Central Australian Standard */
    { "CADT", (10 * 60 + 30) * 60 }, /* Central Australian Daylight */
    { "EAST",  10 * 60 * 60 },       /* Eastern Australian Standard */
    { "EADT",  11 * 60 * 60 },       /* Eastern Australian Daylight */
    { "NZST",  12 * 60 * 60 },       /* New Zealand Standard */
    { "NZST",  13 * 60 * 60 },       /* New Zealand Daylight */

I tried to get rid of some of it, but I got pushback from people who
didn't want existing dates to suddenly stop parsing, and finally gave up
and just tossed the whole time zone list from parsedate in when I rewrote
the date parser.

As I recall, something in the range of 10% of that list was actually in
use when I checked.

To: [email-masked]
Subject: Re: OT: "UT"
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 07 Jun 2005 10:39:35 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:
> On Mon June 6 2005 19:08, Russ Allbery wrote:

>> I don't recall any usage of UT at all.

> Well, there's substantial use of "UTC", though that is not and never has
> been a valid standard zone abbreviation in date-time in the message
> format.  It's a violation of RFCs 561, 724, 733, 822, 850, 1036, and
> 2822.  Fortunately, nobody has proposed accepting it as if it were
> valid.

I was curious, so here are some actual numbers.

% cd spool/overview
% find -name \*.DAT -print > /tmp/files
% xargs cut -f 4 < /tmp/files | sed 's/ (.*)$//' > /tmp/dates
% wc -l /tmp/dates
3632585 /tmp/dates
% foreach zone ( GMT UT UTC EDT MDT CDT PDT )
foreach? echo -n "${zone}: "
foreach? grep $zone'$' /tmp/dates | wc -l
foreach? end
GMT:  780813
UT:       13
UTC:    3756
EDT:   30721
MDT:       1
CDT:      21
PDT:     216

Which is more than I thought.  MDT, CDT, and PDT are a lot less than I
thought.  Apparently someone does actually use UT, but maybe only one
person.  :)

To: [email-masked]
Subject: Re: Injection-Info issues
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 07 Jun 2005 10:13:53 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> I think Russ made a very good argument that revealing the authentication
> identity in the Injection-Info is a mistake.  Name it
> identifier-that-the-sysadmin-can-use-to-figure-out-who-sent-this, and it
> may make sense.

I think it's important to distinguish between world-wide public newsgroups
and private newsgroups for a particular organization.  It may make quite a
bit more sense to include the authentication identity in the latter.

On the other hand, don't we already have an opaque token explicitly for
the use of the local sysadmin to encode trace information?  That might be
sufficient.

Date: Tue, 07 Jun 2005 12:59:37 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

Charles Lindsey wrote:

> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> 
>>The current (RFC 1036) definition of "followup" includes current --
>>indeed long-standing -- practices such as multi-part FAQs and other
>>multi-part articles.
> 
> 
> No, the consensus view here is that RFC 1036 does NOT include multi-part
> FAQS etc in that manner, which is why the issue needs to be clarified.

RFC1036 does not define "follow-up".  Any "definition of followup" from
RFC1036 is only found "between the lines".  I think Bruce and Charles
are wrong to refer to it as a definition.

Here are links to all the RFC1036 sections which use the term:

   http://www.mibsoftware.com/userkt/usefor/rfc1036/References.htm

   http://www.mibsoftware.com/userkt/usefor/rfc1036/Newsgroups.htm

   http://www.mibsoftware.com/userkt/usefor/rfc1036/Subject.htm

   http://www.mibsoftware.com/userkt/usefor/rfc1036/Summary.htm

I find no wording above which definitely excludes or includes
multi-parts.  So appeals to RFC1036 as an authority on this are
rhetoric, not reason.

To: [email-masked]
Xref: clerew local.usefor:21093
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Tue, 7 Jun 2005 13:47:23 GMT
Lines: 60
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>The current (RFC 1036) definition of "followup" includes current --
>indeed long-standing -- practices such as multi-part FAQs and other
>multi-part articles.

No, the consensus view here is that RFC 1036 does NOT include multi-part
FAQS etc in that manner, which is why the issue needs to be clarified.

>>     To prevent excessively long lines, message 
>>     identifiers from the parent's References field can be omitted, provided
>>     that the first and last one or more in that field are retained.

>o "excessively long lines" is a non-issue due to the ability to fold
>  physical lines wherever linear whitespace is allowed, as has been the
>  case with RFC 822 message format and its predecessors even before that
>  format was adopted by RFC 850.
>o there is of course no limit to the length of a logical (i.e. logically
>  unfolded) field.  Never has been.

That's the theory, but it ain't the practice.

There was a long running thread on the ietf-822 list, which you may well
remember. The References header got longer and longer. Different agents
folded it in different ways (some even took a whole new line for each
msg-id, which was actually quite nice if you wanted to look through it for
some particular message). Moreover, some agents when replying took the old
References list, added the new item, and refolded the whole thing in a
completely different way. All quite permitted by RFC 2822, of course.

BUT there was at least one agent (I never discovered which) which unfolded
the whole line, added the new entry, and then did not refold, thus undoing
all the good done by the preceding agents. Naturally, the total length
eventually got beyoind 998. At which point some agent (not sure whether it
was the same culprit or the next one in line) truncated the header after
exactly 998 characters (usually in the middle of a <msg-id>) and passed it
on, to the great confusion of the next agent to process it.

I remember that, for some messages, I had to edit the text of the message
by hand before my browser would show it to me (or was it before I could
respond, I am not sure).

So long as such broken agents exist in the wild, the only safe procedure
is to prune the References so that it never exceeds 998, folded or not.

And preferably well before 998, because in practice a reading agent never
needs more than two or three msg-ids in order to generate a proper
threading.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21090
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Tue, 7 Jun 2005 13:03:07 GMT
Lines: 59
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> EVERY header in USEFOR needs a brief intrpductory section
>> explaining "what this header is good for".

>Not if it's essentially the same as in 2822.

Agreed. There is no need to write introductory text for the From header,
because we have nothing to add to RFC 2822.

But that is not the case for the References header.

>>> A References header indicates the previously posted context
>>> in which the article was created.

>That's not always and not necessarily the case.

>> effectively, you can deduce a tree or, more generally, a
>> DAG

>Is that a "directed acyclic graph" ?

Yes.

>  Whatever it is, this is
>not always and not necessarily the case.

Yes it is. If you are not trying to describe at least a DAG (of which
trees and linear lists are a particular case), then you have no business
putting a References header in your article. That is what Forrest is
worried about.

Note that I don't want to include the term "DAG" in any published text.
And note also that my current USEPRO text is only telling you how to
construct a tree (though it does point out, as does RFC 2822, that it has
deliberately limited itself to that).

Nevertheless, you cannot prevent non-trees. Every time someone tries to
followup to two precursors, and manually attempts to fix up the References
to reflect that (and people DO try to do that from time to time), then
that someone has constructed a DAG. We have to live with that, because we
cannot prevent it.

But I am not aware of anyone (trolls apart) who has ever tried to
construct, for example, a References header with a loop in it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21089
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Tue, 7 Jun 2005 12:50:42 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>It would surely kill USEFOR for good, yes.  No "threads", no
>nonsense, no aclycic graphs.  A msg-id-list is simply a list
>of msg-id-s, ready.  Harald's wording (or your version of it)
>was fine, any additional comma in 3.2.2 is asking for trouble.

No, the <msg-id-list> in the References header is not any random
collection of <msg-id>s. It is subject to constraints (e.g. that a
precursor does not come later in the list that any followup to it).

You can express this by referring to the rule for constructing it in
USEPOR (which is what my currently proposed text does), or you can express
it by writing some similar rules into USEFOR (which is what Forrest has
suggested), or you can express it by talking about parents coming earlier
than children (which is what my earlier suggestion did, though not using
the words 'parent' and 'child').

What you cannot do is to say nothing at all.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21092
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Info issues
Date: Tue, 7 Jun 2005 13:28:29 GMT
Lines: 54
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>...  Add authentication= with a reference
>to the new NNTPauth, Russ said that this is a good idea.
>Alexey would warn us if NNTPauth is blocked.

OK, that makes 3 people happy with that idea.

>That didn't change "x-", 2045 had attribute := token, and
>x-token is a special case of extension-token unrelated to
>attribute.  

>x-token is only used in mechanism (7bit, 8bit, etc.), in
>subtype, and in type (indirectly), not in any parameter.

OK, I looked again, and x-token is for use in experimental media types
(text/x-my-funny-text) and encodings (x-yenc) that have not been
standardized or registered.

RFC 2048 (which is the proper place to say such things) is silent on
x-tokens used as attributes, though the folklore is certainly that they
are available for use, and would never be standardized/registered as
such.

But, coming back to Injection-Info, there is no media type involved, so we
write whatever we think fit. In which case, I most certainly think that we
should allow x-attributes for injecting agents that wish to include
further information that does not fit into any of the categories provided
(and that is also what we said in draft-13, so we have already decided it
once).

>2049 is "minimal MIME conformance", 2231 did not update 2049.
>Anyway, for USEFOR we need security considerations about 2231:

>  MIME security considerations are discussed in [RFC2046].
>  Note that applying some [RFC2231] extensions for parameters
>  like multi-line paramters on a boundary parameter as defined
>  in [RFC2046] might be abused to bypass simple algorithms
>  trying to analyze MIME parts.

Posibly. Does anyone else want to see that written into our Security
Considerations?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21091
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
Date: Tue, 7 Jun 2005 13:11:33 GMT
Lines: 39
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> There is substantial existing usage of UT and GMT.  Therefore it _has_
>> to be a MUST accept.

>There is substantial existing usage of PST/MST/EST too.

Well I have never seen one, whereas when I last looked something like 20%
of articles in my news spool used GMT (and they weren't all from people
who lived in Greenwich :-) ).

>  (And a quibble:
>there is substantial existing usage of GMT.  I don't recall any usage of
>UT at all.)

That is true. UT just got dragged into the draft alongside GMT. It could
be omitted without loss.

>However, in the Date header, EST has a protocol-defined meaning.

Only in an obs- part of RFC 2822 which we have chosen to exclude (though
agents MAY use that interpretation if they want, and I suppose any
Australians who use it will be in breach of something, and their articles
will be discarded as 'stale', and they will put it down to some dastardly
pommie plot :-( ).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Russ Allbery <[email-masked]>
Subject: OT: "UT"
Date: Tue, 7 Jun 2005 11:26:58 -0400
Cc: [email-masked]
Sender: [email-masked]

On Mon June 6 2005 19:08, Russ Allbery wrote:
> I don't recall any usage of
> UT at all.

Well, there's substantial use of "UTC", though that is not
and never has been a valid standard zone abbreviation in
date-time in the message format.  It's a violation of RFCs
561, 724, 733, 822, 850, 1036, and 2822.  Fortunately, nobody
has proposed accepting it as if it were valid.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
Date: Tue, 7 Jun 2005 11:16:42 -0400
Cc: [email-masked]
Sender: [email-masked]

On Mon June 6 2005 12:36, Charles Lindsey wrote:

> >> You MUST NOT generate "EST" or any of the other things in <obs-zone>.
> 
> >Already covered by 2822.
> 
> Indeed, but worth reiterating because it would actually break the
> protocols.

Precisely how would "PDT" "break the protocols" in a way that "UT"
would not?

> It has said SHOULD NOT generate UT and GMT in our drafts for many years,
> and nobody has complained before.

This is not the first time the issue has come up.  In any event, your
argument is a misplaced "appeal to tradition" fallacy.

> I could easily be persuaded to promote 
> it to MUST NOT if someone asks for it. Are you formally proposing that?

I have said about as clearly as possible that I feel that we should
conform to RFC 2822 format as closely as possible.  Since 2822 clearly
says MUST NOT generate for alphabetic zone indication, yes, that is
what I have proposed.  Specifically, I propose no deviations from
RFC 2822 regarding date-time format.
 
> >>  News articles MUST conform to the syntax specified in Section 3 of
> >>  [RFC2822]. The obsolete syntax specified in Section 4 of [RFC2822] is
> >>  excluded (except for some special use of <obs-zone> in section 3.1.2
> >>  below [and any other exceptions we choose to make ....]); Netnews agents
> >>  MAY accept that obsolete syntax but MUST NOT generate productions of it.
> 
> >Making it longer and more convoluted isn't "helpful".  Eliding it in
> >favor of "[RFC2822]" would be helpful.
> 
> That would be a technical change to the standard we are proposing. You are
> usually the first to complain when editors make technical changes that the
> WG has not discussed.
> 
> This WG decided. more years ago than I care to remember, that it was
> ludicrous to REQUIRE all News software to be upgraded to accept
> constructes that had Never Ever been seen in Netnews.

You are mistaken in your implied assertion that handling of the standard
alphabetic zones for parsing would would be "constructes [sic] that had
Never [sic] Ever [sic] been seen in Netnews".  RFC 850:

   2.1.4  Date  The Date line (formerly   "Posted")   is  the
   date,  in  a  format  that  must be acceptable both to the
   ARPANET and to the getdate routine,

RFC 1036:

    The "Date" line (formerly "Posted") is the date that the message was
    originally posted to the network.  Its format must be acceptable
    both in RFC-822 and to the getdate(3) routine that is provided with
    the Usenet software.

I can of course dig up the aforementioned getdate routine source and
demonstrate that it accepts all of the standard RFC 2822 (deprecated
for new generation, required for parsing) alphabetic zone abbreviations
plus a slew of others.

As Russ has noted, there is *substantial* existing usage of alphabetic
zone abbreviations.

> That decision have never been challenged since then. Why are you raising
> it now?

Again, this has been discussed before.

> IIRC, it was you who asked for <obs-zone> to be brought within our
> standard.

References, please.

Date: Tue, 07 Jun 2005 08:56:49 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

Frank Ellermann wrote:

>>Forrest>A References header indicates the previously posted context
>>Forrest>in which the article was created.
> 
> 
> That's not always and not necessarily the case.

I understand people wish to relax the requirement that an article
be "posted." I disagree.  But, do you have any other cases which are
outside the description?

Frank, it seems like you want to push USEFOR to be basically
ABNF only, with almost no semantics.

Why would such a terse document merit its own separate RFC?

Date: Tue, 07 Jun 2005 14:24:29 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: "Forrest J. Cavalier III" <[email-masked]>
Cc: "Forrest J. Cavalier III" <[email-masked]>, [email-masked]
Subject: Re: CFWS in References header
Sender: [email-masked]

References is defined in 2822, while Xref and Supersedes are defined in 
USEFOR.
I'll open a ticket on whether comments in References should be listed as 
SHOULD NOT generate or MUST NOT generate - saying MUST NOT generate would, 
I think, make it equivalent with replacing CFWS with FWS.

               Harald

--On 7. juni 2005 07:26 -0400 "Forrest J. Cavalier III" 
<[email-masked]> wrote:

> Harald Tveit Alvestrand wrote:
>>
>> New subject....
>>
>> I was told earlier that the WG had a previous consensus that it would:
>> a) not change definitions from RFC 2822 (where References has CFWS)
>> b) allow CFWS wherever possible, with a "MUST accept/SHOULD NOT
>> generate" clause for certain headers (including References).
>>
>> If this is indeed the previously documented consensus, then Forrest's
>> comment is out of order - a reopening of past issues.
>
> CFWS was changed to FWS for Xref and Supersedes between April 2005
> (usefor-usefor-03) and May 2005 (usefor-usefor-04), so I thought any
> "previously documented consensus" must not have been all that important.
>
> If you are you asking me to withhold objections until Last Call,
> I will.  We seem to finally have a real push to end this torture,
> with either success or failure, in 10 weeks.  I'm all for that.
>
>
>
>
>
>

Date: Tue, 07 Jun 2005 07:26:06 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: Harald Tveit Alvestrand <[email-masked]>
CC: "Forrest J. Cavalier III" <[email-masked]>, [email-masked]
Subject: Re: CFWS in References header
Sender: [email-masked]

Harald Tveit Alvestrand wrote:
> 
> New subject....
> 
> I was told earlier that the WG had a previous consensus that it would:
> a) not change definitions from RFC 2822 (where References has CFWS)
> b) allow CFWS wherever possible, with a "MUST accept/SHOULD NOT 
> generate" clause for certain headers (including References).
> 
> If this is indeed the previously documented consensus, then Forrest's 
> comment is out of order - a reopening of past issues.

CFWS was changed to FWS for Xref and Supersedes between April 2005
(usefor-usefor-03) and May 2005 (usefor-usefor-04), so I thought any
"previously documented consensus" must not have been all that important.

If you are you asking me to withhold objections until Last Call,
I will.  We seem to finally have a real push to end this torture,
with either success or failure, in 10 weeks.  I'm all for that.

Date: Tue, 07 Jun 2005 10:03:53 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Russ Allbery <[email-masked]>, [email-masked]
Subject: Re: Injection-Info issues
Sender: [email-masked]

--On 6. juni 2005 16:05 -0700 Russ Allbery <[email-masked]> wrote:

> Authentication identity is
> very internal to the application and not generally of use or interest to
> anyone outside that application except for audit purposes.
>
>> So I propose a new Injection-Info parameter
>>     authentication=<value>
>
>> Would people be happy with that?
>
> Works for me.

I think Russ made a very good argument that revealing the authentication 
identity in the Injection-Info is a mistake.  Name it 
identifier-that-the-sysadmin-can-use-to-figure-out-who-sent-this, and it 
may make sense.

                       Harald

Date: Tue, 07 Jun 2005 09:27:32 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: "Forrest J. Cavalier III" <[email-masked]>, [email-masked]
Subject: CFWS in References header
Sender: [email-masked]

New subject....

I was told earlier that the WG had a previous consensus that it would:
a) not change definitions from RFC 2822 (where References has CFWS)
b) allow CFWS wherever possible, with a "MUST accept/SHOULD NOT generate" 
clause for certain headers (including References).

If this is indeed the previously documented consensus, then Forrest's 
comment is out of order - a reopening of past issues.

Did I hear it wrong that this was a previous consensus?

                    Harald

--On 3. juni 2005 01:45 -0400 "Forrest J. Cavalier III" 
<[email-masked]> wrote:

> BTW, CFWS in References is a silly unnecessary complication.  (CFWS in
> any structured header
> usepro Transitional Arrangements even admits comments in References will
> break existing software.
>
> Is the idea to make References irrelevant and unreliable, and then we
> don't

Date: Tue, 07 Jun 2005 09:09:09 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

--On 6. juni 2005 18:35 +0000 Charles Lindsey <[email-masked]> wrote:

> What does "thread" actually mean in the "Real World" (TM). IN this
> context, surely it means a "linear list", which is near enough to its
> meaning as used in so called "threaded newsreaders" to be OK in this sort
> of context.

I have rarely seen an example of a linear thread.....

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Injection-Info issues
Date:  Tue, 07 Jun 2005 03:09:16 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 72
Sender: [email-masked]

Charles Lindsey wrote:
 
> the IANA registry for each media type includes details
> of all the parameters appropriate for that media type.

Checking draft-lilly-text-troff-04.txt and RfC 3676, yes,
it does.

> we have not proposed an IANA Registry for Injection-Info
> parameters (thoug we could if we were so minded).

Not at the moment.  Add authentication= with a reference
to the new NNTPauth, Russ said that this is a good idea.
Alexey would warn us if NNTPauth is blocked.

> Which means that <attribute>s are different from <token>s,
> which is what I said.

That didn't change "x-", 2045 had attribute := token, and
x-token is a special case of extension-token unrelated to
attribute.  

x-token is only used in mechanism (7bit, 8bit, etc.), in
subtype, and in type (indirectly), not in any parameter.

Maybe I should test x-9bit for the nonets in RfC 4042. 

> 2045 most certainly did "strange things" with <x-token>s,
> and <attribute>s inherit those same "strange things".

In my copy <attribute> comes nowhere near to any <x-token>,
<extension-token>, <iana-token>, or <ietf-token>.  It's in
"appendix A - collected grammar", the last three pages.

"x-attribute" does not exist, it's a hallucination.  If you
say that you want to create it from scratch it's fine, you
could use the SPF trick.  Or merge it with 2231 <shudder />

> You just tell people that, if they post through your server,
> then the Injection-Info header will identify them. If they
> don't like that, then they have to go to a different server.
> That is the EU law (slightly simplified, I grant you).

Anoymous and pseudonumous access is a right in a German law,
and the threads discussing this issue put together in a row
are worse than our "Re: ferences" here.  I'm not directly
affected, my favourite server is news.clara.net in the UK ;-)

> In general, the Americans are less squeamish :-( .

They can't spell privacy without a dictionary.  They don't
understand why any @gmail address is begging for a *PLONK*.

> That vulnerability is nothing to do with RFC 2231.

It's about MIME, and this part of the debate was about USEFOR
2.3 "MIME Conformance":

| User agents MUST meet the definition of MIME-conformance in
| [RFC2049] and MUST also support [RFC2231].

2049 is "minimal MIME conformance", 2231 did not update 2049.
Anyway, for USEFOR we need security considerations about 2231:

  MIME security considerations are discussed in [RFC2046].
  Note that applying some [RFC2231] extensions for parameters
  like multi-line paramters on a boundary parameter as defined
  in [RFC2046] might be abused to bypass simple algorithms
  trying to analyze MIME parts.

That's all.  Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  OT: EST (was: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?)
Date:  Tue, 07 Jun 2005 01:36:13 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 11
Sender: [email-masked]

Charles Lindsey wrote:

> BTW, please will someone remind me the name of that timezone
> in Australia (wherever) that is abbreviated as "EST"

See <http://mid.gmane.org/[email-masked]> for
an opinion, but the whole thread is funny - Mark Davis meets
Clive Feather to discuss strange beasts like "BDT" = "British
Daylight saving Time"... ;-)  Apparently the NNTP "last call"
goes well, and LTRU is also almost ready with 3066bis.  Bye.

To: [email-masked]
Subject: Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 06 Jun 2005 16:08:20 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> There is substantial existing usage of UT and GMT.  Therefore it _has_
> to be a MUST accept.

There is substantial existing usage of PST/MST/EST too.  (And a quibble:
there is substantial existing usage of GMT.  I don't recall any usage of
UT at all.)

> Indeed, but worth reiterating because it would actually break the
> protocols. BTW, please will someone remind me the name of that timezone
> in Australia (wherever) that is abbreviated as "EST").

Eastern Standard Time.  :)

However, in the Date header, EST has a protocol-defined meaning.

To: [email-masked]
Subject: Re: Injection-Info issues
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 06 Jun 2005 16:05:00 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Ah! So it depends what "hat" the guy is wearing at a particular time.

Right.  The authorization identity is the "hat," so to speak, and the
authentication identity is who you proved you were in order to get to wear
the hat.

> E.g.  Alice and Bob may both be entitled to wear the "sysadmin" hat, but
> each of them is not necessarily wearing that hat each time they
> authenticate (they sometimes just post as themselves). So when Alice
> authenticates (as Alice), she may (or may not) claim the authorization
> "sysadmin", and presumably the server knows that Alice and Bob have
> "sysadmin" privileges, but that Mallet most definitely does not.

Right.

>> Your authorization identity is generally going to be the thing that
>> goes into sender and the like.

> Hmm! I am not quite convinced of that. RFC 2822 (which we follow) is
> fairly clear as to what goes into the Sender header.

Yes, it's the authorization identity:

   The "Sender:" field specifies the mailbox of the agent responsible for
   the actual transmission of the message.  For example, if a secretary
   were to send a message for another person, the mailbox of the secretary
   would appear in the "Sender:" field and the mailbox of the actual
   author would appear in the "From:" field.

What's confusing is that there is yet another level of even weaker binding
in the From header, but the key part of the above is "the agent
responsible for the actual transmission of the message."  Whether Alice or
Bob sent the message, "secretary" was the agent that sent the message.
There are just various people who are authorized to use that agent and act
as the secretary.  Alice and Bob may not even *have* their own mailboxes;
they just have separate identities in the authentication system because
that's a good security practice.

The easiest way to think about it from the security standpoint is to
observe that, in general, the *only* thing that cares about the
authentication identity is your authorization component.  Once you finish
the authentication and authorization process, the authorization identity
is the effective identity for the rest of the application; the
authentication identity is used only for the authorization process and for
internal audit trails.  Sender is intended to capture the case where one
agent sends out mail on behalf of another agent, *not* the case where
multiple people can control a single agent. Authentication identity is
very internal to the application and not generally of use or interest to
anyone outside that application except for audit purposes.

> So I propose a new Injection-Info parameter
>     authentication=<value>

> Would people be happy with that?

Works for me.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date:  Tue, 07 Jun 2005 01:00:51 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 44
Sender: [email-masked]

Charles Lindsey wrote:
 
>> I don't think it is efficient for the group to consider this
>> problem once now for USEFOR and then once again later for
>> USEPRO.
 
> I agree entirely with the above.

And I disagree entirely.  The data structure (USEFOR) is clear,
the algorithms (USEPRO) will fit.

> There is no way that we can publish USEFOR before we publish
> USEPRO.

There is no way that this WG will still exist in ten weeks
without an approved USEFOR RfC submitted to the in-queue.  Or
at the very minimum in "last call".

> EVERY header in USEFOR needs a brief intrpductory section
> explaining "what this header is good for".

Not if it's essentially the same as in 2822.  And saying that
a msg-id-list is a list of msg-ids is a waste of time.  It's
like saying that id-domain is a domain.  The only reason why
you are forced to say the latter is because you insist on the
vague and bad id-right idea found in 2822.

>> A References header indicates the previously posted context
>> in which the article was created.

That's not always and not necessarily the case.

> effectively, you can deduce a tree or, more generally, a
> DAG

Is that a "directed acyclic graph" ?  Whatever it is, this is
not always and not necessarily the case.  The msg-id-list is a
list of msg-ids.
 
> it is probably time to post a fresh version of those texts,
> taking account of discussions these last few days.

That would be the "internal last call" for USEFOR, good.  Bye.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date:  Tue, 07 Jun 2005 00:29:33 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Charles Lindsey wrote:
 
> We are agreed that the References heaader should only be used
> in situations where threading the articles by reading agents
> would make sense.

This would be an agreement over my dead body, but I don't feel
like I'm dead.  We "agreed" to describe the followup procedure
as is in USEPRO.  The mere word "thread" is taboo for USEFOR.

> So using that word, in what is only meant as an introductory
> remark to the References header section, would surely
> reinforce that intention.

It would surely kill USEFOR for good, yes.  No "threads", no
nonsense, no aclycic graphs.  A msg-id-list is simply a list
of msg-id-s, ready.  Harald's wording (or your version of it)
was fine, any additional comma in 3.2.2 is asking for trouble.

                         Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:21079
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Mon, 6 Jun 2005 18:35:31 GMT
Lines: 42
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> Yes, that says the right things

>+1

>> How about:

>Also fine, maybe better.

> [3.2.x propossal]
>>| in order to facilitate the display or retrieval, by reading
>>| and other agents, of threads of related articles.

>It's not only for "threads of", it can be anything the poster
>wants as a "relation" for "display or rertrieval", it can be
>also a "relation" unknown for most readers (e.g. follwup in a
>different TLH, or article posted as reply to a mail in a list)

But we don't formally define "threads". We are agreed that the References
heaader should only be used in situations where threading the articles by
reading agents would make sense. So using that word, in what is only meant
as an introductory remark to the References header section, would surely
reinforce that intention.

What does "thread" actually mean in the "Real World" (TM). IN this
context, surely it means a "linear list", which is near enough to its
meaning as used in so called "threaded newsreaders" to be OK in this sort
of context.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21077
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Info issues
Date: Mon, 6 Jun 2005 17:02:03 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>You could also use the trick in the most recent SPF drafts -
>not as good as an RfC, but the ABNF passed Bill's checker:

>| header           = "Received-SPF:" [CFWS] result FWS [comment FWS]
>|                    [ key-value-list ] CRLF
>[...]
>| key-value-list   = key-value-pair *( ";" [CFWS] key-value-pair )
>|                    [";"]
>|
>| key-value-pair   = key [CFWS] "=" ( dot-atom / quoted-string )

>| key              = "client-ip" / "envelope-from" / "helo" /
>|                    "problem" / "receiver" / "identity" /
>|                    mechanism / "x-" name / name

>Continued later, I've to Cc: this part to the SPF editor, bye.

Then I need to wait for your "continued later", because what you have said
gives no guidance as to which <-atom / quoted-string> is syntactically
allowed for each of the various <key>s.

For example, "client-ip" presumably expects an <IPv4address> or
<IPv6address>. "envelope-from" expects an <addr-spec>. And so on.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21078
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Info issues
Date: Mon, 6 Jun 2005 18:23:45 GMT
Lines: 121
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> RFC 2045 makes a special case for <x-token>s, and we want to
>> buy into that.

>2045 / 2048 define IANA registries for the Content-Type among
>others, but AFAIK not for any attribute=value parameters.

AIUI, the IANA registry for each media type includes details of all the
parameters appropriate for that media type.

>And I won't buy into any "let's create another IANA registry"
>scheme - unless my aim is a new IESG "[Discuss]" record... ;-)

No, we have not proposed an IANA Registry for Injection-Info parameters
(thoug we could if we were so minded).

>> by the time we get to RFC 2231, <token>s have morphed into
>> <attribute>s.

>Our copies of 2045 are apparently not strictly identical, 2231
>only adds initial-section and other-sctions to attribute, and
>for that trick it removes "*" from the attribute-char set.

Which means that <attribute>s are different from <token>s, which is what I
said.

>And maybe because they were lazy they also removed "'" and "%"
>using the same attribute-char set also for the values, and not
>only for the attribute.

>2045 or 2231 don't do strange things with x-attributes, that's
>a hallucination.

2045 most certainly did "strange things" with <x-token>s, and <attribute>s
inherit those same "strange things".

>  IMHO we don't need x-attributes.

Wwe have little choice. They come with the territory.

>> Whatever is wrong with designing formats that simplify the
>> things that people are going to use them for?

>If some Injection-Info parameters allow to identify users by
>third parties without the user's consent no legit German news
>server can use them, and so their potential "user-friendliness
>for killfiling" is moot.  Forget it, that's irrelevant.

You just tell people that, if they post through your server, then the
Injection-Info header will identify them. If they don't like that, then
they have to go to a different server. That is the EU law (slightly
simplified, I grant you).

But for servers that don't care to include names, there are other options.
In general, the Americans are less squeamish :-( .

>>>> they SHOULD come in a prescribed order
>>> See (a), it's not worth it.
>> It _could_ be a USEAGE matter

>2045 says that "the ordering of parameters is not significant".

Indeed, and this makes a SHOULD exception. But I have already said USEAGE
would be good enough for that bit.

>> It the "posting-account" parameter runs into legal
>> difficulties, then use a "logging-data" parameter which is
>> designed to be inscrutable without the cooperation of the ISP

>Okay, for USEFOR all we need is the syntax, no legal advice.

USEFOR doesn't give any legal advice. It just provides some facilities
which which injecting agents may or may not choose to use. There might be
some legal advice in USEAGE.

>>>| IMPORTANT NOTE:  These mechanisms end up being somewhat
>>>| gibbous when they actually are used. As such, these
>>>| mechanisms should not be used lightly; they should be
>>>| reserved for situations where a real need for them exists.

>> Yes, but that is similar to my wording.

>Sure, but either you want 2231 I18N, then gibbous or not it's
>the way it is, or you don't want it, then the SPF trick with
>its key-value-pair bypasses 2231.

So? My wording was quite happy for the 2231 I18N to be used. It was the
multiline stuff that was being discouraged.

>Wanting 2231 and at the same time not wanting it is the weird
>"MUST accept SHOULD NOT use" stuff again - I really hate this.

There is no "MUST accept SHOULD NOT use" wording in any text proposed for
2231, AFAIAA.

>> Please be more specific which risks/vulnerabilities you are
>> talking about.

>http://www.kb.cert.org/vuls/id/836088

That vulnerability is nothing to do with RFC 2231. It affects
message/partial which is hardly used in News, so I do not see any point
even in mentioning it in Security Considerations. But if people want it
mentioned, that is no problem.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21073
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
Date: Mon, 6 Jun 2005 16:36:14 GMT
Lines: 104
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Fri June 3 2005 08:17, Charles Lindsey wrote:

>> You MUST accept "UT" and "GMT".

>RFC 2822 already says that. No "exception" here.

Sigh! But we have already excluded that bit of RFC 2822 (so this is an
"exception" to that "exclusion").

>> You SHOULD NOT generate "UT" and "GMT".

>There has never been any convincing argument that there should be a
>deviation from RFC 2822 (and RFC 1123) here.

>  Ancient implementations
>might well be non-conforming to what we specify for requirements on
>new generation, and no argument has been presented as to why that
>should differ from 2822 (viz. use numeric offsets only for generation).
>I.e., while this does represent an "exception", there has been no
>justification for such.

There is substantial existing usage of UT and GMT. Therefore it _has_ to
be a MUST accept. It is, morevoer, important to get it right because the
News protocols _require_ <date-time>s to be compared for earlier/later (a
requirement that never arises in Email AFAIAW).

>> You MUST NOT generate "EST" or any of the other things in <obs-zone>.

>Already covered by 2822.

Indeed, but worth reiterating because it would actually break the
protocols. BTW, please will someone remind me the name of that timezone in
Australia (wherever) that is abbreviated as "EST").

>....  Especially, there is no justification for simultaneously
>being more restrictive (MUST accept -> MAY accept) for some alphabetic
>zones and less restrictive (MUST NOT generate -> SHOULD NOT generate)
>for one particular zone ...

It has said SHOULD NOT generate UT and GMT in our drafts for many years,
and nobody has complained before. I could easily be persuaded to promote
it to MUST NOT if someone asks for it. Are you formally proposing that?

> ... (purely coincidental that that happens to be
>a Document Editor's zone, and that said Editor's mail to this mailing
>list remains non-compliant with 2822 while the vast majority of
>messages are compliant... :-).

I do not find your smiley amusing, since you have made this snide remark
before. As I have explained to you previously, I read this mailing list in
a local newsgroup, and I have better things to do that spend time
upgrading my News software while this WG is still debating what News is
supposed to look like. As for the gateway from my news-server to this
list, there seemed little point in changing that "GMT", because it is a
MUST accept for Email.
>  
>> So that paragraph in 2.1 could more helpfully say:
>> 
>>  News articles MUST conform to the syntax specified in Section 3 of
>>  [RFC2822]. The obsolete syntax specified in Section 4 of [RFC2822] is
>>  excluded (except for some special use of <obs-zone> in section 3.1.2
>>  below [and any other exceptions we choose to make ....]); Netnews agents
>>  MAY accept that obsolete syntax but MUST NOT generate productions of it.

>Making it longer and more convoluted isn't "helpful".  Eliding it in
>favor of "[RFC2822]" would be helpful.

That would be a technical change to the standard we are proposing. You are
usually the first to complain when editors make technical changes that the
WG has not discussed.

This WG decided. more years ago than I care to remember, that it was
ludicrous to REQUIRE all News software to be upgraded to accept
constructes that had Never Ever been seen in Netnews. They "MAY" accept
them, of course, and agents that are intended for joint Email/News use
likely will.

That decision have never been challenged since then. Why are you raising
it now?

>> What I would like to see is <obs-phrase> (possibly renamed as
>> <extended-phrase>) treated as a special case with similar status to "UT"
>> and "GMT".

>As noted above, there seems to be no justification for the deviations;
>it appears to be a personal prejudice.

IIRC, it was you who asked for <obs-zone> to be brought within our
standard.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21074
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Path header - suggested text
Date: Mon, 6 Jun 2005 16:40:58 GMT
Lines: 26
Sender: [email-masked]

In <[email-masked]> Henry Spencer <[email-masked]> writes:

>> Eh? What is "AAAA"?

>An AAAA record is the IPv6 equivalent of an A record.  IPv6 addresses are
>four times as long as IPv4 addresses, so the name seemed obvious... :-)

Ah! I see.

It now says:

|  1. A fully qualified domain name (FQDN) associated with an "A" or
|     "AAAA" record, which SHOULD identify the actual host inserting
|     this <path-identity> and, ideally, should also be "mailable" (see
|     below).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21075
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Path header - suggested text
Date: Mon, 6 Jun 2005 16:45:54 GMT
Lines: 32
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Fri June 3 2005 12:34, Henry Spencer wrote:

>> We used both -- rather cautiously -- as references when we wrote C News. 

>And a year ago we discussed what C news (and INN) do and do not
>treat as delimiters.

>Why are we still flogging this horse?

We indeed flogged it very thoroughly, and one thing we concluded was that
aguments related to what some implementation of B News did simply did not
wash.

But the text now proposed is the final outcome of that discussion a year
ago, and what is and is not allowed in both <path-identity>s and
<path-delimiters> has been reduced to a minimum that should cause no
problems whatsoever on the existing network.

So stop flogging.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21076
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Info issues
Date: Mon, 6 Jun 2005 16:56:12 GMT
Lines: 72
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> Whichever. I find the distinction made by the SASL standards far from
>> clear. Is a man allowed to choose any string he likes as his
>> "authorization identity", or does it have to be something previously
>> agreed with the server?

>It's fairly clear if you understand the distinction between authentication
>and authorization.  Authentication says who you are.  Authorization says
>what you're allowed to do.  The authorization identity is the identity
>you're assuming on the server and with which ACLs are associated -- it may
>be, for instance, a group account shared by several people, or a role in
>which you're acting.  The authentication identity is who you actually are,
>namely some identified user who has access to that role.  Depending on the
>authentication mechanism, the two identities may live in different
>namespaces.

Ah! So it depends what "hat" the guy is wearing at a particular time. E.g.
Alice and Bob may both be entitled to wear the "sysadmin" hat, but each of
them is not necessarily wearing that hat each time they authenticate (they
sometimes just post as themselves). So when Alice authenticates (as
Alice), she may (or may not) claim the authorization "sysadmin", and
presumably the server knows that Alice and Bob have "sysadmin" privileges,
but that Mallet most definitely does not.

>Your authorization identity is generally going to be the thing that goes
>into sender and the like.

Hmm! I am not quite convinced of that. RFC 2822 (which we follow) is
fairly clear as to what goes into the Sender header.

>> OK, just call it "auth":

>>    Injection-Info: dodgy.net; posting-host=dialup-123.dodgy.net;
>>                    auth="some-string"

>It's generally wise to never use the term "auth" in a security context
>because it's ambiguous on whether you're talking about the authentication
>identity or the authorization identity.  You need at least "authn" or
>"authz", and it's probably better to just spell it out.

>> Or else use it in a "posting-account" parameter.

>That would be the authorization identity, not the authentication identity.

Ah! That makes sense.

So I propose a new Injection-Info parameter
    authentication=<value>

Would people be happy with that?

>> I just want to make sure that such authentication-identities can be
>> used, and that there exists a parameter clearly suitable for the
>> purpose.

>Yup.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21070
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Mon, 6 Jun 2005 15:48:20 GMT
Lines: 123
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>Harald Tveit Alvestrand wrote:

>> I have deliberately not discussed USEPRO. I believe that we can do one 
>> thing at a time.

>You cannot change a definition in USEFOR and ignore that it requires
>a change in USEPRO.  Fragility due to dependency chains is a well-studied
>effect.

>These are companion documents split only because they were too verbose.
>I do not believe they can be edited, issued, or discussed independently.

>I do not think it is going to be a "win" to push out USEFOR, and then
>see how in the world we can write a USEPRO that requires no changes to
>USEFOR.

>I don't think it is efficient for the group to consider this problem once
>now for USEFOR and then once again later for USEPRO.

I agree entirely with the above. There is no way that we can publish
USEFOR before we publish USEPRO. Maybe we can leave USEAGE until the other
two are out of the way.

>> At the moment, USEFOR contains very little information about how to 
>> interpret ANY header. Are you suggesting to change this?

>I wrote "how to interpret," but I did not mean "how to display."

Indeed. EVERY header in USEFOR needs a brief intrpductory section explaining
"what this header is good for". I think References is the only header
lacking that in USEFOR-04, but I have already proposed text to fill that
gap.

Often, that introduction is sufficient for the "semantics" of that header,
especially if the protocols in USEPRO tells you exactly what to do when you
encounter it.

But if agents are going to use it for a variety of purposes, then more may
be needed, since USEPRO is not the place to go into detail of how to
display things (though USEAGE might be).

>So, instead of making an attempt at specifying the conditions that
>it is acceptable, we have to allow it without limitation?

No. We describe the intended "purposes" of the header. In this case to
facilitate "threading", "retrieval" (of parents) and "killfiling" (for
killing whole subthreads) (my proposed introductory text mentions the
first two of those, which I think is enough). If you create a References
header, then you should be aware it is likely to be used for those
purposes, and if those are not what you intend, then you shouldn't be
creating it.

>> 
>> But changing USEFOR to contain this level of information would change 
>> the split between USEFOR and USEPRO, and I don't see a good reason for 
>> that.

>I think that's because you confuse semantics and implementation.

>Here is a specification of "semantics" (my proposed References definition
>for USEFOR, with some RFC2822 text, and a correct trimming algorithm,
>unlike the ambiguity currently in USEPRO.)

>    A References header indicates the previously posted context in which
>    the article was created.

That is an "introductory text", but it does not really tell you anything
interesting. The one I proposed goes further.

>    The "References:" field will contain the contents of the parent's
>    "References:" field (if any) followed by the contents of the parent's
>    "Message-ID:" field.  To prevent excessively long lines, message
>    identifiers from the parent's References field can be omitted, provided
>    that the first and last one or more in that field are retained.

But that is just telling you how to construct it, and since that is going
to be described in USEPRO, it does not need to be mentioned here
(especially the bit about pruning).

You might argue that if (from USEPRO) you know how to construct it, then
you have sufficient information to work out what it "means". I don't
entirely buy that, and in an earlier draft of my "suggestions" I had some
"semantics" text telling what parent/child relationships you could deduce
from it (effectively, you can deduce a tree or, more generally, a DAG).

But I took that text out because people did not think it necessary. I
could be persuaded to put it back.

>Here is a specification of "implementation" (text from USEPRO-03)

>    If the precursor did not have a References header (F-3.2.1), ....

Yes, but that text is way out of date. I have already posted my suggested
text, which tells you how to contruct a References header in the general
case given a "parent" article, and then invokes that for the followup case
in particular. It also ties up the pruning bit more precisely, because
Frank asked for that.

>Semantics based is clearer, complete, and shorter already, but the USEPRO
>text I showed here is not complete.

That is because you have ignored the more recent texts posted to this
list.

Come to think of it, it is probably time to post a fresh version of those
texts, taking account of discussions these last few days.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21071
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Mon, 6 Jun 2005 15:57:15 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> "Ruud H.G. van Tol" <[email-masked]> writes:

>Forrest J. Cavalier III:

>> This is in RFC2822, which I copied...
>> 
>>     The "References:" field will contain the contents of the parent's
>>     "References:" field (if any) followed by the contents of the
>>     parent's "Message-ID:" field (if any).
>> 
>> and one of your criticisms was that "contents of" was nebulous, and
>> you provided an example.
>> 
>> Are you saying the phrase "contents of" is not nebulous in RFC2822,
>> but would be nebulous in USEFOR?  Can you explain?

>The contents of the parent's "Message-ID:"-field will sometimes 
>contain more than just a <msg-id>, like trailing whitespace, or 
>a comment, or worse.

Yes, we know that the definition of "content" in USEFOR is not consistent
with the way it is used in USEPRO. That was pointed out on this list
months ago, and the document editors are discussing ways to fix it.

May I suggest that you all stick to the important technical issues under
discussion, and leave it to the document editors to sort out matters of
consistency in the texts.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21072
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Mon, 6 Jun 2005 16:06:53 GMT
Lines: 46
Sender: [email-masked]

In <[email-masked]> Eivind Tagseth <[email-masked]> writes:

>* Charles Lindsey <[email-masked]> [2005-06-03 15:01:37 +0000]:

>> 3.2.1  References
>> 
>> |  The References header is used to refer to earlier articles (e.g.
>> |  followups refer to their precursors), in order to facilitate the

>More confusion.  You now say that the references header refers to
>"earlier" articles, while an important point of using this complicated
>followup/references definitions is to allow for multi-part FAQs to
>be posted "at the same time".  "earlier" seems to contradict this.

OK s/earlier/previous/

Clearly, References headers are only appropriate when there as a sequence
of articles with some ordering. "Earlier" implies an ordering in time
(which is true for normal followups). "Previous" is regularly used to
imply an ordering in other senses, e.g. of items in a list.

>I'll try this again:

>    A "followup" is an article that follows up on the contents of another
>    article, its "precursor".  Every followup includes a References header
>    identifying that precursor.

No, I think the consensus of this group is to keep "followup" (and its
converse "precursor") for the genuine reply/response situation, and
Harald's text (or my slight variation of it) expresses that.

The word "parent" may be useful, and indeed I have proposed to use it in
USEPRO.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Henry Spencer <[email-masked]>
Subject: Re: Path header - suggested text
Date: Sun, 5 Jun 2005 09:03:12 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 3 2005 16:40, Henry Spencer wrote:
> 
> On Fri, 3 Jun 2005, Bruce Lilly wrote:
> > > Because you again tried to resolve vagueness in 1036 by appealing to the
> > > B2.11 code, as if it was a model implementation of 1036, which it wasn't. 
> > 
> > No, the code provides insight into what the 1036 text "punctuation"
> > means.
> 
> The only way it can provide such insight is if you first assume that it is
> meant to be an exact implementation of 1036 (or equivalently that 1036 was
> written as a specification matching the B2.11 code).

Incorrect.  The only "assumption" is that when the RFC 1036/B News 2.11
authors -- the same people -- write "punctuation" or an abbreviation thereof
in the same limited context in the same timeframe with an explicit cross-
reference between documents, that it in fact means the same thing in that
limited context.

> That assumption is 
> unsubstantiated, and must be considered highly questionable given the
> numerous differences between what 1036 says and what B2.11 does.

Utter nonsense.  The real, minimal "assumption" is quite reasonable, unlike
other claims that have been made in this WG, e.g. that the term is based
on some Unicode or POSIX set (both Unicode and POSIX post-date 1036).

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Injection-Info issues
Date:  Sat, 04 Jun 2005 02:18:48 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 86
Sender: [email-masked]

Charles Lindsey wrote:

> RFC 2045 makes a special case for <x-token>s, and we want to
> buy into that.

2045 / 2048 define IANA registries for the Content-Type among
others, but AFAIK not for any attribute=value parameters.

And I won't buy into any "let's create another IANA registry"
scheme - unless my aim is a new IESG "[Discuss]" record... ;-)

> by the time we get to RFC 2231, <token>s have morphed into
> <attribute>s.

Our copies of 2045 are apparently not strictly identical, 2231
only adds initial-section and other-sctions to attribute, and
for that trick it removes "*" from the attribute-char set.

And maybe because they were lazy they also removed "'" and "%"
using the same attribute-char set also for the values, and not
only for the attribute.

2045 or 2231 don't do strange things with x-attributes, that's
a hallucination.  IMHO we don't need x-attributes.  As long as
the parameters are unique there's no additional 2231 problem.

>>> a) folding SHOULD NOT happen in silly places
>> Let's better forget it.
> Well, it's in the present text of USEFOR.

Okay.  OTOH that's a general rule, 2.2.3 in 2822 says "folding
SHOULD be limited to placing the CRLF at higher-level syntactic
breaks".  For Injection-Info that's the semicolon.

> Whatever is wrong with designing formats that simplify the
> things that people are going to use them for?

If some Injection-Info parameters allow to identify users by
third parties without the user's consent no legit German news
server can use them, and so their potential "user-friendliness
for killfiling" is moot.  Forget it, that's irrelevant.

>>> they SHOULD come in a prescribed order
>> See (a), it's not worth it.
> It _could_ be a USEAGE matter

2045 says that "the ordering of parameters is not significant".

2231 adds that "while MIME does prohibit modification of MIME
headers during transport, it is still possible that parameters
will be reordered when user agent level processing is done."

If you want to "buy into" this 2231 I18N you get it wholesale.

> It the "posting-account" parameter runs into legal
> difficulties, then use a "logging-data" parameter which is
> designed to be inscrutable without the cooperation of the ISP

Okay, for USEFOR all we need is the syntax, no legal advice.

>>| IMPORTANT NOTE:  These mechanisms end up being somewhat
>>| gibbous when they actually are used. As such, these
>>| mechanisms should not be used lightly; they should be
>>| reserved for situations where a real need for them exists.

> Yes, but that is similar to my wording.

Sure, but either you want 2231 I18N, then gibbous or not it's
the way it is, or you don't want it, then the SPF trick with
its key-value-pair bypasses 2231.

Wanting 2231 and at the same time not wanting it is the weird
"MUST accept SHOULD NOT use" stuff again - I really hate this.

> Please be more specific which risks/vulnerabilities you are
> talking about.

http://www.kb.cert.org/vuls/id/836088

About 34,000 Google hits for "MIME partial vulnerability".  Is
US-CERT something we could reference in an IETF RfC, or is that
source too obscure ?

We could also mention the potential 2231 boundary problems and
forget the message/partial stuff, it's old news.  Bye, Frank

Date: Fri, 3 Jun 2005 16:40:40 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: [email-masked]
Subject: Re: Path header - suggested text
Sender: [email-masked]

On Fri, 3 Jun 2005, Bruce Lilly wrote:
> > Because you again tried to resolve vagueness in 1036 by appealing to the
> > B2.11 code, as if it was a model implementation of 1036, which it wasn't. 
> 
> No, the code provides insight into what the 1036 text "punctuation"
> means.

The only way it can provide such insight is if you first assume that it is
meant to be an exact implementation of 1036 (or equivalently that 1036 was
written as a specification matching the B2.11 code).  That assumption is
unsubstantiated, and must be considered highly questionable given the
numerous differences between what 1036 says and what B2.11 does.  The
B2.11 code cannot provide *reliable* insight into ambiguities in 1036. 

> We know that it does NOT mean underscore, and that it does mean colon.

We might *speculate* to that effect, but we don't *know* it.

The fact is, 1036 doesn't say exactly what "punctuation" means.  Period. 

Moreover, it doesn't really matter.  We're trying to write a new document,
not to help 1036's authors revise theirs; what they *intended* is simply
irrelevant. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Path header - suggested text
Date:  Fri, 03 Jun 2005 22:30:18 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Bruce Lilly wrote:

> We know that it does NOT mean underscore, and that it does
> mean colon.

| Letters, digits, periods and hyphens are considered part of
| host names; other punctuation, including blanks, are
| considered separators.

That's 100% clear, and whether it's wrong is beside the point.

If "we" want the underscore for UUCP host names, then it needs
a "note" like the existing "note" for the colon.  Or it needs
s-o-1036 published as RfC, that should simplify some problems
of inheritance.

I'd just drop underscore in path-identity and be done with it.
Allowing it only in the tail-entry is no syntactical problem.

                        Bye, Frank

Date: Fri, 3 Jun 2005 22:06:05 +0200
From: Eivind Tagseth <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Mail-Followup-To: Eivind Tagseth <[email-masked]>, [email-masked]
Sender: [email-masked]

* Charles Lindsey <[email-masked]> [2005-06-03 15:01:37 +0000]:

> Yes, that says the right things, but I think the text can be tidied up a
> bit. How about:
> 
>    A "followup" is an article containing a response to the contents of an
>    earlier article, its "precursor". Every followup includes a References
>    header identifying that precursor (but note that non-followup articles
>    may also use a References header).

> 3.2.1  References
> 
> |  The References header is used to refer to earlier articles (e.g.
> |  followups refer to their precursors), in order to facilitate the

More confusion.  You now say that the references header refers to
"earlier" articles, while an important point of using this complicated
followup/references definitions is to allow for multi-part FAQs to
be posted "at the same time".  "earlier" seems to contradict this.

I'll try this again:

    A "followup" is an article that follows up on the contents of another
    article, its "precursor".  Every followup includes a References header
    identifying that precursor.

    The References header is used to identify the parentage of the article
    by identifying its precursors.

Based on dictionary lookups, I cannot see why multi-part FAQs aren't
covered by this.

www.wordsmyth.net:

Parentage:
 Definition	1.	descent from parents and ancestors; lineage.
 Definition	2.	the condition or relationship of a parent; parenthood.

Follow-up:
 Definition	1.	the act or process of following up, esp. with something that may increase the effectiveness of an earlier action.
 Definition	2.	the means, such as a phone call or letter, used to follow up

merriam-webster:

Follow up:
 1 : to follow with something similar, related, or supplementary <following up his convictions with action -- G. P. Merrill>
 2 : to maintain contact with (a person) so as to monitor the effects of earlier activities or treatments
 3 : to pursue in an effort to take further action <the police follow up leads>

Precursor:
 1 a : one that precedes and indicates the approach of another b : PREDECESSOR
 2 : a substance, cell, or cellular component from which another substance, cell, or cellular component is formed

Eivind

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Henry Spencer <[email-masked]>
Subject: Re: Path header - suggested text
Date: Fri, 3 Jun 2005 15:53:51 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 3 2005 15:28, Henry Spencer wrote:

> Because you again tried to resolve vagueness in 1036 by appealing to the
> B2.11 code, as if it was a model implementation of 1036, which it wasn't. 

No, the code provides insight into what the 1036 text "punctuation"
means.

We know that it does NOT mean underscore, and that it does mean colon.

Date: Fri, 3 Jun 2005 12:50:21 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

Seth Breidbart <sethb@xxxxxxxxx>:

>"Thread it" isn't necessarily a display issue. 

It is only a display issue. Nothing in the news system cares how an 
article is threaded other than the display agent.

>That makes the assumption that "followup" iff "contains References
>header".  We voted against that assumption.  Deal with it.

We have no authority to vote out sections of RFC1036, which is the current
RFC dealing with news. Currently, the standard says "required/prohibited".
There is no "please display this article this way" header; References
deals only with expressing followup relationships. Thus, the poster cannot
possibly be saying "please display me this way", or even "please thread me
as if I were a followup", because there is no header that expresses that
sentiment.

>But based on both the vote here and reality, the poster is _not_
>calling the article a followup,

No vote here can change why a poster called his article a followup (or
that he did so), and the current standard says that his article MUST be a
followup -- otherwise it is not a valid article, and invalid articles are
outside the scope of our standard. Invalid articles are not supposed to be
transported, much less processed as if they said "display this article
this way".

>> All we know is that he DID call it a followup for some reason
>> because it contains a header that is allowed only in followups.

>We voted otherwise.

Link to proof? Link to authority to vote out existing standards?

Date: Fri, 3 Jun 2005 15:28:16 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: [email-masked]
Subject: Re: Path header - suggested text
Sender: [email-masked]

On Fri, 3 Jun 2005, Bruce Lilly wrote:
> > We used both -- rather cautiously -- as references when we wrote C News. 
> 
> And a year ago we discussed what C news (and INN) do and do not
> treat as delimiters.
> Why are we still flogging this horse?

Because you again tried to resolve vagueness in 1036 by appealing to the
B2.11 code, as if it was a model implementation of 1036, which it wasn't. 

The vagueness in 1036 is inherent and unfixable.  We need to think about
what the best resolution of it is today, rather than trying to deduce what
1036's authors intended nearly 20 years ago. 

The practices followed by existing implementations are relevant only
insofar as they impose backward-compatibility constraints.  In that
regard, B2.11 can safely be ignored, and even C News is marginal. 

                                                          Henry Spencer
                                                       [email-masked]

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Henry Spencer <[email-masked]>
Subject: Re: Path header - suggested text
Date: Fri, 3 Jun 2005 15:04:50 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 3 2005 12:34, Henry Spencer wrote:

> We used both -- rather cautiously -- as references when we wrote C News. 

And a year ago we discussed what C news (and INN) do and do not
treat as delimiters.

Why are we still flogging this horse?

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Forrest J. Cavalier III" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Fri, 3 Jun 2005 15:02:00 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 3 2005 12:25, Forrest J. Cavalier III wrote:
> 
> Bruce Lilly wrote:
> 
> > I see no error in the relevant sections of RFC 2822.
> > 
> 
> This is in RFC2822, which I copied...
> 
>     The "References:" field will contain the contents of the parent's
>     "References:" field (if any) followed by the contents of the parent's
>     "Message-ID:" field (if any).
> 
> and one of your criticisms was that "contents of" was nebulous, and
> you provided an example.
> 
> Are you saying the phrase "contents of" is not nebulous in RFC2822, but
> would be nebulous in USEFOR?  Can you explain?

It's indeed nebulous, but falls short of an error.  It's unfortunate that
it lacks clarity, but this WG isn't tasked with rewriting 2822.  It should
suffice to reference 2822 and hope that the text will be clarified in its
successor.  I don't see much point in copying the text verbatim into our
documents.  At most, we might have a note clarifying the matter.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date:  Fri, 03 Jun 2005 20:52:10 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 6
Sender: [email-masked]

Forrest J. Cavalier III wrote:

> I don't know what you mean by "worse."

Take this.

Date: Fri, 03 Jun 2005 14:22:06 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: "Ruud H.G. van Tol" <[email-masked]>
CC: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

> The contents of the parent's "Message-ID:"-field will sometimes 
> contain more than just a <msg-id>, like trailing whitespace, or 
> a comment, or worse.
> 

The draft states that comments are forbidden in a Message-ID.

You can read the draft here:

    http://www.mibsoftware.com/userkt/usefor/usefor-usefor-04/Message-ID.htm

But since the draft allows CFWS in References, it apparently isn't
a concern that comments appear there.  (I disagree.)

I don't know what you mean by "worse."

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Fri, 3 Jun 2005 19:38:41 +0200
Organization: Chaos rules.
Sender: [email-masked]

Forrest J. Cavalier III:
> Bruce Lilly:

>> I see no error in the relevant sections of RFC 2822.

> This is in RFC2822, which I copied...
> 
>     The "References:" field will contain the contents of the parent's
>     "References:" field (if any) followed by the contents of the
>     parent's "Message-ID:" field (if any).
> 
> and one of your criticisms was that "contents of" was nebulous, and
> you provided an example.
> 
> Are you saying the phrase "contents of" is not nebulous in RFC2822,
> but would be nebulous in USEFOR?  Can you explain?

The contents of the parent's "Message-ID:"-field will sometimes 
contain more than just a <msg-id>, like trailing whitespace, or 
a comment, or worse.

-- 
Grtz, Ruud

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Injection-Info issues
Date:  Fri, 03 Jun 2005 19:28:58 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 146
Cc: wayne <[email-masked]>
Sender: [email-masked]

Charles Lindsey wrote in USEFOR:
 
>> Is anything wrong with copying RfC 3834 opt-parameter-list ?
>> I'd be very interested if that's not more state of the art.
 
> That would cope fine with <ext-param>, but it does not help
> to show the full syntax of, say, the <post-host-param>

You could also use the trick in the most recent SPF drafts -
not as good as an RfC, but the ABNF passed Bill's checker:

| header           = "Received-SPF:" [CFWS] result FWS [comment FWS]
|                    [ key-value-list ] CRLF
[...]
| key-value-list   = key-value-pair *( ";" [CFWS] key-value-pair )
|                    [";"]
|
| key-value-pair   = key [CFWS] "=" ( dot-atom / quoted-string )

| key              = "client-ip" / "envelope-from" / "helo" /
|                    "problem" / "receiver" / "identity" /
|                    mechanism / "x-" name / name
[.,.]
| dot-atom         = <unquoted word as per [RFC2822]>
| quoted-string    = <quoted string as per [RFC2822]>
| comment          = <comment string as per [RFC2822]>
| CFWS             = <comment or folding white space as per [RFC2822]>
| FWS              = <folding white space as per [RFC2822]>
| CRLF             = <standard end-of-line token as per [RFC2822]>

Note the ... / "x-" name / name
Argh, but stay away from "CRLF as in 2822", of course 2234bis.

Continued later, I've to Cc: this part to the SPF editor, bye.

 (it needs to be clear when
> <quoted-string>s are needed, and that all sorts of RFC 2231 nonsense
> can/may be used). The problem with RFC 3834 is that it defines no
> parameters of its own, and thus sets no example of how to do so.
> 
> >> 1. We now have <ext-param>s, which allow for further
> >>   parameters in extensions. There is also a query:
> >>    [[Should also allow for x-attributes?]]
> 
> >RfC 2231 <attribute-char> allows "x" and "-" among others.
> >Why reserve "x-" if there's no IANA registry ?  RfC 3834
> >has not a single additional word about opt-parameter-list.
> 
> RFC 2045 makes a special case for <x-token>s, and we want to buy into
> that. But by the time we get to RFC 2231, <token>s have morphed into
> <attribute>s.
> 
> >>| allow <parameter>s with x-attributes
> 
> >Is x-attributes a defined term ?  I know what you mean, but we
> >would get "[x] draft should try to avoid hacker slang" for it.
> 
> Yes, ir should be defined (RFC 2231 should have done it, but didn't).
> 
> >> a) folding SHOULD NOT happen in silly places
> 
> >Let's better forget it.
> 
> Well, it's in the present text of USEFOR.
> 
> >  An article format designed to simplify
> >"killfiling" can't be a good idea for USEFOR.  Maybe put it in
> >USEAGE.  Some uses of sender= or posting-account= are probably
> >against the law in my country, and some legal uses won't allow
> >"killfiling".
> 
> Whatever is wrong with designing formats that simplify the things that
> people are going to use them for?
> 
> The Injection-Info provides a variety of paramaters, if some of them are
> illegal in your country, then you will just have to use one of the others.
> 
> >> b) if several parameters are present, they SHOULD come in a
> >>    prescribed order
> 
> >See (a), it's not worth it.
> 
> It _could_ be a USEAGE matter, but it wasn't considered so in draft-13.
> 
> >> c) if a <post-account-param> is used, a given sender SHOULD
> >>    always show up as the same posting-account, at least in
> >>    the short term.
> 
> >Dito, that's a legal nightmare, let's forget it, or discuss
> >it with some legal advisors and privacy officers - IANAL.
> 
> But that one definitely belongs here. It the "posting-account" parameter
> runs into legal difficulties, then use a "logging-data" parameter which is
> designed to be inscrutable without the cooperation of the ISP. But that is
> just its problem. It is well known that spammers and other Bad Guys are
> attracted to "uncooperative" ISPs (ones where abuse@ is diverted to
> /dev/null, and which never make any attempt to discipline their
> customers). The whole purpose of the "posting-account" parameter is to
> identify the same source of articles when it turns up again (even if you
> don't get to know the guy's real name or address).
> 
> We went through all these arguments when we designed the Injection-Info
> header way back (see discussion in USEAGE 4.1.4). There were tensions
> between those who wanted to ensure posters' privacy and those who wanted
> to ensure that spammers could be traced. You cannot achieve both of those
> ideals together. So the standard provides a variety of tools (some more
> private than others), and it is left to the marketplace (perhaps with soe
> guidance from USEAGE) to decide which of them get used.
> 
> >> we can still discourage its unnecessary use
> 
> >It does this by itself, we don't need to worry about this:
> 
> >| IMPORTANT NOTE:  These mechanisms end up being somewhat
> >| gibbous when they actually are used. As such, these
> >| mechanisms should not be used lightly; they should be
> >| reserved for situations where a real need for them exists.
> 
> Yes, but that is similar to my wording. My wording deliberately pinched
> some text from RFC 2231 itself, in order to show that even RFC 2231 was
> deprecating itself :-) .
> 
> >What we really need are some security considerations about the
> >MIME boundary parameter.  While we're at it we could also add
> >a note about the known message/partial risks, it's essentially
> >the same issue -  IIRC with a registered "MIME vulnerability"
> >number for the latter, we could even add it to the informative
> >references, if that's allowed and good style (?)
> 
> Please be more specific which risks/vulnerabilities you are talking about.
> 
> --
> Charles H. Lindsey ---------At Home, doing my own thing------------------------
> Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
> Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
> PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Fri, 3 Jun 2005 13:24:04 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

John Stanley <[email-masked]> wrote:
> Seth Breidbart <sethb@xxxxxxxxx>:

>>Exactly as before: thread it as if the article were a followup.
>>That's the reason for inserting the References header.

> Sorry, but that cannot be the reason. Display of articles is outside
> the scope of our standard, and of RFC1036, so "please display it
> this way" cannot be the function of any header defined therein.

"Thread it" isn't necessarily a display issue.  But if you prefer,
change my language to "Treat this article as if it were a followup."

> The only reason for inserting the References header is to say "this
> is a followup to article X".

That makes the assumption that "followup" iff "contains References
header".  We voted against that assumption.  Deal with it.

> Now, MAYBE the poster is trying to say "I call this a followup
> because...", but we cannot divine that internal discussion from the
> appearance of the article.

But based on both the vote here and reality, the poster is _not_
calling the article a followup, he's just indicating that it would be
useful to treat it as one.

If that means you can't distinguish between "this article is a
followup" and "this article ought to be treated as a followup" then so
what?  Since you ought to treat them the same anyway, of what use
would being able to make that distinction be?

> All we know is that he DID call it a followup for some reason
> because it contains a header that is allowed only in followups.

We voted otherwise.

Seth

Date: Fri, 3 Jun 2005 13:03:47 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

On Fri, 3 Jun 2005, Charles Lindsey wrote:
> Yes, that says the right things, but I think the text can be tidied up a
> bit. How about:
> 
>    A "followup" is an article containing a response to the contents of an
>    earlier article, its "precursor". Every followup includes a References
>    header identifying that precursor (but note that non-followup articles
>    may also use a References header).

Looks good to me.

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date:  Fri, 03 Jun 2005 18:50:01 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Charles Lindsey wrote:

> Yes, that says the right things

+1

> How about:

Also fine, maybe better.

 [3.2.x propossal]
>| in order to facilitate the display or retrieval, by reading
>| and other agents, of threads of related articles.

It's not only for "threads of", it can be anything the poster
wants as a "relation" for "display or rertrieval", it can be
also a "relation" unknown for most readers (e.g. follwup in a
different TLH, or article posted as reply to a mail in a list)

                          Bye, Frank

Date: Fri, 3 Jun 2005 09:57:41 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

Seth Breidbart <sethb@xxxxxxxxx>:

>Exactly as before: thread it as if the article were a followup.
>That's the reason for inserting the References header.

Sorry, but that cannot be the reason. Display of articles is outside the 
scope of our standard, and of RFC1036, so "please display it this way" 
cannot be the function of any header defined therein.  

The only reason for inserting the References header is to say "this is a
followup to article X". It has no other defined meaning. If the poster
includes a References header, which it otherwise prohibited, it can only
be to indicate "this is a followup". It cannot be "display it with X". 

Now, MAYBE the poster is trying to say "I call this a followup
because...", but we cannot divine that internal discussion from the
appearance of the article. All we know is that he DID call it a followup
for some reason because it contains a header that is allowed only in
followups.

Date: Fri, 3 Jun 2005 09:42:40 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>I have deliberately not discussed USEPRO. I believe that we can do one 
>thing at a time.

You are wrong. You must consider the effects of USEFOR on USEPRO and vice 
versa. We must discuss them together, because they are intended to be 
implemented together.

>...people are currently using the References: header in messages that can 
>only be called "followup" by a rather stretched definition of that term,

Here's how far it has to be stretched: add "or as a result of". Not very 
far, and it is exactly how people are treating it today. It is current 
practice. Why is documenting current practice suddenly "rather stretched"?

>I disagree with your statement about RFC 2822.

You are free to disagree, but only one of us can be right. Do you have a
section of RFC2822 that says that References is EVER mandatory in a
message? I don't think so. I have one that says it is optional in replies,
and I have a section of USEFOR that says we defer to RFC2822 for
References.

>I'll let current practitioners speak for themselves.

I'm happy to observe current practice and discuss it. If current 
practitioners (other than ourselves, who can also be current 
practitioners) don't want to come here, they don't have to.

>The others can speak for themselves, too.

Yes, and they are. But RFCs do not have a voice of their own, so someone 
has to speak up for them. 

Date: Fri, 3 Jun 2005 12:34:45 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: [email-masked]
Subject: Re: Path header - suggested text
Sender: [email-masked]

On Fri, 3 Jun 2005, Bruce Lilly wrote:
> Been there, discussed that.  1036 describes "version B2.11 of the News
> program".

Been *there*, discussed *that*.  No, it doesn't.  Possible intent
notwithstanding, there are many discrepancies between the two, enough that
the B2.11 code is not useful as a model implementation of 1036. 

We used both -- rather cautiously -- as references when we wrote C News. 
They not infrequently disagree. 

Note also that 1036 does not claim to "describe" B2.11.  Rather, it says
it is an update of 850, "reflecting" B2.11 -- that is, that changes have
been made to 850 based on things done in B2.11.  That doesn't imply that
1036 and B2.11 are even intended to match exactly.

>Underscore was not a Path field delimiter as documented by "punctuation"
>in RFC 1036.

1036 never says exactly what "punctuation" is supposed to mean for its
purposes, although it does say that the characters "considered part of
host names" are "letters, digits, periods and hyphens". 

                                                          Henry Spencer
                                                       [email-masked]

Date: Fri, 03 Jun 2005 12:25:38 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

Bruce Lilly wrote:

> I see no error in the relevant sections of RFC 2822.
> 

This is in RFC2822, which I copied...

    The "References:" field will contain the contents of the parent's
    "References:" field (if any) followed by the contents of the parent's
    "Message-ID:" field (if any).

and one of your criticisms was that "contents of" was nebulous, and
you provided an example.

Are you saying the phrase "contents of" is not nebulous in RFC2822, but
would be nebulous in USEFOR?  Can you explain?

Date: Fri, 03 Jun 2005 12:19:17 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
CC: "Forrest J. Cavalier III" <[email-masked]>
Subject: Re: You critiqued USEPRO....Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

Forrest J. Cavalier III wrote:

> Private email.

Ah, I guess not private.  No big deal.

As you can see I thougth private would avoid extra inconvenience
to Bruce and the list when pointing out that some of the
harsh criticism Bruce had was actually against text
posted from the current drafts.

I think it is clear he did not realize it when he
wrote.  Actually, I think that had an extremely valuable
result....a fresh criticism of the wording in the
current USEPRO draft.

I didn't need to send that publicly, unless other people
also didn't realize where the "implementation" text
from my posting came from.

I guess I assumed that everyone participating here can
recognize text when it comes from RFC2822 and our
drafts.  Wrong assumption.

I admit to not knowing the drafts as well as I could,
but I try to be familiar with the parts as they are
discussed before I comment.  I split the documents into
sections on mibsoftware.com to facilite that.

I have made my opinions on the general state of
the drafts known to every chair over the years, and
I have started being more public with those comments
too.  So that part of my intended private email is
not a revelation.

To: [email-masked]
Xref: clerew local.usefor:21035
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Info issues
Date: Fri, 3 Jun 2005 14:47:34 GMT
Lines: 119
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

> [CFWS issues for parameters]
>> I don't want to discuss these here, because they are better
>> resolved offline.

>Is anything wrong with copying RfC 3834 opt-parameter-list ?
>I'd be very interested if that's not more state of the art.

That would cope fine with <ext-param>, but it does not help to show the
full syntax of, say, the <post-host-param> (it needs to be clear when
<quoted-string>s are needed, and that all sorts of RFC 2231 nonsense
can/may be used). The problem with RFC 3834 is that it defines no
parameters of its own, and thus sets no example of how to do so.

>> 1. We now have <ext-param>s, which allow for further
>>   parameters in extensions. There is also a query:
>>    [[Should also allow for x-attributes?]]

>RfC 2231 <attribute-char> allows "x" and "-" among others.
>Why reserve "x-" if there's no IANA registry ?  RfC 3834
>has not a single additional word about opt-parameter-list.

RFC 2045 makes a special case for <x-token>s, and we want to buy into
that. But by the time we get to RFC 2231, <token>s have morphed into
<attribute>s.

>>| allow <parameter>s with x-attributes

>Is x-attributes a defined term ?  I know what you mean, but we
>would get "[x] draft should try to avoid hacker slang" for it.

Yes, ir should be defined (RFC 2231 should have done it, but didn't).

>> a) folding SHOULD NOT happen in silly places

>Let's better forget it.

Well, it's in the present text of USEFOR.

>  An article format designed to simplify
>"killfiling" can't be a good idea for USEFOR.  Maybe put it in
>USEAGE.  Some uses of sender= or posting-account= are probably
>against the law in my country, and some legal uses won't allow
>"killfiling".

Whatever is wrong with designing formats that simplify the things that
people are going to use them for?

The Injection-Info provides a variety of paramaters, if some of them are
illegal in your country, then you will just have to use one of the others.

>> b) if several parameters are present, they SHOULD come in a
>>    prescribed order

>See (a), it's not worth it.

It _could_ be a USEAGE matter, but it wasn't considered so in draft-13.

>> c) if a <post-account-param> is used, a given sender SHOULD
>>    always show up as the same posting-account, at least in
>>    the short term.

>Dito, that's a legal nightmare, let's forget it, or discuss
>it with some legal advisors and privacy officers - IANAL.

But that one definitely belongs here. It the "posting-account" parameter
runs into legal difficulties, then use a "logging-data" parameter which is
designed to be inscrutable without the cooperation of the ISP. But that is
just its problem. It is well known that spammers and other Bad Guys are
attracted to "uncooperative" ISPs (ones where abuse@ is diverted to
/dev/null, and which never make any attempt to discipline their
customers). The whole purpose of the "posting-account" parameter is to
identify the same source of articles when it turns up again (even if you
don't get to know the guy's real name or address).

We went through all these arguments when we designed the Injection-Info
header way back (see discussion in USEAGE 4.1.4). There were tensions
between those who wanted to ensure posters' privacy and those who wanted
to ensure that spammers could be traced. You cannot achieve both of those
ideals together. So the standard provides a variety of tools (some more
private than others), and it is left to the marketplace (perhaps with soe
guidance from USEAGE) to decide which of them get used.

>> we can still discourage its unnecessary use

>It does this by itself, we don't need to worry about this:

>| IMPORTANT NOTE:  These mechanisms end up being somewhat
>| gibbous when they actually are used. As such, these
>| mechanisms should not be used lightly; they should be
>| reserved for situations where a real need for them exists.

Yes, but that is similar to my wording. My wording deliberately pinched
some text from RFC 2231 itself, in order to show that even RFC 2231 was
deprecating itself :-) .

>What we really need are some security considerations about the
>MIME boundary parameter.  While we're at it we could also add
>a note about the known message/partial risks, it's essentially
>the same issue -  IIRC with a registered "MIME vulnerability"
>number for the latter, we could even add it to the informative
>references, if that's allowed and good style (?)

Please be more specific which risks/vulnerabilities you are talking about.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21036
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Fri, 3 Jun 2005 15:01:37 GMT
Lines: 45
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>2) All "followups" have a References: header.
>   There exist articles that have References:, but are not "followups"

>1) To section 1.5:

>NEW:

>   A "followup" is an article containing a response to the contents of
>   an earlier article. (the followup's "precursor" indicated in the
>   "References" header).
>   All "followups" have a "References" header.
>   Note: Other articles may also use the "References" header.

Yes, that says the right things, but I think the text can be tidied up a
bit. How about:

   A "followup" is an article containing a response to the contents of an
   earlier article, its "precursor". Every followup includes a References
   header identifying that precursor (but note that non-followup articles
   may also use a References header).

And note that either text is consistent with the currently proposed
introduction to the References header:

3.2.1  References

|  The References header is used to refer to earlier articles (e.g.
|  followups refer to their precursors), in order to facilitate the
|  display or retrieval, by reading and other agents, of threads of
|  related articles. It is the same as that specified in Section 3.6.4
|  of [RFC2822] with the added restrictions detailed in Section 2.2 and
|  those listed below:

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21037
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Fri, 3 Jun 2005 15:07:45 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> "Ruud H.G. van Tol" <[email-masked]> writes:

>So now I have to share this story on something that happened about an
>hour ago:

>So what he did was follow-up to an earlier posting and replacing the
>body as if he was reacting on a posting that he still had to received
>from his ISP's newsserver. He wrote that he did that to be closer
>involved in the discussions. He obviously didn't manually update the
>"References" header, so his reactions popped-up in all the wrong places
>on people's screens.

>Luckily the discussions were in the nl-hierarchy, and he could be
>persuaded to start using the good & free news://fb1.euro.net

Which is a perfect example of how Usenet governs itself. If someone steps
out of line, there is always someone in each newsgroup to bring him back
into the fold.

But sadly, that mechanism does not work for email. Which is why
innovations spread rather rapidly through Usenet (though not necessarily
all useful ones, which is why we have this WG to tidy up the mess).
Whereas in Email the standards people can beaver away for years and still
people do not adopt the new things :-( .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Info issues
Date: Fri, 3 Jun 2005 12:12:52 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 3 2005 09:40, Charles Lindsey wrote:
> 
> In <[email-masked]> Russ Allbery <[email-masked]> writes:

> >I think you mean authentication identity; authorization identity is the
> >thing that we had before.  The authentication identity may, in fact, may
> >often look like a <mailbox> but not actually be one in any traditional
> >sense.
> 
> Whichever. I find the distinction made by the SASL standards far from
> clear.

See FYI 36.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Forrest J. Cavalier III" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Fri, 3 Jun 2005 12:09:36 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 3 2005 11:40, Forrest J. Cavalier III wrote:
> 
> Bruce Lilly wrote:
> 
> > If that's supposed to be a shining example of technical writing, we're in
> > deep trouble:
> > o terminology is inconsistent. Some of the terminology is OK; other uses
> >   are inconsistent with definitions in normative references (esp. RFC 2822)
> >   and in standard glossaries such as FYI 18.
> 
> Yes we are in deep trouble, especially since I copied most of what you objected
> to verbatim from RFC2822.
> 
> Take a look...
> 
>     http://www.mibsoftware.com/userkt/usefor/rfc2822/Identification_fields.htm

I see:
  "Message-ID:" field
  "In-Reply-To:" and "References:" fields
  "Message-ID:" field
  "References:" and "In-Reply-To:" field
  "Message-ID:" field
  header fields
  trace fields
  resent fields
etc., and the definition (consistent with FYI 18):
   A message consists of header fields (collectively called "the header
   of the message") followed, optionally, by a body.  The header is a
   sequence of lines of characters with special syntax as defined in
   this standard. The body is simply a sequence of characters that
   follows the header and is separated from the header by an empty line
   (i.e., a line with nothing preceding the CRLF).

> If we can't be consistent with RFC2822, then all is lost indeed.

Agreed.
 
> Thanks for the criticism, shall I make an attempt to improve it,

Your choice.

> and should 
> I submit some errata for RFC2822 based on your comments?

I see no error in the relevant sections of RFC 2822.

Date: Fri, 3 Jun 2005 12:05:21 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Sender: [email-masked]

"Charles Lindsey" <[email-masked]> wrote:
> In <[email-masked]> Seth Breidbart
> <[email-masked]> writes:

> Sure. What I meant was that there might have been a long thread with "Old
> Subject" which morphed into "New Subject". And, after a couple of "News
> Subject" articles somebody might decide it wasn't anything to do with "Old
> Subject" anymore and so, when generating the next followup, he might prune
> all the "Old Subject" articles out of the References, just leaving the two
> "New Subject" articles.
>
> Technically in breach of the draft, but quite sensible nevertheless.

Of course, it would have no effect (on any newsreader I know), unless
the parent and grandparent articles hadn't arrived so the newsreader
didn't know they were part of the old thread.

I thought you meant removing the entire References header, which I
don't think is sensible in an article with (e.g.) 3 levels of quoting.

>>But then, we can't control user behavior anyway.

So this is all moot.

Seth

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>, Frank Ellermann <[email-masked]>
Subject: Re: Path header - suggested text
Date: Fri, 3 Jun 2005 12:00:28 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 3 2005 09:18, Charles Lindsey wrote:
> 
> In <[email-masked]> Frank Ellermann <[email-masked]> writes:

> >  Note, in particular, that RFC 1036 treats "_" as  a  delim-
> >  iter, but in fact it is known to appear in relayer names
> >  occasionally.
> 
> >So "_" is apparently not exactly compatible with RfC 1036.
> 
> It is impossible to tell from RFC 1036 exactly what is a delimiter and
> what is not.

Been there, discussed that.  1036 describes "version B2.11 of the News
program".  We know precisely which characters were delimiters, and that
information has been presented on this WG's mailing list.  See
http://www.imc.org/ietf-usefor/2004/Jun/0566.html
and
http://www.imc.org/ietf-usefor/2004/Jul/0265.html

Underscore was not a Path field delimiter as documented by "punctuation"
in RFC 1036.  That term means the set of characters:
#define NETCHRS �"!:@^%,"/* Punct. chars used for various networks � � � */

Note that that set does include the colon character.

Date: Fri, 03 Jun 2005 11:45:24 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: You critiqued USEPRO....Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

Private email.

Did you realize that most of your criticism actually applied
to text that appears in the current USEPRO-03?  It isn't
mine.
    http://www.mibsoftware.com/userkt/usefor/usefor-usepro-03/Duties_of_a_Followup_Agent.htm

I agree with the criticisms you have.

The trouble is that the ENTIRE USEPRO is like that.

If people actually read the drafts, as you were just prompted
to, there would be as much criticism for every section.

Date: Fri, 03 Jun 2005 11:40:17 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

Bruce Lilly wrote:

> If that's supposed to be a shining example of technical writing, we're in
> deep trouble:
> o terminology is inconsistent. Some of the terminology is OK; other uses
>   are inconsistent with definitions in normative references (esp. RFC 2822)
>   and in standard glossaries such as FYI 18.

Yes we are in deep trouble, especially since I copied most of what you objected
to verbatim from RFC2822.

Take a look...

    http://www.mibsoftware.com/userkt/usefor/rfc2822/Identification_fields.htm

If we can't be consistent with RFC2822, then all is lost indeed.

I do take the blame/responsibility for the wording of the "previously posted context"
definition.

The trimming algorithm came from the current USEPRO draft, but I reworded it.

Thanks for the criticism, shall I make an attempt to improve it, and should
I submit some errata for RFC2822 based on your comments?

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
Date: Fri, 3 Jun 2005 11:31:13 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 3 2005 08:17, Charles Lindsey wrote:

> Actually, that is not quite right, because there is also an exception for
> a part of <obs-zone> in section 3.1.2. 
> 
> You MUST accept "UT" and "GMT".

RFC 2822 already says that. No "exception" here.

> You SHOULD NOT generate "UT" and "GMT".

There has never been any convincing argument that there should be a
deviation from RFC 2822 (and RFC 1123) here.  Ancient implementations
might well be non-conforming to what we specify for requirements on
new generation, and no argument has been presented as to why that
should differ from 2822 (viz. use numeric offsets only for generation).
I.e., while this does represent an "exception", there has been no
justification for such.

> You MUST NOT generate "EST" or any of the other things in <obs-zone>.

Already covered by 2822.

> You MAY accept "EST" or any of the other things in <obs-zone> (because of
> that "MAY" in 2.1).

2822 says (in effect) MUST accept.  Again, no justification for a
difference, particularly as practice is and existing documents specify
acceptance.  Especially, there is no justification for simultaneously
being more restrictive (MUST accept -> MAY accept) for some alphabetic
zones and less restrictive (MUST NOT generate -> SHOULD NOT generate)
for one particular zone (purely coincidental that that happens to be
a Document Editor's zone, and that said Editor's mail to this mailing
list remains non-compliant with 2822 while the vast majority of
messages are compliant... :-).
  
> So that paragraph in 2.1 could more helpfully say:
> 
>  News articles MUST conform to the syntax specified in Section 3 of
>  [RFC2822]. The obsolete syntax specified in Section 4 of [RFC2822] is
>  excluded (except for some special use of <obs-zone> in section 3.1.2
>  below [and any other exceptions we choose to make ....]); Netnews agents
>  MAY accept that obsolete syntax but MUST NOT generate productions of it.

Making it longer and more convoluted isn't "helpful".  Eliding it in
favor of "[RFC2822]" would be helpful.  A note explaining that legacy
UAs might be non-compliant with *this* specification could be argued
as either being a clarification or as being redundant.  My opinion is
the latter; at most a statement to the effect that non-conforming
implementations are non-conforming should be relegated to an Appendix.
"At most". 

> What I would like to see is <obs-phrase> (possibly renamed as
> <extended-phrase>) treated as a special case with similar status to "UT"
> and "GMT".

As noted above, there seems to be no justification for the deviations;
it appears to be a personal prejudice.

> I.e. MUST accept it, but SHOULD/MUST/whatever NOT generate it. 
> That aligns us with the way RFC 2822 is leading us. I would prefer SHOULD
> NOT to MUST NOT generate, but that is a secondary issue.

SHOULD NOT implies that generation is officially allowed, and would
adversely affect backward compatibility unless you can conclusively
show that no deployed parsers will experience difficulties (note that
the 2822 change was specifically to address such observed difficulties).
Lacking such a detailed analysis and in the absence of a convincing
argument in favor of a deviation, again "[RFC2822]" should suffice.

> >My instinct is "don't mess with other standards" and leave it as it is. Is 
> >someone arguing strongly that we should change it?
> 
> I am trying not to "mess". Just to adopt a piece if RFC 2822 that we have
> not adopted so far.

Then what precisely are the justifications for the deviations (BCP 14
keyword changes in date-time related issues and in your suggested
MUST NOT -> SHOULD NOT for obs-phrase)?

To: [email-masked]
Subject: Re: Path header - suggested text
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 03 Jun 2005 08:11:48 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Frank Ellermann <[email-masked]> writes:

>> Sorry, now that Bruce is back I could stop this, but...
>> 
>>> | 1. A fully qualified domain name (FQDN) associated with an
>>> |    A record, which SHOULD identify the actual machine
>> 
>> ...we need "or AAAA", don't we ?

> Eh? What is "AAAA"?

The IPv6 equivalent of an A record.

To: [email-masked]
Subject: Re: Injection-Info issues
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 03 Jun 2005 08:08:43 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> I think you mean authentication identity; authorization identity is the
>> thing that we had before.  The authentication identity may, in fact,
>> may often look like a <mailbox> but not actually be one in any
>> traditional sense.

> Whichever. I find the distinction made by the SASL standards far from
> clear. Is a man allowed to choose any string he likes as his
> "authorization identity", or does it have to be something previously
> agreed with the server?

It's fairly clear if you understand the distinction between authentication
and authorization.  Authentication says who you are.  Authorization says
what you're allowed to do.  The authorization identity is the identity
you're assuming on the server and with which ACLs are associated -- it may
be, for instance, a group account shared by several people, or a role in
which you're acting.  The authentication identity is who you actually are,
namely some identified user who has access to that role.  Depending on the
authentication mechanism, the two identities may live in different
namespaces.

So, for instance, if I authenticate to my local news server with the
KERBEROS_V4 SASL mechanism, my authorization identity (the user string in
readers.conf that I'm assigned) is "[email-masked]", but my
authentication identity is "[email-masked]", which is a principal in
our Kerberos v4 realm.

Your authorization identity is generally going to be the thing that goes
into sender and the like.  Authentication identity is an additional trace
showing who was actually using a shared account or role that in many cases
will just be identical to the authorization identity.

This is fairly standard stuff in the security community, but Usenet
protocols have not, up until recently, really dealt with security in a
fashion consistent with the rest of the world.

> OK, just call it "auth":

>    Injection-Info: dodgy.net; posting-host=dialup-123.dodgy.net;
>                    auth="some-string"

It's generally wise to never use the term "auth" in a security context
because it's ambiguous on whether you're talking about the authentication
identity or the authorization identity.  You need at least "authn" or
"authz", and it's probably better to just spell it out.

> Or else use it in a "posting-account" parameter.

That would be the authorization identity, not the authentication identity.

> I just want to make sure that such authentication-identities can be
> used, and that there exists a parameter clearly suitable for the
> purpose.

Yup.

Date: Fri, 3 Jun 2005 11:04:19 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Path header - suggested text
Sender: [email-masked]

On Fri, 3 Jun 2005, Charles Lindsey wrote:
> >>| 1. A fully qualified domain name (FQDN) associated with an
> >>|    A record, which SHOULD identify the actual machine
> >...we need "or AAAA", don't we ?
> 
> Eh? What is "AAAA"?

An AAAA record is the IPv6 equivalent of an A record.  IPv6 addresses are
four times as long as IPv4 addresses, so the name seemed obvious... :-)

                                                          Henry Spencer
                                                       [email-masked]

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: "Forrest J. Cavalier III" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Fri, 3 Jun 2005 10:56:30 -0400
Cc: [email-masked]
Sender: [email-masked]

On Fri June 3 2005 02:50, Forrest J. Cavalier III wrote:
> 
> Harald Tveit Alvestrand wrote:
> 
> > I have deliberately not discussed USEPRO. I believe that we can do one 
> > thing at a time.
> 
> You cannot change a definition in USEFOR and ignore that it requires
> a change in USEPRO.

"one thing at a time" != "ignore".

> > I believe the other participants claim that this is a hole that is 
> > currently open, and should not be closed - people are currently using 
> > the References: header in messages that can only be called "followup" by 
> > a rather stretched definition of that term, and should not be prevented 
> > from doing so.

This was discussed at length when the change was first made without WG
consensus or prior discussion; it was going on a year ago:
http://www.imc.org/ietf-usefor/2004/Jun/0130.html
That the same argument over the same unrequested change which is 
inconsistent with RFCs 850 and 1036 is still going on today is an
indication that the WG is dysfunctional.

The simple fact is that RFCs 850, 1036 and drafts prior to that change
used "followup" as shorthand for "an article containing a References
field".  And as noted long ago in the ensuing discussion, that shorthand
is objective; it does not require guesses regarding the intent of a
human user, it does not require knowing how the article was prepared or
submitted, and it does not require examination of large numbers of
articles -- it concisely describes a fact about a particular article
which can be objectively assessed by examining that article in isolation.

> So, instead of making an attempt at specifying the conditions that
> it is acceptable, we have to allow it without limitation?
> 
> That's not current practice.  It is obvious that more than limited
> use outside of followups would significantly degrade existing practice,
> and that means it is outside the charter.

The current (RFC 1036) definition of "followup" includes current --
indeed long-standing -- practices such as multi-part FAQs and other
multi-part articles.
 
> This isn't a hard concept for anyone familiar with writing software
> specifications that separate requirements, design, and implementation.
> 
> Here is a specification of "semantics" (my proposed References definition
> for USEFOR, with some RFC2822 text, and a correct trimming algorithm,
> unlike the ambiguity currently in USEPRO.)
> 
>     A References header indicates the previously posted context in which
>     the article was created.
> 
>     The "References:" field will contain the contents of the parent's
>     "References:" field (if any) followed by the contents of the parent's
>     "Message-ID:" field.

If that's supposed to be a shining example of technical writing, we're in
deep trouble:
o terminology is inconsistent. Some of the terminology is OK; other uses
  are inconsistent with definitions in normative references (esp. RFC 2822)
  and in standard glossaries such as FYI 18.
o "previously posted context" is just plain wrong; it does not account for
  multi-part FAQs which may be prepared together (including References fields)
  and posted (nearly) simultaneously.  Likewise for other fragmented messages.
o "the contents of" is at best nebulous; at worst can lead to interoperability
  issues:
    References: <[email-masked]> Message-ID: <[email-masked]>
  is "the contents of" a References field followed by "the contents of" a
  Message-ID field.  We have suitable terminology, viz. "field body", and
  more precisely "msg-ids".

>     To prevent excessively long lines, message 
>     identifiers from the parent's References field can be omitted, provided
>     that the first and last one or more in that field are retained.

o "excessively long lines" is a non-issue due to the ability to fold
  physical lines wherever linear whitespace is allowed, as has been the
  case with RFC 822 message format and its predecessors even before that
  format was adopted by RFC 850.
o there is of course no limit to the length of a logical (i.e. logically
  unfolded) field.  Never has been.
 
> Here is a specification of "implementation" (text from USEPRO-03)
> 
>     If the precursor did not have a References header (F-3.2.1), the

o where is your definition of "precursor", precisely how does it differ
  from your use above of "parent", and does it account for use with multi-
  part FAQs and fragmented message parts (RFC 2046 section 5.2.2.2)?

>     content of the followup's References header MUST be inherited from

o why "MUST" -- what damage or interoperability problems (see BCP 14)
  are involved? (N.B. "some user doesn't like it" isn't an
  interoperability issue in the same sense as "limiting retransmissions")
  In particular, if some article has a clearly broken field where either
  the first or last identifier is malformed, why "MUST" that error be
  propagated? 
o see above re. "content of"
o where is your definition of "inherited"?  What precisely are the rules
  for ``inheritance'' -- may comments be elided?  Physical lines refolded?
  Etc.

>     that of the Message-ID header of the precursor. A followup to an
>     article which already had a References header MUST have a
>     References header comprising the precursor's References header

o you are presumably trying to define "followup" but have not yet done
  so.  The current definition means "article with a References field", so
  the text above is at least partially redundant (of course a followup
  must (N.B. lower case, indicating a logical relationship per section 2.14
  of ftp://ftp.rfc-editor.org/in-notes/rfc-editor/instructions2authors.txt)
  have a References field).

>     (subject to trimming as described below) followed by CFWS and the
>     Message-ID header content of the precursor.

o see above re "content of"
 
>     If the resulting References header is excessively long, it MAY be

o try to define "excessively long".  Then try to achieve consensus for
  your definition.  This WG long ago abandoned that exercise in
  futility.

>     trimmed, but the first and the last two message identifiers MUST
>     NOT be removed.

o see above re. BCP 14 and use of imperatives.
 
> By specifying semantics properly, it is clear how to implement
> a followup agent, a reading agent, and how to post multi-parts
> and FAQs using the References header.

The text you have supplied is certainly not clear.  It doesn't explicitly
address multi-part articles (FAQs or otherwise) directly, and the
"previously posted" part implies that it may not be used for some types
of multi-part articles.  It is hopelessly vague about "excessively long"
and "content of" and does not address reading agents at all.

> Semantics based is clearer, complete,

In which alternate reality is that either clear or complete, much less
both?

[...]
> there must be some section added for how the references get constructed
> in a non-followup.

By definition, a followup contains a References field and an article with
no References field is not a followup.  "how the references get constructed
in a non-followup" is a non sequitur.  Unless of course you are trying to
redefine "followup" to mean something other than what it has meant since
RFC 850, in which case there are many problems.  If it hurts, don't do it.

Complete text regarding References can be a short as:
  Refer to section 3.6.4 of [RFC2822] and section 5.2.2.2 of [RFC2046].

> And then there must be a section about how to 
> display them.

Why "must be"?  We (the IETF) do not usually specify display aspects of
implementation (when we specify implementation detail at all, as opposed
to specifying format and function); exceptions where display is specified
are typically limited to internationalization and/or localization issues
which do not apply to the References field or to display ordering of
articles or to indications of relationships.

> I think the current drafts are all terribly written because of this.
> They are too long.  They are easy to use for one purpose, but not
> another.

You may be right regarding criticism of current drafts, but frankly your
suggested text (which past history shows would either be ignored or
twisted beyond recognition anyway) hardly seems clear or concise, nor
is delving into implementation choices that do not affect
interoperability (as distinct from product features) appropriate for
Standards Track documents.

To: [email-masked]
Xref: clerew local.usefor:21021
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
Date: Fri, 3 Jun 2005 12:17:19 GMT
Lines: 64
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>I have filed ticket #1022 to cover this issue.

>At the moment, section 2.1 of USEFOR-04 says:

> News articles MUST conform to the syntax specified in Section 3 of
> [RFC2822]. Netnews agents MAY also accept the obsolete syntax
> specified in Section 4 of [RFC2822], but they MUST NOT generate
> productions of such syntax.

Actually, that is not quite right, because there is also an exception for
a part of <obs-zone> in section 3.1.2. 

You MUST accept "UT" and "GMT".
You SHOULD NOT generate "UT" and "GMT".
You MUST NOT generate "EST" or any of the other things in <obs-zone>.
You MAY accept "EST" or any of the other things in <obs-zone> (because of
that "MAY" in 2.1).

So that paragraph in 2.1 could more helpfully say:

 News articles MUST conform to the syntax specified in Section 3 of
 [RFC2822]. The obsolete syntax specified in Section 4 of [RFC2822] is
 excluded (except for some special use of <obs-zone> in section 3.1.2
 below [and any other exceptions we choose to make ....]); Netnews agents
 MAY accept that obsolete syntax but MUST NOT generate productions of it.

>I have failed to find a consistent message that someone is arguing that we 
>should change this. I think Charles is arguing for a change, but I'm not 
>sure   - either whether he really wants to change it, or what particular 
>change he'd propose.

What I would like to see is <obs-phrase> (possibly renamed as
<extended-phrase>) treated as a special case with similar status to "UT"
and "GMT". I.e. MUST accept it, but SHOULD/MUST/whatever NOT generate it.
That aligns us with the way RFC 2822 is leading us. I would prefer SHOULD
NOT to MUST NOT generate, but that is a secondary issue.

Note that this exception was already included, years ago, in our 'article'
series of drafts, so I cannot understand why it was not automatically
carried forward into USEFOR.

2.1 would be the place to say it (see the "....." carefully left in my
syggested text), and there should be a pointer to that paragraph in RFC
2822 to explain the purpose of the exception.

>My instinct is "don't mess with other standards" and leave it as it is. Is 
>someone arguing strongly that we should change it?

I am trying not to "mess". Just to adopt a piece if RFC 2822 that we have
not adopted so far.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21027
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Info issues
Date: Fri, 3 Jun 2005 13:40:19 GMT
Lines: 56
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> 3. There is now a draft (with Ken's name on it) for SASL authentication
>> within NNTP. That may give rise AIUI to an "authorization identity"
>> which may or may not be in the form of a <mailbox>.

>I think you mean authentication identity; authorization identity is the
>thing that we had before.  The authentication identity may, in fact, may
>often look like a <mailbox> but not actually be one in any traditional
>sense.

Whichever. I find the distinction made by the SASL standards far from
clear. Is a man allowed to choose any string he likes as his
"authorization identity", or does it have to be something previously
agreed with the server?

>> An injecting agent might then decide to use this identity in the
>> Injection-Info header. Question: do we need a seperate "SASL-identity"
>> parameter for this, or can we just use the "sender" parameter and, if
>> so, should its syntax be extended to suit?

>If anyone expects sender to be an actual mailbox, using it for the
>authentication identity is a bad idea, since the authentication identity
>may be indistinguishable from a mailbox but not be usable for the same
>purpose.

Yes, my feeling was that the <sender-param> was not the right place for
this.

>SASL-identity is a bad choice since it unnecessarily restricts what the
>field may be used for.  I'd rather use a more generic name.  The field
>should be described as an opaque token that may only have meaning for the
>originating site.

OK, just call it "auth":

   Injection-Info: dodgy.net; posting-host=dialup-123.dodgy.net;
                   auth="some-string"

Or else use it in a "posting-account" parameter.

I just want to make sure that such authentication-identities can be used,
and that there exists a parameter clearly suitable for the purpose.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21023
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Path header - suggested text
Date: Fri, 3 Jun 2005 13:18:09 GMT
Lines: 52
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> This is, I hope, the final form of the Path header.

>Thanks, I found no problems, maybe two and a half nits:

>>| 1. A fully qualified domain name (FQDN) associated with an
>>|    A record, which SHOULD identify the actual machine

>RfC 1034 (3.6) offers s/machine/host/ if it has an A record.
>I stumbled over it while checking that we get no "_" here.

OK, I changed that.

>>| this SHOULD then be followed by CRLF and WSP if it would
>>| otherwise result in a line longer than 79 characters.

>See below.

>I tried to check the magic number 79 in s-o-1036, found nothing,
>but stumbled over this part:

>  Note, in particular, that RFC 1036 treats "_" as  a  delim-
>  iter, but in fact it is known to appear in relayer names
>  occasionally.

>So "_" is apparently not exactly compatible with RfC 1036.

It is impossible to tell from RFC 1036 exactly what is a delimiter and
what is not.

>If the 79 is an almost arbitrary magic number I'd prefer one
>of the known magic numbers (78 in 2822, or 76 in 2047), too
>many magic numbers are confusing.

OK, I have changed it into "... result in a line of excessive length."
(and the same under Relaying Agents).  Anybody who wants to know what
"excessive" means can go look in USEAGE.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21022
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Date: Fri, 3 Jun 2005 13:03:31 GMT
Lines: 28
Sender: [email-masked]

In <[email-masked]> Seth Breidbart <[email-masked]> writes:

>If the connection between his article and its predecessor were as
>tenuous as "That reminds me of an interesting story . . ." that would
>be fine.  Otherwise, he's breaking the connection between his article
>and its parent, which is likely not such a good idea.

Sure. What I meant was that there might have been a long thread with "Old
Subject" which morphed into "New Subject". And, after a couple of "News
Subject" articles somebody might decide it wasn't anything to do with "Old
Subject" anymore and so, when generating the next followup, he might prune
all the "Old Subject" articles out of the References, just leaving the two
"New Subject" articles.

Technically in breach of the draft, but quite sensible nevertheless.

>But then, we can't control user behavior anyway.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21024
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Path header - suggested text
Date: Fri, 3 Jun 2005 13:19:14 GMT
Lines: 21
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Sorry, now that Bruce is back I could stop this, but...
> 
>>| 1. A fully qualified domain name (FQDN) associated with an
>>|    A record, which SHOULD identify the actual machine
> 
>...we need "or AAAA", don't we ?

Eh? What is "AAAA"?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21026
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: IMAP definitions on threading
Date: Fri, 3 Jun 2005 13:30:04 GMT
Lines: 27
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>Perhaps the problem is that you think of threads as "lumps of articles"  
>and not the more realistic "group of articles with a defined hierarchical
>relationship"? And that lumping articles together by similar subject is
>"close enough", when you don't care what the hierarchical relationship is?
>I think current usage of thread is certainly not just "lump", but how
>articles are "strung together" -- and that's why "thread" is used instead
>of "lump". But even for the "lump", the Subject header just doesn't
>contain the necessary information, so even "lumping" by Subject isn't
>really possible.

I move that all occurrences of the word "threading" in our drafts be
replaced with the word "lumping".

:-)

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21025
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Path header - suggested text
Date: Fri, 3 Jun 2005 13:28:13 GMT
Lines: 31
Sender: [email-masked]

In <[email-masked]> Harald Tveit Alvestrand <[email-masked]> writes:

>checking..... is the !! stuff a feature that has seen production use 
>somewhere, or is it a new feature that we are adding?

It is a new feature, but carefully crafted so that it causes no harm on
the existing network.

>since it could concievably be constructed as having to do with security, 
>it's a potential sticky point with the security ADs.

It is indeed connected with security. The Bad Guys were in the habit of
preloading their Path headers with a long string culled from some other
article, so as to make it appear that its point of origin was far removed
from themselves. The change will make it much easier to spot such
malpractice and to determine where it originated.

But of course the benefit will only appear when the majority of
news-servers routinely do it, hence various "MUST"s carefully placed in
USEPRO to encourage them to do so.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Fri, 03 Jun 2005 02:50:48 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> I have deliberately not discussed USEPRO. I believe that we can do one 
> thing at a time.

You cannot change a definition in USEFOR and ignore that it requires
a change in USEPRO.  Fragility due to dependency chains is a well-studied
effect.

These are companion documents split only because they were too verbose.
I do not believe they can be edited, issued, or discussed independently.

I do not think it is going to be a "win" to push out USEFOR, and then
see how in the world we can write a USEPRO that requires no changes to
USEFOR.

I don't think it is efficient for the group to consider this problem once
now for USEFOR and then once again later for USEPRO.

> At the moment, USEFOR contains very little information about how to 
> interpret ANY header. Are you suggesting to change this?

I wrote "how to interpret," but I did not mean "how to display."

USEFOR is supposed to be a syntax and semantics document.

More below....

> 
> I believe the other participants claim that this is a hole that is 
> currently open, and should not be closed - people are currently using 
> the References: header in messages that can only be called "followup" by 
> a rather stretched definition of that term, and should not be prevented 
> from doing so.

So, instead of making an attempt at specifying the conditions that
it is acceptable, we have to allow it without limitation?

That's not current practice.  It is obvious that more than limited
use outside of followups would significantly degrade existing practice,
and that means it is outside the charter.

One of us has a very wrong idea about how useful threading (using
References) is.

> 
> But changing USEFOR to contain this level of information would change 
> the split between USEFOR and USEPRO, and I don't see a good reason for 
> that.

I think that's because you confuse semantics and implementation.

This isn't a hard concept for anyone familiar with writing software
specifications that separate requirements, design, and implementation.

Here is a specification of "semantics" (my proposed References definition
for USEFOR, with some RFC2822 text, and a correct trimming algorithm,
unlike the ambiguity currently in USEPRO.)

    A References header indicates the previously posted context in which
    the article was created.

    The "References:" field will contain the contents of the parent's
    "References:" field (if any) followed by the contents of the parent's
    "Message-ID:" field.  To prevent excessively long lines, message
    identifiers from the parent's References field can be omitted, provided
    that the first and last one or more in that field are retained.

Here is a specification of "implementation" (text from USEPRO-03)

    If the precursor did not have a References header (F-3.2.1), the
    content of the followup's References header MUST be inherited from
    that of the Message-ID header of the precursor. A followup to an
    article which already had a References header MUST have a
    References header comprising the precursor's References header
    (subject to trimming as described below) followed by CFWS and the
    Message-ID header content of the precursor.

    If the resulting References header is excessively long, it MAY be
    trimmed, but the first and the last two message identifiers MUST
    NOT be removed.

By specifying semantics properly, it is clear how to implement
a followup agent, a reading agent, and how to post multi-parts
and FAQs using the References header.

Semantics based is clearer, complete, and shorter already, but the USEPRO
text I showed here is not complete.  Since that is the section on Followup Agent,
there must be some section added for how the references get constructed
in a non-followup.  And then there must be a section about how to
display them.  What advantage is there to that?

I think the current drafts are all terribly written because of this.
They are too long.  They are easy to use for one purpose, but not
another.

To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 02 Jun 2005 23:00:42 -0700
Sender: [email-masked]

Forrest J Cavalier <[email-masked]> writes:

> 2. The References header in usefor-usepro-03 is mentioned in only these
> sections:

> http://www.mibsoftware.com/userkt/usefor/usefor-usepro-03/Transitional_Arrangements.htm
> http://www.mibsoftware.com/userkt/usefor/usefor-usepro-03/Duties_of_a_Followup_Agent.htm
> http://www.mibsoftware.com/userkt/usefor/usefor-usepro-03/Duties_of_a_Reading_Agent.htm
> http://www.mibsoftware.com/userkt/usefor/usefor-usepro-03/Duties_of_a_Relaying_Agent.htm

> Note that none of those contain language to allow any articles except
> followups to have the header.  The proposed text will standardize that
> articles which are not followups can have the header, but nothing in the
> proposed text or usefor or usepro limits those articles, nor indicates
> how they should be treated.

I think I'm with Harald on this -- while I agree that USEPRO should say
something about how References should be used in that context, USEFOR
shouldn't, and it's USEFOR that we're trying to finish.

Now, it's a separate (and good) question as to whether publishing USEFOR
without USEPRO should even be considered, given the clarifications like
this that USEPRO would carry, but considering USEFOR in isolation, with
its strictly limited scope, I don't think we should say anything more than
what Harald proposed about References.

Well, maybe a note saying that all References should be treated the same
regardless of their origins, to cover the last part of your paragraph
above, but I'd think that would be fairly obvious.

> BTW, CFWS in References is a silly unnecessary complication.

I agree, but not diverging from mail is even more important to me than
having to deal with that.

> So, why are comments going to be allowed in the References header?

Because mail allows them.  At least that's what I recall.

I think mail was wrong to ever allow them in the first place, but it's
several decades too late to make that argument, alas.

Date: Fri, 03 Jun 2005 01:45:47 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

Seth Breidbart wrote:
> That's the existing practice.

I agree, but that's exactly my point....

I get the feeling that people have not read the drafts....

The drafts contain no wording that standardizes the existing practice
that Seth described.  And accepting the proposed change with no
other changes is totally unacceptable.

For those who haven't read the draft, I'll spell it out....

1. The References header in usefor-usefor-04 is mentioned in only these sections:

http://www.mibsoftware.com/userkt/usefor/usefor-usefor-04/Definitions.htm
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-04/References.htm
   (which defers to RFC 2822
       http://mibsoftware.com/userkt/usefor/rfc2822/Identification_fields.htm
   )
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-04/Message-ID.htm
http://www.mibsoftware.com/userkt/usefor/usefor-usefor-04/Obsolete_Headers.htm

2. The References header in usefor-usepro-03 is mentioned in only these
sections:

http://www.mibsoftware.com/userkt/usefor/usefor-usepro-03/Transitional_Arrangements.htm
http://www.mibsoftware.com/userkt/usefor/usefor-usepro-03/Duties_of_a_Followup_Agent.htm
http://www.mibsoftware.com/userkt/usefor/usefor-usepro-03/Duties_of_a_Reading_Agent.htm
http://www.mibsoftware.com/userkt/usefor/usefor-usepro-03/Duties_of_a_Relaying_Agent.htm

Note that none of those contain language to allow any articles except followups to
have the header.  The proposed text will standardize that articles which are not
followups can have the header, but nothing in the proposed text or usefor or usepro
limits those articles, nor indicates how they should be treated.

BTW, CFWS in References is a silly unnecessary complication.  (CFWS in any structured header
is a bad idea,IMNSHO. - I implement article processing software.)

usepro Transitional Arrangements even admits comments in References will break existing software.

Is the idea to make References irrelevant and unreliable, and then we don't
care that it cannot be parsed by legacy software?

Date: Fri, 03 Jun 2005 07:40:13 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: John Stanley <[email-masked]>, [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

--On torsdag, juni 02, 2005 16:42:21 -0700 John Stanley <[email-masked]> 
wrote:

>
>
> Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:
>
>> I am pretty sure that John Stanley disagrees with this.
>
> RFC2822 disagrees with this.

I disagree with your statement about RFC 2822.

I'll let current practitioners speak for themselves.

> It would be nice if you would let me speak for myself, and when you list
> what disagrees with what, you don't pretend that I'm standing alone.

The others can speak for themselves, too. You were the loudest voice, so I 
thought it was polite to say that I had noticed.

Date: Fri, 03 Jun 2005 07:21:01 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: "Forrest J. Cavalier III" <[email-masked]>, [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

--On torsdag, juni 02, 2005 23:37:53 -0400 "Forrest J. Cavalier III" 
<[email-masked]> wrote:

>  or
>> who think that the group has or can reach a consensus that is best
>> represented by a different formulation, please speak up.
>
> John and I raised objections which received no adequate response already.
> Will doing it again have any different result?
>
> My primary objection at this time is that your proposed text change is
> incomplete, because other parts of USEFOR and USEPRO must be changed

I have deliberately not discussed USEPRO. I believe that we can do one 
thing at a time.

>    - to provide guidance on how to interpret a references header in a
>      received article,

At the moment, USEFOR contains very little information about how to 
interpret ANY header. Are you suggesting to change this?

>    - to limit conformant applications to insert References headers only
>      when appropriate.  I believe the draft documents lack text
>      allowing or restricting References in Usefor except for followups.
>      That being the case, your change will open up a huge hole unless
>      it gets plugged other ways.

I believe the other participants claim that this is a hole that is 
currently open, and should not be closed - people are currently using the 
References: header in messages that can only be called "followup" by a 
rather stretched definition of that term, and should not be prevented from 
doing so.

> If you show us what you intend for those changes, I might be able
> to respond.  As it is, your statement "Note: Other articles may also
> use the "References" header." will ensure that "threading on References"
> will be as unreliable as "threading on Subject."  This direction is
> contrary
> to the entire point of a "draft standard," which is to define formats
> and protocols so that you can implement RELIABLE BEHAVIOR.

My personal preference is to have USEPRO (probably) say that it's 
reasonable for newsreaders that see a References: header to make it easy 
for people to find the article referred to by the References: header when 
they are looking at the article containing it, and to find articles with a 
References: header referring to the article they are looking at, and say 
something like "One such mechanism is commonly called "threading"".
(Customarily, IETF standards do NOT specify user interfaces)

I think this description should talk only about how to treat a message with 
a References header, and not attempt to guess whether or not the message is 
a "followup".

But changing USEFOR to contain this level of information would change the 
split between USEFOR and USEPRO, and I don't see a good reason for that.

And - again - at this time, I'm only trying to get USEFOR finished.

                            Harald

Date: Fri, 3 Jun 2005 00:11:41 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

"Forrest J. Cavalier III" <[email-masked]> wrote:

> My primary objection at this time is that your proposed text change is
> incomplete, because other parts of USEFOR and USEPRO must be changed
>
>   - to provide guidance on how to interpret a references header in a
>     received article,

Exactly as before: thread it as if the article were a followup.
That's the reason for inserting the References header.

>   - to limit conformant applications to insert References headers only
>     when appropriate.  I believe the draft documents lack text
>     allowing or restricting References in Usefor except for followups.

Appropriate is defined, roughly, as "the poster intends that this
article be treated as a followup to that one" (or use the term
"threaded" as appropriate).

That's the existing practice.

I agree, it wouldn't be wrong to address those (at least the second,
I'd have to check existing wording to see if there are any changes
needed to accommodate the first).

Seth

Date: Thu, 02 Jun 2005 23:37:53 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> I am pretty sure that John Stanley disagrees with this.

I object too, for rational reasons, which have not been addressed.

Surely, "reasons" trump "opinion"?  Oh, wait, this is Usefor,
where people seriously write you can "thread on Subject."

> If there are other people who violently object to this formulation, 

I don't do "violence", unless a formal objection at Last call counts
as violence.

> or 
> who think that the group has or can reach a consensus that is best 
> represented by a different formulation, please speak up.

John and I raised objections which received no adequate response already.
Will doing it again have any different result?

My primary objection at this time is that your proposed text change is
incomplete, because other parts of USEFOR and USEPRO must be changed

   - to provide guidance on how to interpret a references header in a
     received article,

   - to limit conformant applications to insert References headers only
     when appropriate.  I believe the draft documents lack text
     allowing or restricting References in Usefor except for followups.
     That being the case, your change will open up a huge hole unless
     it gets plugged other ways.

If you show us what you intend for those changes, I might be able
to respond.  As it is, your statement "Note: Other articles may also
use the "References" header." will ensure that "threading on References"
will be as unreliable as "threading on Subject."  This direction is contrary
to the entire point of a "draft standard," which is to define formats
and protocols so that you can implement RELIABLE BEHAVIOR.

Date: Thu, 2 Jun 2005 16:42:21 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

> I am pretty sure that John Stanley disagrees with this.

RFC2822 disagrees with this.

RFC1036 and current practice disagree with this.

The previous documents produced by this group that went through last call
disagree with this.

It would be nice if you would let me speak for myself, and when you list 
what disagrees with what, you don't pretend that I'm standing alone. 

Date: Thu, 2 Jun 2005 16:35:54 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: IMAP definitions on threading
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>Your particular words that I found too strong were:

>>    Threading on Subject does not exist. 

??? Too strong? Please.  

>I pointed to the particular IMAP draft as an example of a person using the 
>word "threading" to describe something very like what you don't want to 
>call "threading".

We already have examples of people calling it "threading". We also have 
examples of why it really isn't threading. 

>Your problem. But your claim that this definition is "the truth" is too 
>strong for my taste.

The solution is simple. Answer either or both of the questions I presented
with the correct threading as determined by the Subject header and 
disprove my reading of the RFC. That you did not (and I maintain you 
cannot) is sufficient to indicate that threading is not possible with the 
Subject header, and thus "threading by subject" does not exist as such. 

I am at a loss as to why people care so much about not being correct to
call their sorting algorithm "threading", when the final result is exactly
the same. The only difference is that they don't mislead naive users into
thinking their presentation order has any physical reality behind it when 
they admit they just sort by subject, and they do mislead them when they
claim they can determine threading by subject.

Perhaps the problem is that you think of threads as "lumps of articles"  
and not the more realistic "group of articles with a defined hierarchical
relationship"? And that lumping articles together by similar subject is
"close enough", when you don't care what the hierarchical relationship is?
I think current usage of thread is certainly not just "lump", but how
articles are "strung together" -- and that's why "thread" is used instead
of "lump". But even for the "lump", the Subject header just doesn't
contain the necessary information, so even "lumping" by Subject isn't
really possible.

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Subject: Re: #1008 USEFOR 1.5: Suggested language on "followup"
Date: Fri, 3 Jun 2005 00:11:40 +0200
Organization: Chaos rules.
Sender: [email-masked]

Harald Tveit Alvestrand:

> NEW:
>    A "followup" is an article containing a response to the contents of
>    an earlier article. (the followup's "precursor" indicated in the
>    "References" header).
>    All "followups" have a "References" header.
>    Note: Other articles may also use the "References" header.

I can live with that, although I will still call articles "followup"
that have no "References" header but that are obviously reactions on an
earlier article.

So now I have to share this story on something that happened about an
hour ago:

Someone that posts through his ISP's newsserver (news.chello.nl)
suddenly started reacting on older articles but with quoted text from
newer articles, because he could find the newer articles 90 minutes
earlier on google.

So what he did was follow-up to an earlier posting and replacing the
body as if he was reacting on a posting that he still had to received
from his ISP's newsserver. He wrote that he did that to be closer
involved in the discussions. He obviously didn't manually update the
"References" header, so his reactions popped-up in all the wrong places
on people's screens.

Luckily the discussions were in the nl-hierarchy, and he could be
persuaded to start using the good & free news://fb1.euro.net

-- 
Grtz, Ruud

Date: Thu, 02 Jun 2005 23:52:17 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1022: USEFOR 2.1 obs-phrase: Should dot be permitted in phrases?
Sender: [email-masked]

I have filed ticket #1022 to cover this issue.

At the moment, section 2.1 of USEFOR-04 says:

 News articles MUST conform to the syntax specified in Section 3 of
 [RFC2822]. Netnews agents MAY also accept the obsolete syntax
 specified in Section 4 of [RFC2822], but they MUST NOT generate
 productions of such syntax.

This makes generation of obs-phrase (which allows unquoted dots in 
"phrase") forbidden.

I have failed to find a consistent message that someone is arguing that we 
should change this. I think Charles is arguing for a change, but I'm not 
sure   - either whether he really wants to change it, or what particular 
change he'd propose.

My instinct is "don't mess with other standards" and leave it as it is. Is 
someone arguing strongly that we should change it?

                       Harald

Date: Thu, 02 Jun 2005 23:28:10 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked]
Subject: #1008 USEFOR 1.5: Suggested language on "followup"
Sender: [email-masked]

The thread following my summary doesn't seem to have created many new 
arguments or changes of opinion, so the next step is trying to suggest 
language....

The option among my 4 alternatives that seemed to be least controversial 
was this one:

2) All "followups" have a References: header.
   There exist articles that have References:, but are not "followups"

In order to make this entirely clear in USEFOR, I suggest the following 
edit:

1) To section 1.5:

OLD:

   A "followup" is an article containing a response to the contents of
   an earlier article (the followup's "precursor" indicated in the
   "References" header).

NEW:

   A "followup" is an article containing a response to the contents of
   an earlier article. (the followup's "precursor" indicated in the
   "References" header).
   All "followups" have a "References" header.
   Note: Other articles may also use the "References" header.

I am pretty sure that John Stanley disagrees with this.
If there are other people who violently object to this formulation, or who 
think that the group has or can reach a consensus that is best represented 
by a different formulation, please speak up.

                    Harald

Date: Thu, 02 Jun 2005 23:18:41 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: John Stanley <[email-masked]>, [email-masked]
Subject: Re: IMAP definitions on threading 
Sender: [email-masked]

--On torsdag, juni 02, 2005 10:58:16 -0700 John Stanley <[email-masked]> 
wrote:

> Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:
>
>> John, I believe you are using language stronger than is justified.
>
> I see nothing improper in my language.
>
>> If you check out draft-ietf-imapext-thread-13 (the draft is expired,
>> unfortunately), you will find the method called ORDEREDSUBJECT referred
>> to  as "threading" -
>
> And obviously since it is in an IETF draft it is gospel. I shudder to
> think how many patently wrong things have appeared in USEFOR IETF draft
> products in the past, but apparently I should assume that every other WG
> never says anything daffy in their drafts.
>
>> ... the fact that you do not like this particular name for this way of
>> ordering messages does not mean that you have the right to claim that it
>> "does not exist".
>
> First of all, I did not claim that "that way of ordering articles" does
> not exist. I've been pretty clear that I understand that it is done, and
> that we have no control over it being done that way. So, implying that
> I've said "it does not exist" is nonsense.

Your particular words that I found too strong were:

> Threading on Subject does not exist.

I pointed to the particular IMAP draft as an example of a person using the 
word "threading" to describe something very like what you don't want to 
call "threading".

You may have a definition of "threading" that makes the IMAP command THREAD 
ORDEREDSUBJECT do something that doesn't fit your definition, so you may 
want to say that "THREAD ORDEREDSUBJECT" is not threading.

Your problem. But your claim that this definition is "the truth" is too 
strong for my taste.

                     Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: encoded words in an obs-phrase
Date:  Thu, 02 Jun 2005 22:07:22 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 36
Sender: [email-masked]

Bruce Lilly wrote:

>>>> RfC 2047,
>>> orthogonal to 2822.
>> It _redefines_ phrase.
> Not significantly (note that -- intentionally

Sure, it's clear what it's really doing.  But some subtleties
with specials in Q-encoded words are still fascinating.  Like
attempts to decode without parsing "only to display it", where
"it" was a structured header field.

>> There might be one redefinition too many if USEFOR tries it
>> for the third time,

> Are you suggesting that there's a proposal on the table that
> would do so?

If this table includes "an idea I posted yesterday here", yes,
just replace the 2822 phrase by its obscene incarnation:

 {2822]
| phrase          =       1*word / obs-phrase
| obs-phrase      =       word *(word / "." / CFWS)

 [idea]
  phrase          =       word *(word / "." / CFWS)

> a normative reference to RFC 2822 should suffice

We have a normative "2822 excl. all obs- forms".  The same
strategy as for Date would also work, but that's more than
just a reference, it's two paragraphs of text plus the ANBF.

                       Bye, Frank

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: encoded words in an obs-phrase
Date: Thu, 2 Jun 2005 15:22:34 -0400
Cc: [email-masked]
Sender: [email-masked]

> > If you replace an entire SP-delimited word, you have to
> > consider whether what is on the other side of the SP is
> > or is not an encoded word and is or is not part of the
> > phrase and is or is not a special
> 
> Yes, and so far there never were any specials in a phrase,
> unless they were already encoded / quoted / in a comment.

The veracity of that statement depends on the definition of "in",
e.g. given
  Keywords: =?us?q?foo?= (bar), baz
the comment can be said to be "in" the phrase containing the
encoded-word (both appear before the delimiting comma), and
the left parenthesis is a special but would normally not be
considered to be "in" the comment (it delimits comments in
many structured fields).  This is an ambiguity related to
vagueness of RFC 822, which permits comments but is silent
about whether a comment such as the one above is or is not
"in" the phrase; RFC 2822 tacks on [CFWS] to many productions,
meaning the comment can clearly be said to be "in" the phrase:

atom            =       [CFWS] 1*atext [CFWS]

word            =       atom / quoted-string

phrase          =       1*word / obs-phrase

but unfortunately the 2822 grammar also results sometimes in
adjacent [CFWS] [CFWS], resulting in some remaining ambiguities
(but not in this instance).

> In other words a recipe like "grab anything in a phrase up to
> the next delimiter (= comma in the case of keywords), DQUOTE,
> encoded-word (=?), or comment , and then encode it as you see
> fit under the rules of 2047" worked.

The "under the rules of 2047" entails quite a bit of complexity;
it is definitely not as simple as considering a word without
looking at surrounding context.  Even in the simple case of
encountering a comma (== special), there are rules; an encoded-word
has to be separated from specials by linear whitespace.
 
> > In any event, '.' being a special, it cannot appear in an
> > encoded-word in a phrase.
> 
> Exactly, that's the point (pun intended).  The 2822 obs-phrase
> would break the simple recipe to get it automatically right.

The "simple recipe" isn't as simple as you initially stated, and
the rules handling specials suffice as well for a period as for a
comma.  Note that 2822's rule only states that if an unquoted
period appears in certain places it might be considered part of a
phrase for parsing; it does not permit it for generation.  Your
concern seems to be about generation ("grab anything [...] then
encode"), and as 2822 does not change the prohibition regarding
unquoted periods for generation, there is no problem.
  
> > 2822 obsoletes STD 11
> 
> Don't tell me, tell it to legacy applications.  BTW, the IESG
> has just approved an RfC explaining these issues of backwards
> compatibility:
> 
> | There are many subtleties and variations that may come into
> | play.   Authors should very carefully consider backwards
> | compatibility when devising extensions, and should clearly
> | describe all known compatibility issues.
> 
> Sounds familiar ?

Yes, as does "non-binding informational supplement".  In the
specific case under discussion, there is no change to generation
rules.  Lack of backwards compatibility happens when some new
set of generation rules permits something to be foisted upon
deployed implementations which are unprepared for it.  That doesn't
apply here.

> >> RfC 2047,
> > orthogonal to 2822.
> It _redefines_ phrase.

Not significantly (note that -- intentionally -- the important
characters constituting an encoded-word, '=', '?', and in the
case of the RFC 2231 amendment, '*', are all legal in "atom",
as are the B and Q encoded characters, legal charset and language
tag names (at least as currently defined)).  That was done
precisely for backwards compatibility; an encoded-word *is* an
atom as far as pre-MIME RFC 822 parsing is concerned.  The
"redefinition" is for the benefit of MIME-aware parsers that
recognize encoded-words as a special subset of atoms.  While
both =?us?q?foo?= and =??*-???= are atoms, only the former is an
encoded-word.

> There might be one redefinition too 
> many if USEFOR tries it for the third time,

Are you suggesting that there's a proposal on the table that would
do so?

> But I hate anything making usefor-05 longer / more obscure /
> less readable / unclear / a waste of time.

In this case, a normative reference to RFC 2822 should suffice.  If
there is some operational issue related to it, then the most concise,
clearest, most readable, and simplest way to address the matter would
be along the lines suggested by Mark Crispin (not limited to *this*
issue; it applies to SP-after-colon, length of msg-id, etc. ad nauseum).

Date: Thu, 2 Jun 2005 10:58:16 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: IMAP definitions on threading 
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:

>John, I believe you are using language stronger than is justified.

I see nothing improper in my language. 

>If you check out draft-ietf-imapext-thread-13 (the draft is expired, 
>unfortunately), you will find the method called ORDEREDSUBJECT referred to 
>as "threading" -

And obviously since it is in an IETF draft it is gospel. I shudder to
think how many patently wrong things have appeared in USEFOR IETF draft
products in the past, but apparently I should assume that every other WG
never says anything daffy in their drafts.

>... the fact that you do not like this particular name for this way of 
>ordering messages does not mean that you have the right to claim that it 
>"does not exist".

First of all, I did not claim that "that way of ordering articles" does 
not exist. I've been pretty clear that I understand that it is done, and 
that we have no control over it being done that way. So, implying that 
I've said "it does not exist" is nonsense. 

But then, what part of "the Subject header contains NO article
relationship information" do you see as saying "I don't like this and thus
it does not exist?" If you can read paragraphs that point out obvious
technical limitations in calling something "theading" and pretend that the
only argument being presented is "he doesn't like it", then there is a
problem.

Perhaps you'd like to disprove my statement? Ok, here are four subject
headers. Please put them in the correct threading order, using the
"threading information" you find in those headers. Be explicit, and 
describe exactly what threading information you base your decision on.
You may open your test booklet and begin now.

1: Subject: A question
2: Subject: Re: A question
3: Subject: Re: A question
4: Subject: Stop asking stupid questions

Warning: it's a trick question. You cannot answer correctly. But then,
that's the point. There is no data to generate the right answer. The 
Subject header simply does not allow threading, especially not the 
identification of specific threads that someone here claimed was possible. 
Sorting, yes. Threading, no. And not just "because I don't like it", 
because the RFC doesn't support it. 

Oh, wait, that was synthetic, so it obviously isn't like real-world data.
Ok. Here's four real subjects from a real newsgroup. Put them in the
proper threaded relationship. I'll make it easier, I'll make it multiple
choice, but you still have to show your work. If you simply answer "B"  
without supporting work, you get no credit. "Process of elimination" is 
acceptable.

1:        how does ByteLoader work? 
2:    Re: how does ByteLoader work? 
3:    Re: how does ByteLoader work? 
4:    Re: how does ByteLoader work?  

A: 1->2->3->4
B: 1->2, 1->3, 2->4
C: 1->2->3, 1->4
D: 1->4, 4->2, 2->3
E: 1->4, ?->2, ?->3  (? is "article not listed above")

And the correct answer is: 

Date: Thu, 2 Jun 2005 13:53:11 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Sender: [email-masked]

"Charles Lindsey" <[email-masked]> wrote:

> OTOH, I would have some sympathy for the man who decided that the
> new Subject had changed so far from the original topic that it was
> really a new thread, and who therefore manually trimmed all the
> earlier articles out of the References header.

If the connection between his article and its predecessor were as
tenuous as "That reminds me of an interesting story . . ." that would
be fine.  Otherwise, he's breaking the connection between his article
and its parent, which is likely not such a good idea.

> Yes, that would break the letter of our drafts, but it would be well
> within the spirit.

But then, we can't control user behavior anyway.

Seth

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Injection-Info issues
Date:  Thu, 02 Jun 2005 19:36:33 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 70
Sender: [email-masked]

Charles Lindsey wrote:

 [CFWS issues for parameters]
> I don't want to discuss these here, because they are better
> resolved offline.

Is anything wrong with copying RfC 3834 opt-parameter-list ?
I'd be very interested if that's not more state of the art.

> 1. We now have <ext-param>s, which allow for further
>   parameters in extensions. There is also a query:
>    [[Should also allow for x-attributes?]]

RfC 2231 <attribute-char> allows "x" and "-" among others.
Why reserve "x-" if there's no IANA registry ?  RfC 3834
has not a single additional word about opt-parameter-list.

>| allow <parameter>s with x-attributes

Is x-attributes a defined term ?  I know what you mean, but we
would get "[x] draft should try to avoid hacker slang" for it.

> a) folding SHOULD NOT happen in silly places

Let's better forget it.  An article format designed to simplify
"killfiling" can't be a good idea for USEFOR.  Maybe put it in
USEAGE.  Some uses of sender= or posting-account= are probably
against the law in my country, and some legal uses won't allow
"killfiling".

> b) if several parameters are present, they SHOULD come in a
>    prescribed order

See (a), it's not worth it.

> c) if a <post-account-param> is used, a given sender SHOULD
>    always show up as the same posting-account, at least in
>    the short term.

Dito, that's a legal nightmare, let's forget it, or discuss
it with some legal advisors and privacy officers - IANAL.

No idea for your point 3.  If you want to put something in a
sender= that's neither "verified" nor a mailbox you have to
allow it.  Unless this NNTP AUTH stuff has the syntactical
form of a mailbox of course.

> (as Frank would have us do in his parallel universe)

With the MUST in 2.3 I lose.  It's still strange, RfC 2822 and
mail manage to avoid this issue, why are we unable to do this ?
What's wrong with our zero-I18N newsgroup names etc., that we
still need this RfC 2231 MUSTard ?

> we can still discourage its unnecessary use

It does this by itself, we don't need to worry about this:

| IMPORTANT NOTE:  These mechanisms end up being somewhat
| gibbous when they actually are used. As such, these
| mechanisms should not be used lightly; they should be
| reserved for situations where a real need for them exists.

What we really need are some security considerations about the
MIME boundary parameter.  While we're at it we could also add
a note about the known message/partial risks, it's essentially
the same issue -  IIRC with a registered "MIME vulnerability"
number for the latter, we could even add it to the informative
references, if that's allowed and good style (?)  Bye, Frank

To: [email-masked]
Subject: Re: Injection-Info issues
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 02 Jun 2005 09:42:50 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> 3. There is now a draft (with Ken's name on it) for SASL authentication
> within NNTP. That may give rise AIUI to an "authorization identity"
> which may or may not be in the form of a <mailbox>.

I think you mean authentication identity; authorization identity is the
thing that we had before.  The authentication identity may, in fact, may
often look like a <mailbox> but not actually be one in any traditional
sense.

> An injecting agent might then decide to use this identity in the
> Injection-Info header. Question: do we need a seperate "SASL-identity"
> parameter for this, or can we just use the "sender" parameter and, if
> so, should its syntax be extended to suit?

If anyone expects sender to be an actual mailbox, using it for the
authentication identity is a bad idea, since the authentication identity
may be indistinguishable from a mailbox but not be usable for the same
purpose.

SASL-identity is a bad choice since it unnecessarily restricts what the
field may be used for.  I'd rather use a more generic name.  The field
should be described as an opaque token that may only have meaning for the
originating site.

> 4. Everyone on the WG heartily dislikes RFC 2231, and would like to
> deprecate its use except where absolutely necessary.

Not true.  I'm just tired of arguing about it.  I think it's a workable,
if ugly, solution to the (very hard) problem it was trying to solve, and
the *real* problem is using MIME-style parameters in places where they
don't belong.

To: [email-masked]
Subject: Re: Path header - suggested text
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 02 Jun 2005 09:36:42 -0700
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> checking..... is the !! stuff a feature that has seen production use
> somewhere, or is it a new feature that we are adding?

It's a new feature.

To: [email-masked]
Xref: clerew local.usefor:21006
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Injection-Info issues
Date: Thu, 2 Jun 2005 15:31:54 GMT
Lines: 71
Sender: [email-masked]

Draft-usefor-03 added proper texts for the various parameters of the
Injection-Info header, and draft-usefor-04 added <ext-param>s, and fixed a
problem with the <complainto-param>. But I still have a few issues I would
like to raise.

NOTE: There are further issues regarding the way <parameter>s are defined,
where CFWS is allowed and where quoting is needed, but I don't want to
discuss these here, because they are better resolved offline.

So, my issues:

1. We now have <ext-param>s, which allow for further parameters in
extensions. There is also a query:
   [[Should also allow for x-attributes?]]

I suggest the following text:
   <ext-params> are provided to allow additional <parameter>s to be
   defined in future extensions, and also to allow <parameter>s with
   x-attributes to be used in circumstances where the defined ones
   appear to be insufficient.

2. A major use of this header will be for killfiling by users on
particular parameters, or as an aid to detecting spams, spews, etc by
netcops (in the best sense of that term). There were a several features in
the old draft-13 to facilitate this:
   a) folding SHOULD NOT happen in silly places, and <comment>s should
come at the end (this provision is already in the draft).
   b) if several parameters are present, they SHOULD come in a prescribed
order (such as the order they are defined in the syntax) so as to
facilitate writing regular expressions.
   c) if a <post-account-param> is used, a given sender SHOULD always
show up as the same posting-account, at least in the short term.

I would like to see fetures b and c reinstated, plus an explanatory NOTE
mentioning killfiles. The texts in draft-13 could easily be reinstated or
adapted.

3. There is now a draft (with Ken's name on it) for SASL authentication
within NNTP. That may give rise AIUI to an "authorization identity" which
may or may not be in the form of a <mailbox>. An injecting agent might
then decide to use this identity in the Injection-Info header. Question:
do we need a seperate "SASL-identity" parameter for this, or can we just
use the "sender" parameter and, if so, should its syntax be extended to
suit?

4. Everyone on the WG heartily dislikes RFC 2231, and would like to
deprecate its use except where absolutely necessary. Assuming that we are
not in a position to get rid of RFC 2231 entirely (as Frank would have us
do in his parallel universe), we can still discourage its unnecessary
use. Here is a text that I have suggested before (though there may be a
better place to put it than in the Injection-Info section):

     NOTE: Although the provisions of RFC 2231 for Non-ASCII characters in
     <value>s of <parameter>s provide useful additional functionality,
     those for splitting <value>s over multiple lines offer no additional
     benefit, since the only length limit on header lines specified by
     this standard is an overall 998 octets. That latter mechanism is,
     moreover, somewhat gibbous, and as RFC 2231 itself recommends its use
     only where a real need exists, it is better avoided within Netnews.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Thu, 02 Jun 2005 17:59:13 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: Path header - suggested text
Sender: [email-masked]

--On onsdag, juni 01, 2005 19:54:44 +0000 Charles Lindsey 
<[email-masked]> wrote:

> The <keyword> "POSTED"
>    indicates that the agent to its left injected the article.  The use
>    of the <path-delimiter> "!!" indicates that the agent to its left
>    claims that the agent to its right was the verified source of the
>    article (whereas the <path-delimiter> "!" implies no such claim).

checking..... is the !! stuff a feature that has seen production use 
somewhere, or is it a new feature that we are adding?

since it could concievably be constructed as having to do with security, 
it's a potential sticky point with the security ADs.

                  Harald

To: [email-masked]
Xref: clerew local.usefor:20998
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Misc. ABNF nits
Date: Thu, 2 Jun 2005 11:23:06 GMT
Lines: 25
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> So what is your problem?

>Attackers could try to hide behind a foo_bar.example.com
>path identity, because it doesn't work as an abuse@ FQDN

But even if '_' was forbidden as a <path-identity> character, the attacker
could still try to hide behind foobar.example.com, and I doubt
[email-masked] would work either (attackers are not in the habit
of using email addresses that "work", unless they are trying to generate a
mailbomb).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:20999
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: encoded words in an obs-phrase (was: Differences between RFC 2822 and Usefor)
Keywords: =?utf-8?Q?K._Moore?= , =?utf-8?Q?Jonathan_(Jon)_Postel?=
Date: Thu, 2 Jun 2005 11:35:49 GMT
Lines: 46
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>>>  display-name    =       word *(word / "." / CFWS)
>>> This syntax would still disallow obscene keywords.

>> If we are going to put it into USEFOR (and I think we
>> should), then it may as well go into Keywords too. It is
>> clearly allowed in Keywords in RFC 2822

>Only by a very obs-cure rule, hiding itself as obs-olete,
>and I'm not sure how it's supposed to work with RfC 2047:

>Keywords: =?utf-8?Q?K._Moore?=

>That's illegal, see 2047 5.3.  Dito for a <display-name>.

It is a MUST accept (but NOT generate) in RFC 2822. But it is not a valid
<encoded-word> according to RFC 2047 (because RFC 2047 has not caught up
with the change introduced by RFC 2822). But you could write

Keywords: =?utf-8?Q?K=2E_Moore?=

and then everyone should be happy.

But the whole point is that people are going to write
   From: K.Moore <keith@wherever>
whether we like it or not. So RFC 2822 said that MUST be accepted, and I
(and Bruce, it seems) are saying that USEFOR should say essentially the
same.

And if it does it for <addr-spec>s, then it may as well do it for Keywords
too, just to avoid yet more exceptions in the grammar (for sure, it won't
break anything to allow that).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:21000
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Date: Thu, 2 Jun 2005 11:39:54 GMT
Lines: 26
Sender: [email-masked]

In <[email-masked]> Eivind Tagseth <[email-masked]> writes:

>A followup agent has to obey some rules when following up to
>an article with a references header, the subject header can never
>override this.  If a followup agent ignores the references header
>when creating a followup just because the subject has changed, it
>would be broken.

OTOH, I would have some sympathy for the man who decided that the new
Subject had changed so far from the original topic that it was really a
new thread, and who therefore manually trimmed all the earlier articles
out of the References header.

Yes, that would break the letter of our drafts, but it would be well
within the spirit.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: I-D ACTION:draft-ietf-usefor-usefor-04.txt
Date:  Thu, 02 Jun 2005 15:32:45 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 22
Sender: [email-masked]

Harald Tveit Alvestrand wrote:

> RT claims that you should instead bookmark this horror:
> https://rt.psg.com/Search/Listing.html?Bookmark=FrT%3B%404%7C%256%7C%241%7C1%241%7C2%241%7C3%258%7C%2411%7CDESCRIPTION%245%7CFIELD%248%7COPERATOR%245%7CVALUE%2414%7CQueue%20%3D%20usefor%245%7CQueue%241%7C%3D%246%7Cusefor%258%7C%2411%7CDESCRIPTION%245%7CFIELD%248%7COPERATOR%245%7CVALUE%2413%7CStatus%20%3D%20open%246%7CStatus%241%7C%3D%244%7Copen%258%7C%2411%7CDESCRIPTION%245%7CFIELD%248%7COPERATOR%245%7CVALUE%2412%7CStatus%20%3D%20new%246%7CStatus%241%7C%3D%243%7Cnew%241%7C4%241%7C0%241%7C0&TicketsSortBy=&TicketsSortOrder=&RowsPerPage=50

Ouch.  For a given ticket number like 1021 the shorter URL...

<https://rt.psg.com/Ticket/Display.html?id=1021>

...works.  I've just tested to add a comment, but when I wanted
to spam the world (using my own address and Alexey's in a CC:
as test victims) it fortunately said njet.

> Login with user "ietf", password "ietf".

A shorter version of the horror link apparently also works:

<https://rt.psg.com/Search/Listing.html?QueueOp=%3D&ValueOfQueue=48>

It only wants some queue "=" "48" to be happy for USEFOR with
the search defaults for owner, sort order, status, etc.  Bye

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Date: Thu, 2 Jun 2005 15:05:31 +0200
Organization: Chaos rules.
Sender: [email-masked]

Charles Lindsey schreef:

> the whole discussion 
> would have made more sense if I had just sorted it on Date within
> Subject. 

For mailing-lists, I read-by-date-only. For mailing-lists that 
share a large group of contributors, I even let those messages 
be delivered to a common mailbox, and read-by-date-only.
I rather use date-received than the Date-header.

-- 
Grtz, Ruud

Date: Thu, 02 Jun 2005 14:42:30 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: John Stanley <[email-masked]>, [email-masked]
Subject: IMAP definitions on threading (Re: #1008 USEFOR 1.5 "followup": summary so far)
Sender: [email-masked]

This thread is far enough off topic that I'm changing the subject line...

--On onsdag, juni 01, 2005 15:10:10 -0700 John Stanley <[email-masked]> 
wrote:

>> With "threading on Subject",
>
> Threading on Subject does not exist. It is "sorting by subject". You
> obviously missed the entire paragraph where I pointed out that the Subject
> header has NO article relationship information, and thus cannot be used to
> identify threads.

John, I believe you are using language stronger than is justified.

If you check out draft-ietf-imapext-thread-13 (the draft is expired, 
unfortunately), you will find the method called ORDEREDSUBJECT referred to 
as "threading" - the fact that you do not like this particular name for 
this way of ordering messages does not mean that you have the right to 
claim that it "does not exist".

                     Harald

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Path header - suggested text
Date:  Thu, 02 Jun 2005 14:24:17 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 7
Sender: [email-masked]

Sorry, now that Bruce is back I could stop this, but...
 
>| 1. A fully qualified domain name (FQDN) associated with an
>|    A record, which SHOULD identify the actual machine
 
...we need "or AAAA", don't we ?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Path header - suggested text
Date:  Thu, 02 Jun 2005 14:10:04 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 42
Sender: [email-masked]

Charles Lindsey wrote:

> This is, I hope, the final form of the Path header.

Thanks, I found no problems, maybe two and a half nits:

> path-identity   =  ( ALPHA / DIGIT )
>                    *( ALPHA / DIGIT / "-" / "." / ":" / "_" )

See below.

>| 1. A fully qualified domain name (FQDN) associated with an
>|    A record, which SHOULD identify the actual machine

RfC 1034 (3.6) offers s/machine/host/ if it has an A record.
I stumbled over it while checking that we get no "_" here.

>| this SHOULD then be followed by CRLF and WSP if it would
>| otherwise result in a line longer than 79 characters.

See below.

>| All such prepended material SHOULD be followed by CRLF and
>| WSP if it would otherwise result in a line longer than 79
>| characters.

I tried to check the magic number 79 in s-o-1036, found nothing,
but stumbled over this part:

  Note, in particular, that RFC 1036 treats "_" as  a  delim-
  iter, but in fact it is known to appear in relayer names
  occasionally.

So "_" is apparently not exactly compatible with RfC 1036.  If
Henry doesn't publish s-o-1036 before its grandson the latter
could say so.  Most probably irrelevant.

If the 79 is an almost arbitrary magic number I'd prefer one
of the known magic numbers (78 in 2822, or 76 in 2047), too
many magic numbers are confusing.
                                  Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:20997
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Date: Thu, 2 Jun 2005 11:19:02 GMT
Lines: 48
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>In message <[email-masked]>, John
>Stanley <[email-masked]> writes

>>There is never a necessity to sort by Subject (not "thread", since Subject 
>>isn't a threadable header.). 

>apologies, I was imprecise I meant "attempt to locate specific thread
>using Subject (fuzzy matching as needed) and place into position using
>other clues such as Date"

Which is sometimes referred to as "Poor man's threading" (there is even an
ID with those words in it).

But USEFOR does not define "threading" as a specific technical term, so it
means whatever it means "out there in the real world", where is is
sometime used loosely to describe several threading-like activities. That
loose "real world" meaning is plenty good enough for the limited use of
the term in our drafts.

>>If it isn't ever necessary, 

>I think it is :)  I like threads. So do almost all the people who use
>the software I designed. That's quite a few.

OTOH, pure threading is not always the best. I was recently reviewing a
discussion on this WG dating from 2001. A large tree of sub-threads was
generated over several days and, reading those articles in the usual "depth
first traversal", I found that I kept coming across messages which
predated (and appeared to contradict) messages from those same people
which I had already read. The problem was that people were replying in one
sub-thread of the tree with full knowledge of what people had said in
parallel sub-threads.

I managed to work it all out, but I concluded that the whole discussion
would have made more sense if I had just sorted it on Date within Subject.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: encoded words in an obs-phrase
Date:  Thu, 02 Jun 2005 03:19:45 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 81
Sender: [email-masked]

Bruce Lilly wrote:

 [2047]
> I think you mean section 5, paragraph marked "(3)";
> there is no 5.3.

Yes, your assumption is correct.

> If you replace an entire SP-delimited word, you have to
> consider whether what is on the other side of the SP is
> or is not an encoded word and is or is not part of the
> phrase and is or is not a special

Yes, and so far there never were any specials in a phrase,
unless they were already encoded / quoted / in a comment.

In other words a recipe like "grab anything in a phrase up to
the next delimiter (= comma in the case of keywords), DQUOTE,
encoded-word (=?), or comment , and then encode it as you see
fit under the rules of 2047" worked.

> N.B. Q encoding differs from QP encoding.

Yes, they are <quote rfc=2047> similar </quote>.

> In any event, '.' being a special, it cannot appear in an
> encoded-word in a phrase.

Exactly, that's the point (pun intended).  The 2822 obs-phrase
would break the simple recipe to get it automatically right.

> Why one would use encoding with utf-8 in the absence of
> language tagging for text which can be represented in the
> default us-ascii charset is a complete mystery.

utf-8 was shorter than us-ascii for my example.  Think of any
reasonable case, e.g. =?utf-8*en?q?Moore?= combined with voice
output.  =?utf-8*de?q?Moore?= would sound like "more a" and is
the German plural of "moor".

> 2822 obsoletes STD 11

Don't tell me, tell it to legacy applications.  BTW, the IESG
has just approved an RfC explaining these issues of backwards
compatibility:

| There are many subtleties and variations that may come into
| play.   Authors should very carefully consider backwards
| compatibility when devising extensions, and should clearly
| describe all known compatibility issues.

Sounds familiar ?

>> RfC 2047,
> orthogonal to 2822.

It _redefines_ phrase.  There might be one redefinition too
many if USEFOR tries it for the third time, and it could...

| Cause a deployed implementation to behave differently from
| how it would behave in the absence of the proposed change,
| in ways that are not intended by the proposal.  That is a
| failure of the proposal to remain backwards compatible with
| the deployed base of implementations.

>> s-o-1036.
> Not a standard of any kind.  Not an RFC of any kind.

The best available standard available for Netnews articles, if
you have difficulties to decide which of the 13 + 4 usefor
drafts might be better than 1036.  Probably it inspired some
developers over the years, and resulted in deployed software.

Unlike msg-id = "<" id-local "@" id-domain ">" and potential
2231-boundary-risks I don't care much about this period-issue.

But I hate anything making usefor-05 longer / more obscure /
less readable / unclear / a waste of time.

                            Bye, Frank

Date: Wed, 1 Jun 2005 15:10:10 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Sender: [email-masked]

"Ruud H.G. van Tol" <rvtol@xxxxxxxxxxxx>:

>You obviously missed the "only" in the first sentence that you quote
>from my text.

I obviously did no such thing. Sorting by subject does not include date 
headers, otherwise it would be called "sorting by making things up" and 
not just "sort by subject".

>With "threading on Subject", 

Threading on Subject does not exist. It is "sorting by subject". You
obviously missed the entire paragraph where I pointed out that the Subject
header has NO article relationship information, and thus cannot be used to
identify threads.

>It is threading,

Nonsense.

>because the software uses whatever it finds to
>detect followup-relations between messages, 

The Subject header contains NO information regarding followup relations
between messages, so using the Subject header to detect followup relations
is impossible. The Date header contains no information about followup
relations, so ditto. The body is UNSTRUCTURED. Nothing plus nothing plus
nothing is nothing, not "just like a References header". 

>and presents them in a
>tree-structure just like if there was a References-header to go by.

The fact that some clients lie to the user and pretend that there is some
relationship when none can be detected is unfortunate and should be
corrected (but not by us), and is not an excuse for an IETF working group
to propogate the lie. A thread is a group of related articles, which can
only be detected by the References header.

>Such algorithms are also nice to use 

Obviously you missed the part where I said yeah, people LIKE sorting by 
subject etc, but that still doesn't make it threading, and that sorting by 
Subject is neither a) necessary nor b) relevant to a discussion about 
References. Nothing we do about References headers will make sorting by 
Subject either more or less correct, even ignoring that the current level 
of "correct" is zero. 

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Frank Ellermann <[email-masked]>
Subject: Re: encoded words in an obs-phrase (was: Differences between RFC 2822 and Usefor)
Date: Wed, 1 Jun 2005 14:36:55 -0400
Cc: [email-masked]
Sender: [email-masked]

On Wed June 1 2005 12:23, Frank Ellermann wrote:
> 
> Charles Lindsey wrote:
> 
> >>  display-name    =       word *(word / "." / CFWS)
> >> This syntax would still disallow obscene keywords.
> 
> > If we are going to put it into USEFOR (and I think we
> > should), then it may as well go into Keywords too. It is
> > clearly allowed in Keywords in RFC 2822
> 
> Only by a very obs-cure rule, hiding itself as obs-olete,
> and I'm not sure how it's supposed to work with RfC 2047:
> 
> Keywords: =?utf-8?Q?K._Moore?=
> 
> That's illegal, see 2047 5.3.  Dito for a <display-name>.

Leaving aside for the moment whether or not putting Keith's initial
and surname in a Keywords field makes sense, I think you mean
section 5, paragraph marked "(3)"; there is no 5.3.

> Normally you can split a phrase without quoted-string at
> any word boundary and replace one or more words by an QP
> encoded-word.

No, it's not like that.  If you replace an entire SP-delimited word,
you have to consider whether what is on the other side of the SP is
or is not an encoded word and is or is not part of the phrase and is
or is not a special, and you may have to include an encoded SP --
underscore for Q encoding, more complex for B encoding -- depending
on that analysis.  If you did that, the encoded-word length limit
might be exceeded, and you'd therefore have to choose a different or
additional splitting point.  You can't simply replace a word.  And
N.B. Q encoding differs from QP encoding.

In any event, '.' being a special, it cannot appear in an
encoded-word in a phrase.  RFC 2047 uses and is based on RFC 822
syntax rules, not 2822's, and 2822 does not discuss MIME other than
say that it's outside of 2822's scope.  Whether or not 
   Keywords: =?utf-8?Q?K?= . =?utf-8?Q?_Moore?=
might or might not be legal to generate under some hypothetical
successors of RFCs 2047 and 2822 is academic.  Under 2822, unquoted
'.' in a phrase is illegal for generation. And
   Keywords: =?utf-8?Q?K?= ". " =?utf-8?Q?Moore?=
is legal under 2047 + (822 or 2822).  Why one would use encoding
with utf-8 in the absence of language tagging for text which can be
represented in the default us-ascii charset is a complete mystery.
  
> This obsure RfC 2822 rule messses with specials, STD 11,

2822 obsoletes STD 11, and defines specials as the same set of twelve
characters as the obsoleted STD 11.

> RfC 2047,

orthogonal to 2822.

> and s-o-1036. 

Not a standard of any kind.  Not an RFC of any kind.  Predated
RFC 2822 and the observed widespread syntax error which 2822 accounts
for.

> Okay, that's kind of obvious, RfC 2047 only promises that
> you can encode word(s) in a phrase, and "(Jon)" is no word.

'.' is not a word either.

As for keywords, there might be some legitimate use, e.g. "X.25" or "T.3"
or "H.263" as keywords.  For now, they must be quoted.  The decision
regarding possible future codifcation in generation rules is up to an
RFC 2822 successor. 
----------
The reason for this to be part of USEFOR is that we have agreed that
USEFOR is to be based on 2822, and it is part of 2822.  There also seems
to be support for minimizing differences between the article format and
the Internet Message Format on which it is based wherever possible.

And where there is some operational issue of concern, Mark Crispin's
suggestion of an implementation note seems to be a vastly superior
method for handling the issue than a horde of "like 2822, except..."
clauses, incompatible redefinitions of RFC 2822 terms and grammar
productions, and/or endless debates over whether a difference should be
expressed by SHOULD or MUST:
http://www.imc.org/ietf-usefor/2003/Feb/0486.html
http://www.imc.org/ietf-usefor/2003/Feb/0493.html
http://www.imc.org/ietf-usefor/2003/Feb/0554.html

Had more of us listened to Mark's advice, we might have finished our
work two years ago.

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Date: Wed, 1 Jun 2005 22:49:32 +0200
Organization: Chaos rules.
Sender: [email-masked]

John Stanley:
> Ruud H.G. van Tol:

>> To thread on Subject, doesn't mean that only the value of the
>> Subject: header is used. Other information that can be used is
>> the Date and the attributions  ...
>
> Simply remarkable. If it isn't sorting by the Subject header, then it
> isn't "sort by Subject", it's "sort by date" or "make stuff up
> sorting".

You obviously missed the "only" in the first sentence that you quote
from my text. With "threading on Subject", the Subject is an important
value, but a combination of values like Subject + Date +
part-of-the-body
is needed to find relations on which to thread.

> It's a guessing game which works sometimes and fails miserably at
> other times

I never said it isn't or doesn't.

> The problem with what you
> are saying is not that sorting isn't done by some clients that way or
> that people don't like articles sorted that way, it is that you keep
> claiming that this is "threading", and it simply ain't so.

It is threading, because the software uses whatever it finds to
detect followup-relations between messages, and presents them in a
tree-structure just like if there was a References-header to go by.

Such algorithms are also nice to use with archives of messages from
before the Received-header. Example:
http://groups.google.co.uk/groups?threadm=anews.Aucbvax.1635

-- 
Grtz, Ruud

To: [email-masked]
Xref: clerew local.usefor:20985
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Path header - suggested text
Date: Wed, 1 Jun 2005 19:54:44 GMT
Lines: 215
Sender: [email-masked]

This is, I hope, the final form of the Path header. It was pretty well
agreed last time, but Alexey asked me to move the material regarding the
form of <path-identity>s from USEFOR to USEPRO, which I have now done,
resulting in a new "Identification of news-servers" section in USEPRO.

There are a few minor fixes required in USEFOR, including a slight syntax
change for <tail-entry> which I suggested here a couple of days ago, the
removal of an unnecessary reference to the <tail-entry> in USEPRO, and a
sentence about <IPv6address>s which did not get moved to USEPRO.

In USEFOR

3.1.6  Path 

   The Path header indicates the route taken by an article since its
   injection into the Netnews system.  Each agent that processes an
   article is required to prepend one (or more) identities to this
   header.  This is primarily to enable relaying agents to avoid sending
   articles to sites already known to have them, in particular the site
   they came from, and additionally to permit tracing the route articles
   take in moving over the network, and for gathering Usenet statistics.

   path            =  "Path:" SP path-list CRLF

   path-list       =  [FWS]
                      *( ( path-identity / path-keyword ) [FWS]
                      path-delimiter [FWS] )
                      tail-entry [FWS]

   path-identity   =  ( ALPHA / DIGIT )
                      *( ALPHA / DIGIT / "-" / "." / ":" / "_" )

   path-keyword    = "POSTED" / "MISMATCH"

|  tail-entry      =  ( ALPHA / DIGIT )
|                     *( ALPHA / DIGIT / "-" / "." / ":" / "_" )
[To avoid any suggestion that the <tail-entry> inherits any of the
semantic properties of a <path-identity> (see USEPRO). Any other syntax
that avoids that problem would do.]

   path-delimiter  =  "!" / "!!"

   Each <path-identity> in the <path-list> (excluding the one in the
   <tail-entry>) indicates, from right to left, the successive agents
   through which the article has passed.  The <keyword> "POSTED"
   indicates that the agent to its left injected the article.  The use
   of the <path-delimiter> "!!" indicates that the agent to its left
   claims that the agent to its right was the verified source of the
   article (whereas the <path-delimiter> "!" implies no such claim).
   The <keyword> "MISMATCH" indicates that the agent to its right failed
   to be so verified.  The full procedure for constructing a Path header
|  as well as the specific format and use of the <path-identity> are
|  discussed in [USEPRO].
[No need to refer to USEPRO for the <tail-entry>, because the following
NOTE contains all that needs to be said.]

      NOTE: Historically, the <tail-entry> indicated the name of the
      sender.  If not used for this purpose, the string "not-for-mail"
      is often used instead (since at one time the whole path could be
      used as a mail address for the sender).

      NOTE: Although case-insensitive, it is intended that the <path-
      keyword>s "POSTED" and "MISMATCH" should be in upper case, to
      distinguish them from the <path-identity>s which are traditionally
      in lower case.

|     NOTE: The character ':' is now permitted within a <path-identity>
|     to facilitate the use of an <IPv6address> therein.
[That sentence belongs here, rather than in USEPRO.]

In USEPRO

2.1.  Defining the Architecture

....................

   Posting, reading and followup agents (which are usually just
   different services provided by the same piece of software) are known
   collectively as "user agents".

|  Injecting, relaying and serving agents (which are often just
|  different services provided by the same piece of software) are known
|  collectively as "news-servers".

2.2.  Identification of news-servers

|  News-servers need to identify themselves, by inserting their <path-
|  identity>s (F-3.1.6), into Path, Injection-Info and Xref headers. An
|  injecting agent MUST identify itself with the same <path-identity> in
|  both Path and Injection-Info headers.  To ensure that a <path-
|  identity> provides a unique identity for the news-server concerned,
|  it SHOULD be one of:
 
|  1. A fully qualified domain name (FQDN) associated with an A record,
|     which SHOULD identify the actual machine inserting this <path-
|     identity> and, ideally, should also be "mailable" (see below).
 
|  2. A fully qualified domain name (FQDN) associated with an MX record,
|     which MUST be "mailable".
 
|  3. An arbitrary name believed to be unique and registered at least
|     with all other news-servers receiving articles directly from the
|     given one.
 
|  4. An encoding of an IP address - <IPv4address> or <IPv6address> [RFC
|     3986].
 
|  The FQDN of a news-server is "mailable" if its administrators can be
|  reached by email using both of the forms "usenet@<FQDN>" and
|  "news@<FQDN>", in conformity with [RFC 2142].
 
|  For an injecting agent prepending to a Path header (7.2.2), the
|  <path-identity> MUST be option 1 or 2 and the FQDN MUST be mailable,
|  and if the agent offers its services to the general public the form
|  "abuse@<FQDN>" MUST also be available, unless a more specific
|  complaints address has been provided in a <complainto-param> of an
|  Injection-Info header (F-3.2.14).  For other agents, options 1
|  through 3 are to be preferred.

7.2.1.  Proto-articles

        NOTE: An article that is offered for reinjection has, by
        definition, already been injected once, and is not therefore to
        be considered as a proto-article.  Hence a genuine proto-article
|       will not contain any Injection-Date header nor any "POSTED" in
|       its Path header, though that header MAY contain <path-identity>s
|       corresponding to machines traversed between the posting agent
|       and the injecting agent proper.

7.2.  Duties of an Injecting Agent

   10.It MUST then prepend the <path-identity> of the injecting agent,
|     followed by a '!', the <path-keyword> "POSTED" and a further "!"
|     (or "!!" if appropriate) to the content of the Path header; this
|     SHOULD then be followed by CRLF and WSP if it would otherwise
|     result in a line longer than 79 characters.  The prepended
|     <path-identity> MUST be an FQDN mailable address (F-3.1.6).
 
|       NOTE: This could result in more that one "POSTED" <path-keyword>
        in the case of reinjection.

7.3.  Duties of a Relaying Agent

   1. It MUST establish the trusted identity of the source of the
      article and compare it with the leftmost <path-identity> of the
      Path header's content. If it matches it MUST then prepend its own
|     <path-identity> and a '!!' <path-delimiter> to that content. If
|     it does not match then it prepends instead two entries to that
|     content; firstly the true established <path-identity> of the
|     source followed by a '!', the <path-keyword> "MISMATCH" and a
|     further '!', and then, to the left of that, its own <path-
|     identity> followed by a '!!' <path-delimiter> as usual. This
|     prepending of two entries SHOULD NOT be done if the provided and
|     established identities match. All such prepended material SHOULD
|     be followed by CRLF and WSP if it would otherwise result in a line
|     longer than 79 characters.
 
7.3.1.  Path-Header Example

|     Path: foo.isp.example!!foo-server!!bar.isp.example!MISMATCH!
|        2001:DB8:0:0:8:800:200C:417A!!old.site.example!barbaz!!
|        baz.isp.example!POSTED!!dialup123.baz.isp.example!not-for-mail
 
        NOTE: That article was injected into the news stream by
|       baz.isp.example, as indicated ny the <path-keyword> "POSTED"
        (complaints may be addressed to [email-masked]). The
|       injector has chosen to record that it got it from
        dialup123.baz.isp.example. "not-for-mail" is a dummy <tail-
        entry>, though sometimes a real userid is put there.

        The article was relayed, perhaps by UUCP, to the machine known,
        at least to old.site.example, as "barbaz".

        Barbaz relayed it to old.site.example, which does not yet
        conform to this standard (hence the '!' <path-delimiter). So one
        cannot be sure that it really came from barbaz.

|       Old.site.example relayed it to a site claiming to have the IPv6
|       address [2001:DB8:0:0:8:800:200C:417A], and claiming (by using
|       the '!!'  <path-delimiter>) to have verified that it came from
        old.site.example.

|       [2001:DB8:0:0:8:800:200C:417A] relayed it to "foo-server" which,
        not being convinced that it truly came from
|       [2001:DB8:0:0:8:800:200C:417A], inserted the <path-keyword>
|       "MISMATCH" and then did a reverse lookup on the actual source
        and concluded it was known as bar.isp.example (that is not to
|       say that [2001:DB8:0:0:8:800:200C:417A] was not a correct IPv6
        address for bar.isp.example, but simply that that connection
        could not be substantiated by foo-server). Observe that foo-
        server has now added two entries to the Path.

        "foo-server" is a locally significant name within the complex
        site of many machines run by foo.isp.example, so the latter
|       should have no problem recognizing foo-server and using a '!!'
        <path-delimiter>.  Presumably foo.isp.example then delivered the
        article to its direct clients.

|       It appears that foo-server and barbaz decided to fold the line,
        on the grounds that it seemed to be getting a little too long.

And also several minor consequential changes.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Misc. ABNF nits
Date:  Wed, 01 Jun 2005 20:38:41 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

Alexey Melnikov wrote:

> How is it different from spoofing Received or From headers?

Henry explained that "_" was okay for UUCP hosts names, so my
first idea "get rid of it" was dubious.  But when we want it
for UUCP host names we can't say "MUST be a mailable FQDN".

Okay, USEPRO only says that it's a rule for identities _added_
to the path.  Maybe I didn't get the idea that USEPRO doesn't
cover UUCP (?).  It's just a minor potential inconsistency, I
was curious about the reasoning behind it.  After all the set
of characters allowed in a path-identity isn't arbitrary.  Bye

Date: Wed, 1 Jun 2005 10:28:21 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far, 
Sender: [email-masked]

Harald Tveit Alvestrand <harald@xxxxxxxxxxxxx>:
                                                                                
>>I don't know if you mean "current standards say", "our draft says", "we
>>want to say", or what.
                                                                                
> I thought this was sufficiently clear from the original message:
                                                                                
Until you asked me about what USEFOR currently says, I thought so too.
                                                                                
>I don't see the logic in that. Most of the work of standards is to define
>terms to have a meaning *in the context of the standards* that is not
>identical to an "intuitive" understanding of what the word means.
                                                                                
That's ridiculous. If a term has a common meaning, it is perfectly
acceptable for the standards to use those terms in their common meaning.
In this case, "reply" has a commonly understood english meaning. I'm doing
it right now -- replying to something you wrote. This is a reply. I'm
doing it in email, but if I were doing it in news, it would still be a
reply, and we would also call it a followup. In news, it would have a
References header, and you would have no doubt.
                                                                                
>If we can't define a specific meaning for a word like "followup", we have
>to invent new words like "froobnitz" every time we want a term with a
>specific definition - and I don't see how that aids the clarity of the
>standards.
                                                                                
Taking a word that is already in current usage and redefining it to be
something else certainly does not clarify the standard. People, today,
right now, probably within a second or two of when I write this, are
using the term "followup" to mean "in response to or as a result of". That
part 2 of the FAQ you see today was posted because part 1 was posted.
That's "as a result of". If it contains a References header, then you know
that the poster intended it as a followup. It doesn't get much clearer
than that. It's current practice; let's document it and get on with life.
It's a simple change, or would have been had Charles not gone rampant with
unilateral changes to what wasn't being discussed.
                                                                                
>If you post a reply to my message in news, and don't include a References:
>header, I'm perfectly happy with saying that "this is not a followup as
>defined in USEFOR".                  
                                                                  
I'd be happier with the truth, which is yes, it is a followup but you
cannot detect it as such. I'd be even happier if we can find a
justification so you could identify it as such by requiring a References
header, but it doesn't look likely that we'll ever find an RFC2119
justification for that mandate anymore, so I'll stick with just
documenting existing practice in this area.
                                                                                
And "this is a followup as defined in USEFOR" isn't the actual issue you
asked about. News "Replies" as found in RFC2822 are "followups" as found
in USEFOR, unless you create some really wacky unrealistic definition for
"followup" that doesn't match anything seen so far. If your definition
includes "in response to", then "replies" posted to news are "followups",
and References is optional in those replies, and thus in those followups.

Date: Wed, 1 Jun 2005 10:25:08 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Sender: [email-masked]

"Ruud H.G. van Tol" <rvtol@xxxxxxxxxxxx>:

>To thread on Subject, doesn't mean that only the value of the 
>Subject: header is used. Other information that can be used is 
>the Date and the attributions  ...

Simply remarkable. If it isn't sorting by the Subject header, then it 
isn't "sort by Subject", it's "sort by date" or "make stuff up sorting".

>Threading on Subject, 

Does not exist. Subject is a sortable but not threadable header. There is 
no defined relationship between articles based on the Subject header. Two 
articles with the identical same subject may or may not be related at all, 
much less be parent and child. 

"Ruud H.G. van Tol" <rvtol@xxxxxxxxxxxx>:

>Threading is done that way. By some news reader clients. But I 
>already said that.

No, threading is NOT done that way, by any news reader clients. It's 
called "sorting by Subject", not threading, because there is NO defined 
relationship between articles based on the Subject header. It's a guessing 
game which works sometimes and fails miserably at other times, because 
there is a sometimes similarity in the Subject headers of related 
articles and a sometimes similarity in the Subject headers of unrelated 
articles.

And more than one person has already said that.

>Some people even like it that 
>way, so they activate the option in their news client.

News reading clients are free to SORT articles any way they please. This 
does not mean they are threading them. The problem with what you are 
saying is not that sorting isn't done by some clients that way or that 
people don't like articles sorted that way, it is that you keep claiming 
that this is "threading", and it simply ain't so.

Eivind is right. Listen to him.

Richard Clayton <richard@xxxxxxxxxxxxxx>:

>apologies, I was imprecise I meant "attempt to locate specific thread
>using Subject (fuzzy matching as needed) and place into position using
>other clues such as Date"

You're still only almost right. To locate a SPECIFIC thread you need a
References header. You cannot differentiate between threads by using the 
Subject header, since the Subject header has no defined relationship to a 
thread, and certainly not a specific thread. Using the References header, 
you do not have to guess using "other clues", you have an explicit 
statement as to the parental relationships of articles. A begat B begat C 
begat D ...

> I think it is :)  I like threads. 

First of all, you liking "threads" doesn't make them necessary. And 
second, sorting by Subject has nothing to do with threads, so please stop 
pretending that it does. And third, this issue of the References header 
has nothing at all to do with sorting by Subject, so stop worrying. 

Date: Wed, 01 Jun 2005 17:54:24 +0100
From: Alexey Melnikov <[email-masked]>
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: Misc. ABNF nits
Sender: [email-masked]

Frank Ellermann wrote:

>Charles Lindsey wrote:
>  
>
>>So what is your problem?
>>    
>>
>
>Attackers could try to hide behind a foo_bar.example.com
>path identity, because it doesn't work as an abuse@ FQDN
>  
>
How is it different from spoofing Received or From headers?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Misc. ABNF nits
Date:  Wed, 01 Jun 2005 18:39:59 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 7
Sender: [email-masked]

Charles Lindsey wrote:

> So what is your problem?

Attackers could try to hide behind a foo_bar.example.com
path identity, because it doesn't work as an abuse@ FQDN

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  encoded words in an obs-phrase (was: Differences between RFC 2822 and Usefor)
Date:  Wed, 01 Jun 2005 18:23:46 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 52
Keywords:  =?utf-8?Q?K._Moore?= , =?utf-8?Q?Jonathan_(Jon)_Postel?=
Sender: [email-masked]

Charles Lindsey wrote:

>>  display-name    =       word *(word / "." / CFWS)
>> This syntax would still disallow obscene keywords.

> If we are going to put it into USEFOR (and I think we
> should), then it may as well go into Keywords too. It is
> clearly allowed in Keywords in RFC 2822

Only by a very obs-cure rule, hiding itself as obs-olete,
and I'm not sure how it's supposed to work with RfC 2047:

Keywords: =?utf-8?Q?K._Moore?=

That's illegal, see 2047 5.3.  Dito for a <display-name>.

Normally you can split a phrase without quoted-string at
any word boundary and replace one or more words by an QP
encoded-word.

This obsure RfC 2822 rule messses with specials, STD 11,
RfC 2047, and s-o-1036.

And what about the CFWS in obs-phrase ?  If we'd say...

    phrase =       word *(word / "." / CFWS)

...simple RfC 2047 strategies also fail for the comments:

Keywords: =?utf-8?Q?Jonathan_(Jon)_Postel?=

Okay, that's kind of obvious, RfC 2047 only promises that
you can encode word(s) in a phrase, and "(Jon)" is no word.

Nevertheless, I don't like it.  For the same reasons why I
don't like RfC 2231:  MIME is supposed to be simple, robust,
and compatible.  Any screw after 2049, and it's screwed up.

> come to think of it, I am not aware of any regular news
> software that looks at Keywords at all

Certainly not my UA, unless I enable its "spam analysis"
aka "show headers - all".  It's much less important than
X-Face, and we decided against specifying X-Face... ;-)

> Anyone see any problems with it?

Yes, wasting more than one word about it in USEFOR is a
waste of bandwidth and therefore net abuse. <eg>

                       Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:20978
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Misc. ABNF nits
Date: Wed, 1 Jun 2005 11:16:57 GMT
Lines: 40
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Henry Spencer wrote:
> 
>> Underscores were indeed in use in UUCP names... but there has
>> also been some use of them in domain names, even though it is
>> technically against the rules.

>Only for "mailable FQDN".  Maybe the real problem is the MUST
>in USEPRO and not the underline in USEFOR ?  I like this MUST,
>but if we allow "unmailable" underlines we need a SHOULD.

I copmpletely fail to understand what you are complaining about.

In the Path header, you find <path-identity>s.

A <path-indentity> can be made up of any sequence of letters, digits, '-',
'.', ':' and '_' (except that it must begin with a letter or a digit).

Any arbitrary sequence will do, though some sequences are preferred to
others, and there is a specific REQUIREMENT for the <path-identity>
inserted by the injecting agent.

Some <path-identity>s will be in the form of a domain-name; some won't.
Those that are will clearly not include either of the characters ':' or
'_'. So what? That is no reason to forbid those characters in the other
cases.

So what is your problem?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 1 Jun 2005 15:00:16 +0100
To: John Stanley <[email-masked]>
Cc: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Sender: [email-masked]

In message <[email-masked]>, John
Stanley <[email-masked]> writes

>Richard Clayton <richard@xxxxxxxxxxxxxx>:
>
>>I suspect (guessing here of course) that most of the people who went for
>>#2 would agree that readers often thread on Subject since that's the
>>reality -- but we'd all (another guess) would like to make it far less
>>necessary to do so
>
>There is never a necessity to sort by Subject (not "thread", since Subject 
>isn't a threadable header.). 

apologies, I was imprecise I meant "attempt to locate specific thread
using Subject (fuzzy matching as needed) and place into position using
other clues such as Date"

>If it isn't ever necessary, 

I think it is :)  I like threads. So do almost all the people who use
the software I designed. That's quite a few.

>we cannot make it 
>"far less necessary".

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

To: [email-masked]
Xref: clerew local.usefor:20977
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Differences between RFC 2822 and Usefor
Keywords: is . considered harmful
Date: Wed, 1 Jun 2005 11:00:19 GMT
Lines: 35
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Hiding this in "miscellaneous obsolete tokens" in the chapter
>"obsolete syntax" is more odd than dangerous.  No showstopper
>for USEFOR, we could just redefine the RfC 2822 display-name:

>{2822]
>| display-name    =       phrase
>| phrase          =       1*word / obs-phrase
>| obs-phrase      =       word *(word / "." / CFWS)

>[USEFOR]
>  display-name    =       word *(word / "." / CFWS)

>This syntax would still disallow obscene keywords.

If we are going to put it into USEFOR (and I think we should), then it may
as well go into Keywords too. It is clearly allowed in Keywords in RFC
2822, and I am not aware of any news software that will break if we allow
it too (come to think of it, I am not aware of any regular news software
that looks at Keywords at all).

For good measure, I have included such a Keywords header in this message.
Anyone see any problems with it?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:20976
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles H. Lindsey" <[email-masked]>
Subject: Re: Differences between RFC 2822 and Usefor
Date: Wed, 1 Jun 2005 10:52:24 GMT
Lines: 72
Sender: [email-masked]

In <7720B187EEDCD0754BC8E464@B50854F0A9192E8EC6CDA126> Harald Tveit Alvestrand <[email-masked]> writes:

>--On 30. mai 2005 13:45 +0000 Charles Lindsey <[email-masked]> wrote:

>> It was a newly introduced "feature" of RFC 2822, on a MUST accept but do
>> not generate yet basis. It is widely agreed that calling it "obs-" was a
>> mistake, because it was very much intended for future use. Many agents
>> accept it already. Bruce Lilly asked us to put it in (and for once he was
>> right).

>hey - where did that come from??????

It was hidden away in a section entitled "Obsolete Syntax", and so a lot
of people (myself included) never spotted it.

There was some discussion on the ietf-822 list around 12 months ago,
involving Bruce Lilly, Pete Resnick and others, in which all (Pete
included) agreed that hiding it there was a mistake, and that it ought to
be brought out and called <extended-phrase> if and when a 2822bis
happened.

>the text in RFC 2822 section 4.1 is:

>>    Note: The "period" (or "full stop") character (".") in obs-phrase is
>>    not a form that was allowed in earlier versions of this or any other
>>    standard.  Period (nor any other character from specials) was not
>>    allowed in phrase because it introduced a parsing difficulty
>>    distinguishing between phrases and portions of an addr-spec (see
>>    section 4.4).  It appears here because the period character is
>>    currently used in many messages in the display-name portion of
>>    addresses, especially for initials in names, and therefore must be
>>    interpreted properly.  In the future, period may appear in the
>>    regular syntax of phrase.

>The way I remember the discussions (after 4-10 years, I'm not that sure any 
>more...), people observed that this period was used quite widely, so 
>parsers that refused to handle it were not terribly useful, but the WG 
>refused to recommend it as part of existing syntax.

But clearly recommended it as part of the MUST accept syntax. The wording
above is a classic example of "Here is a mew feature; compliant software
MUST accept it now, but it MUST/SHOULD/whatever NOT be generated yet" (for
some value of "yet", which apparently included regularizing it in a future
standard).

Actually, I doubt any current email software refuses to tramsit and
display such a thing once it get onto the wire. I could well believe that
some software might be confused if asked to reply to a From: header with
it in (I have tried it in this message, as you will see - feel free to try
to reply to it).

Anyway, for News, Bruce Lilly pointed out some years ago that we should
adopt this new "feature", and it duly went into our draft (but entitled
<extended-phrase>). It does not seem to have made it into USEFOR yet, but I
think it should.

>Illustrates the danger of future-looking statements in standards that we're 
>arguing based on this text 4 years later.....

It illustrates the danger of putting important things in places where
nobody will notice them :-( .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Date: Wed, 1 Jun 2005 11:56:52 +0200
Organization: Chaos rules.
Sender: [email-masked]

Eivind Tagseth:
> Ruud H.G. van Tol:
>> Eivind Tagseth:
>>> Ruud H.G. van Tol:

>>>> Threading on Subject, is also used to explain why some users see a
>>>> new thread when others don't. In that case the change in Subject
>>>> overrules the References.

>>> The subject never overrules the References header.

>> I just mentioned when it does. Some people even like it that
>> way, so they activate the option in their news client. Some
>> news clients even came with that option activated. There may
>> even be news clients that do it and that can not be told not
>> to do it.

> It still doesn't overrule the References header. 

Aaargh. The particular news reader shows two threads where 
the References header represents one. You just chose another 
interpretation of 'overrules' than the one I wrote about, 
and then say that it doesn't. Let's please leave it at that.

-- 
Grtz, Ruud

Date: Wed, 1 Jun 2005 10:27:02 +0200
From: Eivind Tagseth <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Mail-Followup-To: Eivind Tagseth <[email-masked]>, [email-masked]
Sender: [email-masked]

* Ruud H.G. van Tol <[email-masked]> [2005-06-01 10:14:59 +0200]:
> Eivind Tagseth:
> > Ruud H.G. van Tol:

> >> Threading on Subject, is also used to explain why some users see a
> >> new thread when others don't. In that case the change in Subject
> >> overrules the References.
> 
> > The subject never overrules the References header.
> 
> I just mentioned when it does. Some people even like it that 
> way, so they activate the option in their news client. Some 
> news clients even came with that option activated. There may 
> even be news clients that do it and that can not be told not 
> to do it.

It still doesn't overrule the References header.  This is why it is 
important to describe the References header for what it _means_ not
how some people expect news readers to _display_ it.

The references header describes the parental relation between the
article and it's precursors, the subject header can never override 
this.  

A followup agent has to obey some rules when following up to
an article with a references header, the subject header can never
override this.  If a followup agent ignores the references header
when creating a followup just because the subject has changed, it
would be broken.

If a newsreader chooses to display an article with a references header
as a new thread, just because the subject was different than it's
parent, then it is free to do so, since the references header
shouldn't put requirements on how newsreaders should display it's 
meaning to the user.

Eivind

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Date: Wed, 1 Jun 2005 10:14:59 +0200
Organization: Chaos rules.
Sender: [email-masked]

Eivind Tagseth:
> Ruud H.G. van Tol:
>> John Stanley:

>>> There is never a necessity to sort by Subject (not "thread", since
>>> Subject isn't a threadable header.). If it isn't ever necessary, we
>>> cannot make it "far less necessary".

>> To thread on Subject, doesn't mean that only the value of the
>> Subject: header is used. Other information that can be used is
>> the Date and the attributions (and further whatever the author
>> of the news client chooses as feasible).

> That's not threading, that's guessing.

Threading is done that way. By some news reader clients. But I 
already said that. I didn't say that I like it.

>> Threading on Subject, is also used to explain why some users see a
>> new thread when others don't. In that case the change in Subject
>> overrules the References.

> The subject never overrules the References header.

I just mentioned when it does. Some people even like it that 
way, so they activate the option in their news client. Some 
news clients even came with that option activated. There may 
even be news clients that do it and that can not be told not 
to do it.

-- 
Grtz, Ruud

Date: Wed, 1 Jun 2005 09:21:58 +0200
From: Eivind Tagseth <[email-masked]>
To: [email-masked]
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Mail-Followup-To: Eivind Tagseth <[email-masked]>, [email-masked]
Sender: [email-masked]

* Ruud H.G. van Tol <[email-masked]> [2005-06-01 08:21:23 +0200]:
> John Stanley:

> > There is never a necessity to sort by Subject (not "thread", since
> > Subject isn't a threadable header.). If it isn't ever necessary, we
> > cannot make it "far less necessary".
> 
> To thread on Subject, doesn't mean that only the value of the 
> Subject: header is used. Other information that can be used is 
> the Date and the attributions (and further whatever the author 
> of the news client chooses as feasible).

That's not threading, that's guessing.

> Threading on Subject, is also used to explain why some users see a 
> new thread when others don't. In that case the change in Subject 
> overrules the References.

The subject never overrules the References header.  The newsreader may be set
up to show subthreads with a Subject change as if they were a new thread, but
that doesn't change the information the References header is conveying, and
it doesn't change how the references header should be treaded when following
up to an article in the subthread.

Eivind

Date: Wed, 01 Jun 2005 09:22:41 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: John Stanley <[email-masked]>, [email-masked]
Subject: Re: #1008 USEFOR 1.5 "followup": summary so far
Sender: [email-masked]

--On tirsdag, mai 31, 2005 11:25:58 -0700 John Stanley <[email-masked]> 
wrote:

> What is really confusing about your mixmaster approach to this is that
> when you say:
>
> 1) All "followups" have a References: header.
>
> I don't know if you mean "current standards say", "our draft says", "we
> want to say", or what.

I thought this was sufficiently clear from the original message:

> --------------------------------------------------------------------
> Subject: USEFOR 1.5: relation between a followup and a References: header?
>
> USEFOR version -04 states:
>
>  A "followup" is an article containing a response to the contents of
>  an earlier article (the followup's "precursor" indicated in the
>  "References" header).
>
>  Discussion has revealed two possibilities:
> - All "followup"s (as defined here) have a References: header.
> - There can be articles that are a "followup" that do not have
> "References:" headers
>
> The other half of the problem is whether there can be articles that
> have References: headers which are not "followups".
> ---------------------------------------------------------------------
> I'd like to have the 11 or so people who have spoken up on the issue (and
> anyone who cares to speak up otherwise) identify themselves as having one
> of four positions on what USEFOR should say about this issue:

I'm only talking about what USEFOR should say.
This group's mandate is (among other things) getting USEFOR out the door. 
We can't change what people are currently doing, we can't change what other 
standards say - but we CAN change what USEFOR says.

As to your comment:

> As for RFC2822 not using the specific term "followup": been there, covered
> that. It uses the generic term "replies". Every reply posted as news is a
> "followup".  Every one. Each time someone replies, in a news context, they
> are posting an article that is "a response to another". If you can find a
> single person here who would call an article I post, in response to
> another, which says "you are wrong when you say X because..." NOT a
> followup, please trot them out. I don't believe they exist. And if they
> think it is not a reply, then they are clearly deluded.

I don't see the logic in that. Most of the work of standards is to define 
terms to have a meaning *in the context of the standards* that is not 
identical to an "intuitive" understanding of what the word means.
If we can't define a specific meaning for a word like "followup", we have 
to invent new words like "froobnitz" every time we want a term with a 
specific definition - and I don't see how that aids the clarity of the 
standards.

If you post a reply to my message in news, and don't include a References: 
header, I'm perfectly happy with saying that "this is not a followup as 
defined in USEFOR".

If that's the consensus position of the group.

                                 Harald