Subject: ISSUE: Format of batched news in response to sendme

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 14 Feb 2007 14:35:54 GMT

In response to a sendme control message, an article is sent, normally
using the news transport mechanism (typically UUCP in such cases). I.e.,
it is on the same "wire" as the sendme control message. Therefore it is in
order to document the (somewhat bizarre) format, as was done in
Son-of-1036 and in Usepro-06. It is documented nowhere else.

Note that, as compared to Usepro-06, some brief mention of variants of the
format for compressed batches would be in order.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 15 Feb 2007 07:34:20 +0100

Second, please.

The -06 specification of the batch format was incomplete; -07 says that it 
is a private matter.

Is ihave/sendme obsolete or not?

--On 14. februar 2007 14:35 +0000 Charles Lindsey <[email-masked]> 
wrote:

> Charles H. Lindsey ---------At Home, doing my own
> thing------------------------ Tel: +44 161 436 6131 Fax: +44 161 436 6133
> Web: http://www.cs.man.ac.uk/~chl Email: [email-masked]      Snail:
> 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K. PGP: 2C15F1A9      Fingerprint:
> 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
>
>

---

From: Russ Allbery <[email-masked]>
Date: Wed, 14 Feb 2007 22:57:29 -0800

> The -06 specification of the batch format was incomplete; -07 says that
> it is a private matter.

The specification of ihave/sendme has always left unspecified how the
messages themselves are conveyed.  RFC 1036 says only:

    This message is part of the ihave/sendme protocol, which allows one
    host (say A) to tell another host (B) that a particular message has
    been received on A.  Suppose that host A receives message
    "<[email-masked]>", and wishes to transmit the message to
    host B.

    A sends the control message "ihave <[email-masked]> A" to
    host B (by posting it to newsgroup to.B).  B responds with the
    control message "sendme <[email-masked]> B" (on newsgroup
    to.A), if it has not already received the message.  Upon receiving
    the sendme message, A sends the message to B.

Son-of-1036 says only:

          The  ihave  message  states that the named relayer has filed
          articles with the specified message IDs,  which  may  be  of
          interest to the relayer(s) receiving the ihave message.  The
          sendme message requests that the relayer receiving  it  send
          the  articles  having the specified message IDs to the named
          relayer.

Both make further passing mentions to UUCP, but neither specify how the
messages be sent.  As far as the ihave/sendme specification is concerned,
the messages could be conveyed by FTP or by HTTP PUT.  More to the point,
they could be sent as separate messages without encoding rather than using
a batch format if two peers so chose, or as mbox files, or as a tar.gz
file, or anything else that the other side knew how to process.

I believe their use has been exclusively or nearly exclusively in
conjunction with UUCP batching because UUCP batching is what news servers
have historically implemented, but neither previous formal specification
has explicitly linked those two things.  I think that's for valid reasons;
there really isn't any justification that I can see to constrain the
method of transmission of the articles, and setting up such a feed
requires out-of-protocol arrangements between peers anyway.

That leaves documentation of the batch format as only useful as an aid for
implementors who want to write a new implementation of UUCP batching and
transmission, and I have no idea why one would want to do such a thing at
this point beyond just for the fun of it.  If someone really feels
inspired to document it, perhaps just for historical curiosity, I'd think
that they'd want to include the gunbatch, cunbatch, and c7unbatch
compression and encoding formats as well.  The whole topic would make for
a simple and self-contained informational RFC that those who were curious
could read.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 19 Feb 2007 16:12:32 GMT

>          the  articles  having the specified message IDs to the named

OK, I have been poking around inside CNews to see exactly what happens
(the source code of CNews is probably the best indication of how this
protocol used to work - and probably still does in a few places).

The Ihave and Sendme control messages are normal articles, except that
they are propagated according to the dreadful "To." hack in the Newsgroups
header (we _have_ documented that). In response to Sendme, the server is
supposed to send the requested article(s), and it does so by using the
normal News Transport mechanism (the same as for the Ihave and Sendme
messages themselves. But what is the "normal News Transport mechanism"?

As far as Cnews is concerned, it is configured with a list of hosts that
it sends stuff to, together with the "mechanism" to be used for each. This
might be UUCP, NNTP, mail2news or any other hack privately agreed.
Bascially, they will all use the basic format (as now defined by USEFOR),
augmented by whatever the particular mechanisn needs. And, as I now see,
it is only those mechanisms designed for UUCP that prepend those lines
starting with "#!rnews..." of "#!cunbatch...", and possibly combining
articles into batches as well.

So, to take an absurd but instructive example, if site A sent an Ihave
control message to site B via NNTP, and it site B understood the message
and played along by returning a Sendme, then site A would respond by
sending the requested article(s) to site B by NNTP, by generating a
suitable NNTP IHAVE command (or just conceivably a POST command). So it
would all work but no "#!rnews..." line would be involved.

And indeed, I now see that the description of the "#!rnews..." stuff,
which does indeed appear in Son-of-1036, is not specifially related to use
with Ihave/Sendme, but merely describes a general transport mechanism for
all purposes (including sending the Ihave/Sendme messages in the first
place).

So, on that basis, I withdraw my Issue request.