Subject: Message fragmentation via message/partial and news-specific header fields

From: Charles Lindsey <[email-masked]>
Date: Tue, 14 Sep 2004 14:22:32 +0100 (BST)

This message, posted to the old list, is now reposted to the new list for
the benefit of anyone no longer reading the old one.

>From:  "Charles Lindsey" <[email-masked]>
>Subject: Re: Message fragmentation via message/partial and news-specific header fields
>Message-ID: <[email-masked]>
>X-Newsreader: NN version 6.5.2 (NOV)
>References: <[email-masked]>
>Date: Thu, 2 Sep 2004 16:05:18 GMT

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>We had had some discussion about message fragmentation via the MIME media type
>message/partial,, and there was some text in earlier drafts, however the
>matter is conspicuously absent from the recently-produced usefor-usepro-00
>draft.

I think it is a matter for usefor rather than usepro, and yes it needs
some mention there.

>First there are some questions that need to be considered:
>1. a) Where is message fragmentation expected to occur (e.g. during transport
>   between transfer agents when it is recognized that the receiving agent
>   has some message size limit), and b) by what criteria (presumably some
>   site-dependent size limit, but how is that communicated to that site's
>   feeding agents)?

At the injecting agent, at the latest.

>2. Where does reassembly take place?

At each recipients reading agent, or later (i.e. he manually feeds it into
a reassembly program, since many reading agents are unlikely to provide the
facility).

>3. How is multiple fragmentation to be handled? [e.g. a 1GB original
>   message might be fragmented into 100 MB pieces at some point, which
>   might then be further fragmented into 10 MB pieces, etc.  That means
>   that a given article may appear multiple times at various places in
>   the network; unfragmented, and broken into various different-sized
>   pieces -- the original message and the various pieces each having
>   separate, unique message identifiers.)

It MUST NOT happen. Relaying agents MUST NOT modify articles in this
manner as they pass through them. Either they pass them unchanged, or they
drop them on the floor (and it it floods around via some slower path, then
well and good). But it is a fundamental principle of Usenet that all copies
of a particular article are supposed to be identical, apart from variant
headers (though if different copies have passed via different gateways,
that might not be possible).

As far as transport and serving within Usenet is concerned, each part
is a distinct article with its own distinct msg-id.

>4. a) During fragmentation, which message header fields are to be copied
>   to the enclosing message header, and which are to be preserved in
>   the enclosed first part message/partial header?  b) Which fields
>   need to be copied to the headers of each of the subsequent message
>   fragments?
>5. During reassembly, which fields of the first part message header
>   are to be retained, which are to be discarded; likewise for the
>   fields in the header of the enclosed first part?

>Specifically regarding question 5, I expect that Approved, Newsgroups,
>Distribution, Path, Followup-To, Expires, Summary, Organization,
>Injection-Date, Injector-Info, and User-Agent can be safely copied to
>the header of the first part and at least Approved, Newsgroups,
>Distribution, Path, Injection-Date, and Expires should be copied to
>the headers of subsequent parts.  Copying these fields to the first
>part is expected behavior when fragmenting per RFC 2046 as amended by
>RFC 3798.  I believe there may be difficulties with Control,
>Supersedes, Archive, Posted-And-Mailed, Mail-Copies-To, and possibly
>Complaints-To.

No, you certainly don't copy Path. It grows as the various parts propagate
through the network, and different parts may arrive at a given host via
different routes. OTOH, any pre-injection portion of the Path, including
the tail entry, present before the split might well be retained in the
Paths of all the parts.

Likewise, Injection-Date and Injection-Info could well be different for
the various parts, depending on where the splitting was done.

The rest of your suggestions seem about right.

Incidentally, what happens with Received headers in email? Presumably the
final reassembled message sees the Received headers from the first part,
and all the rest of them get lost.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Russ Allbery <[email-masked]>
Date: Tue, 14 Sep 2004 11:03:29 -0700

Charles Lindsey <[email-masked]> writes:

> This message, posted to the old list, is now reposted to the new list
> for the benefit of anyone no longer reading the old one.

This is the first time I've seen this message, so either the old list is
no longer working or I've been silently unsubscribed from it for some
reason.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>