Subject: re: Compressed Data

From: [email-masked]
Date: Thu, 26 Oct 2000 10:12:39 -0400

[email-masked] wrote:

>Forwarded from an external source for discussion in case anyone has any
thoughts...

>>>>As a heads up, the French and Norwegian delegates are interested in a
>>>>change to the SMIME RFC. They have said they will propose separately an
>>>>extension of the ID for a compressedData S/MIME content type. Extension
>>>>consists of an additional field to indicate the uncompressed size of the
>>>>encapsulated content. I don't know how soon this will happen.
>>
>>Hmm, I can see some problems with this, it assumes you know in advance how
long
>>the data stream will be which isn't always the case.  Adding an INTEGER
>>OPTIONAL is pretty easy, but I wouldn't want to mandate its use because it'll
>>cause problems for any streaming implementations (for example one of the uses
I
>>mentioned a while back was for securing EDI messages which can be ~100MB long,
>>there's no way to tell in advance just how long they'll be because the systems
>>processing the data can't afford to spool 100MB of data to disk while they
wait
>>for the end to appear).  I don't have a problem with adding something like
>>this, I just wouldn't want to mandate it.

   Couldn't the data be broken into smaller chunks (e.g. spooling 1 MB instead
of 100MB at a time), and the uncompressed (and also compressed) size of each
chunk be indicated?  The recipient could then add the uncompressed sizes of all
the chunks to get the size of the original (uncompressed) data.  Of course, they
would need a way to find all the size specifiers relatively quickly, and a count
of all the chunks would be nice too...

   Networking protocols like TCP/IP and IPX/SPX typically break large blocks of
data up into manageable chunks, which they call "packets", but they also often
have to deal with packets arriving (or not arriving) in arbitrary sequence,
which (thank goodness) is not an issue in this case.

   One thing I'm wondering is how the data compression algorithm will be
specified.  I assume that new OIDs will have to be defined for each, if such
don't already exist.

   I am also wondering if there would be a way to generate clear-signed
compressed content;  that is, if a way could be found to incorporate the
compressed data into a MIME entity which is then signed as part of a
multipart/signed message, and which could be read (and decompressed) by MIME
parsers without S/MIME capability.  A simple mechanism would be to put the data
into a .zip or .tar file attachment and specify that its content disposition is
"inline", but I'm sure better ways are possible.

---

From: "Bonatti, Chris" <[email-masked]>
Date: Thu, 26 Oct 2000 23:49:33 +0200

I should think that in many cases this value could be read from the operating
system, or otherwise known.  Since the size determination would be on the send side,
you're not generally counting data streaming in by the packet.

Of course, somebody will have an exception, but as long as it's optional it seems
like a good idea.

Chris B.

_________________________