Subject: RE: Additional thoughts on text transfers

From: Richard Whalen <[email-masked]>
Date: Tue, 2 Apr 2002 09:00:25 -0500

> -----Original Message-----
> From: Markus Friedl [mailto:[email-masked]]
> Sent: Tuesday, April 02, 2002 3:11 AM
> To: Jeffrey Altman
> Cc: Damien Miller; Richard Whalen; [email-masked]'
> Subject: Re: Additional thoughts on text transfers
> 
> 
> On Tue, Apr 02, 2002 at 12:16:19AM -0500, Jeffrey Altman wrote:
> > > That kinda proves the point that I was trying to make - 
> that the format
> > > decision and processing can be built into the clients (or 
> servers) and not
> > > into the protocol itself.
> > 
> > No it doesn't.  It provies that it must be built into the protocol.
> 
> Then why does everybody write lines and lines of email instead of
> writing a draft using the mechanisms provided by
> 	draft-ietf-secsh-filexfer-02.txt
> 

The mechanisms provided by draft-ietf-secsh-filexfer-02.txt fall short of
what is needed to provide a text transfer mechanism.

The file attributes mechanism has plenty of flexibilty to present
information about the type of information in the file, and whether or not
any translations can be provided.  Right now it says nothing about the type
and that no translations are available.

The OPEN/READ/WRITE mechanism is where the problem lies.  It specifies
particular bytes from the file, and on some systems those bytes may contain
a mixture of actual data and system specific details.  When those system
specific details are transfered to a dissimilar system they become at best
useless. In the worst case they make the entire file useless, as programs on
the second system do not know to ignore the system specific details and
declare the file to be corrupt.

----------------------
Richard Whalen
Process Software

---

From: Markus Friedl <[email-masked]>
Date: Tue, 2 Apr 2002 16:13:01 +0200

On Tue, Apr 02, 2002 at 09:00:25AM -0500, Richard Whalen wrote:
> a mixture of actual data and system specific details.

But when the server sends different data in READ replies (depending on
the client system) then the data get be corrupted, i.e. the MD5 of the
data would not match among different clients.

---

From: nico <[email-masked]>
Date: Wed, 3 Apr 2002 22:05:21 -0500

> and that no translations are available.

I don't see why the attributes wouldn't suffice for the case of ASCII
text files:

Unix client: open(some file, write|create|trunc, attrs: {..., content-type:
			text, encoding: ASCII, eol: LF, ...}, ...)

VMS server: OK, handle

client: write(handle, ...)
client: write(handle, ...)
..
server: OK
server: OK
..
client: close(handle)
server: OK
server: <create/truncate the file, write text sans LF and marking the
	 record boundaries where the LF characters were in the original>

client: open(some other file, read)
server: OK, handle
client: stat(handle)
server: OK, attrs: ... content-type: test, encoding: ASCII, eol: X, ...
client: read(handle,...)
..
client: close(handle)
server: OK

Note that the VMS server has to select some sort of EOL marker,
preferably LF :)

> declare the file to be corrupt.

If the issue is primarily about VMS text files being record-oriented
with variable length records, one record per line with no EOL
characters, I think that problem can be dealt with as above.

But if you want to generally support record-oriented files, both with
fixed- and variable-sized records, then the file xfer protocol does,
indeed, need to specifically support such semantics. But then, the
necessary changes might be few, particularly if random-access can be
maintained even when reading/writing files with variable record sizes.

A canonical file format (preferably as a stream of octets) is needed for
checksumming.

Perhaps SFTP should move to a different working group :)

Cheers,

Nico

---

From: Simon Tatham <[email-masked]>
Date: Thu, 04 Apr 2002 10:06:29 +0100

> I don't see why the attributes wouldn't suffice for the case of ASCII
[...]
> 	 record boundaries where the LF characters were in the original>

If I'm reading that right, you seem to be suggesting that the server
should store the whole file in memory until the client closes it.
Wouldn't this be hideously unpleasant to do with big files?

Simon
-- 
Simon Tatham         "Imagine what the world would be like if
<[email-masked]>    there were no hypothetical situations..."

---

From: Damien Miller <[email-masked]>
Date: Fri, 5 Apr 2002 23:49:46 +1000 (EST)

> what is needed to provide a text transfer mechanism.

Huh? The draft includes a clear and easy to use extension mechanism,
what is wrong with that?

-d