Subject: RE: filexfer-07

From: "Richard Whalen" <[email-masked]>
Date: Fri, 1 Apr 2005 17:24:44 -0500

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On
> Behalf Of Joseph Galbraith
> Sent: Friday, April 01, 2005 4:59 PM
> To: der Mouse
> Cc: [email-masked]
> Subject: Re: filexfer-07
> 
> > 7.1.1.3, describing SSH_FXF_ACCESS_READ_LOCK: s/gaurantee/guarantee/
> > (twice).  Also, it's not clear whether "the exclusive reader" means
> > with respect to other sftp clients or with respect to all other OS
> > activity; and, an exclusive read lock is a very peculiar 
> thing, and not
> > at all what a "read lock" normally does.
> 
> All right; I renamed it to SSH_FXF_ACCESS_EXCLUSIVE_READ,
> and changed it to say:
> 
>    The server MUST guarantee that no other handle has been
>    opened with ACE4_READ_DATA access, and that no other
>    handle will be opened with ACE4_READ_DATA access until
>    the client closes the handle.  (This MUST apply both
>    to other clients and to other processes on the server.)
> 
>    If there is a conflicting lock the server MUST return
>    SSH_FX_LOCK_CONFLICT.  If the server cannot make the locking
>    guarantee, it MUST return SSH_FX_OP_UNSUPPORTED.
> 
>    Other handles MAY be opened for ACE4_WRITE_DATA or any other
>    access, as long as it does not include ACE4_READ_DATA.
> 
> Is this better?
> 

I think that the lock described is still pretty unusual for a file.  When accessing a file the usual methods of locking are:

None - I don't care if there is other current access, I'll take my chances if something modifies it while I'm writing/reading it.

Read - I don't want anyone to modify the file (or its attributes) while I'm reading it.  I don't care about other readers.

Write - I don't want anyone else to be accessing the file while I access it for write. I don't want to risk others getting a corrupt file while I'm changing it.

---

From: Joseph Galbraith <[email-masked]>
Date: Fri, 01 Apr 2005 15:45:21 -0700

Richard Whalen wrote:
> I think that the lock described is still pretty unusual for a file.

It reflects the windows file sharing model.

In windows, you say I'm willing to let others READ, write or
delete (or any combination thereof) this file while I'm
accessing it.

Hence, EXCLUSIVE_READ is I'm willing to let other WRITE or
DELETE this file.

 > When accessing a file the usual methods of locking are:
> 
> None - I don't care if there is other current access, I'll
 > take my chances if something modifies it while I'm
 > writing/reading it.

This is no locking bit set.

> Read - I don't want anyone to modify the file (or its
 > attributes) while I'm reading it.  I don't care about
 > other readers.

This is EXCLUSIVE_WRITE set (and probably DELETE_LOCK)

> Write - I don't want anyone else to be accessing the
 > file while I access it for write. I don't want to
 > risk others getting a corrupt file while I'm changing it.

This is EXCLUSIVE_WRITE|EXCLUSIVE_READ (and possibly
DELETE_LOCK)

The only potential problem is that you may not be able
to support EXCLUSIVE_READ set by itself-- in which case
you'd have to return UNSUPPORTED for that lock request.

And unfortunately, that limitation can't be expressed
in the "supported2" extension.

I'll add a note that clients that use the locking
bits might need to watch out for this.

(I expect most clients will want the default
unix behavior of no bits set.)

Do you have any other suggestions of how to accomadate both
models?

Thanks,

Joseph

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Fri, 01 Apr 2005 18:14:12 -0500

On Friday, April 01, 2005 05:24:44 PM -0500 Richard Whalen 
<[email-masked]> wrote:

> I think that the lock described is still pretty unusual for a file.  When
> accessing a file the usual methods of locking are:
>
> None - I don't care if there is other current access, I'll take my
> chances if something modifies it while I'm writing/reading it.
>
> Read - I don't want anyone to modify the file (or its attributes) while
> I'm reading it.  I don't care about other readers.
>
> Write - I don't want anyone else to be accessing the file while I access
> it for write. I don't want to risk others getting a corrupt file while
> I'm changing it.

I agree, the locking semantics the document describes are pretty bizarre. 
In general, a read lock excludes writers, while a write lock excludes both 
readers and other writers.

Also, it is worth noting that the current text, especially with the 
requirement that locks MUST apply to other processes on the server, 
basically requires the use of operating-system-provided mandatory file 
locking.  This is problematic because on some systems, mandatory locking is 
rarely used, if it is available at all, and the sftp server may not be in a 
position to influence whether mandatory locking is used for a particular 
file.

-- Jeffrey T. Hutzelman (N3NHS) <[email-masked]>
   Sr. Research Systems Programmer
   School of Computer Science - Research Computing Facility
   Carnegie Mellon University - Pittsburgh, PA

---

From: Joseph Galbraith <[email-masked]>
Date: Fri, 01 Apr 2005 17:01:14 -0700

Jeffrey Hutzelman wrote:
> I agree, the locking semantics the document describes are pretty 
> bizarre. In general, a read lock excludes writers, while a write lock 
> excludes both readers and other writers.

The locking model you describe is support by the locking
semantics in the document, and are probably the most
useful.

But, where being able to control all 8 locking modes supported
by the document is most useful is when SFTP is being
used as remote filesystem protocol (of which we have
two implementations that I know of-- so this isn't just
a theoritical use case.)

This allows correct behavior when the client and server
support the same lock model, and reasonable fall back
behavior (by allowing the client to mask off unsupported
bits.)

The only thing that concerns me here is the pontential
for servers to support EXCLUSE_READ|EXCLUSIVE_WRITE, but not
EXCLUSIVE_READ by itself, since that can't be expressed in
the supported2 extension.

I'm tempted to go to a 3-bit enum instead, and have a
locking modes byte in the supported2 extension with
one bit set for each mode supported.  But I'm not sure
it is worth it.

What if we were to call them BLOCK_ instead of LOCK_;
that might not intefere with pre-existing terminalogy
and still expresses the meaning well.  (That is block
as in prevent, not B-LOCK.)

As flags, they'd be:

BLOCK_READ
BLOCK_WRITE
BLOCK_DELETE

And as enums, they'd be:

BLOCK_NONE
BLOCK_READ
BLOCK_WRITE
BLOCK_READ_WRITE
BLOCK_DELETE
BLOCK_DELETE_READ
BLOCK_DELETE_WRITE
BLOCK_DELETE_READ_WRITE

and in supported2:
   byte block-mode
     If bit 0 is on, block mode 0 (BLOCK_NONE) is supported,
     if bit 1 is on, block mode 1 (BLOCK_READ) is supported,
     and so on.

> requirement that locks MUST apply to other processes on the server, 

Hmmm... I'm not sure how useful such a lock is if it doesn't
apply to other process on the server.  How do you think such
a lock would be useful?

> locking.  This is problematic because on some systems, mandatory locking 
> is rarely used, if it is available at all, and the sftp server may not 
> be in a position to influence whether mandatory locking is used for a 
> particular file.

Is there some other locking model you'd like to see?

I expect locking for most FTP-style clients to
be either unwanted or 'nice to have.'  In the
unwanted case, they just don't set any lock
bits.

In the 'nice to have' category, they set them and then
mask against the 'supported2' open flags field to turn
them off if the server doesn't support them.

For example, downloaders will probably want either
no locking or EXCLUSIVE_WRITE, meaning don't let
anybody else write the file while I download it.
(I don't like these names... not sure whether to go
back to the LOCK names or if something else is better.)

Of course, a downloader that asks for EXCLUSIVE_WRITE
won't be able to download an active log file, for
example.  (So probably, it is either optional or a
downloader really wants no locking.)

An uploader is different... they might want
EXLUSIVE_WRITE|EXCLUSIVE_READ (in other words,
don't let anyone else much with it while I upload,
and until I'm done, don't let anyone read the partial
file either.)

But, I suspect that most uploaders will probably not
mind if the server doens't honor that request... in
which case they just allow the 'supported2' mask
to clear the bits.  (This is the case today where
most unix servers don't do any file locking.)

Thanks,

Joseph