Subject: RE: filexfer-07

From: "Richard Whalen" <[email-masked]>
Date: Mon, 4 Apr 2005 08:55:08 -0400

> -----Original Message-----
> From: Joseph Galbraith [mailto:[email-masked]]
> Sent: Friday, April 01, 2005 7:01 PM
> To: Jeffrey Hutzelman
> Cc: Richard Whalen; der Mouse; [email-masked]
> Subject: Re: filexfer-07
> 
> 
:
> 
> What if we were to call them BLOCK_ instead of LOCK_;
> that might not intefere with pre-existing terminalogy
> and still expresses the meaning well.  (That is block
> as in prevent, not B-LOCK.)
> 
> As flags, they'd be:
> 
> BLOCK_READ
> BLOCK_WRITE
> BLOCK_DELETE
> 
> And as enums, they'd be:
> 
> BLOCK_NONE
> BLOCK_READ
> BLOCK_WRITE
> BLOCK_READ_WRITE
> BLOCK_DELETE
> BLOCK_DELETE_READ
> BLOCK_DELETE_WRITE
> BLOCK_DELETE_READ_WRITE
> 
> and in supported2:
>    byte block-mode
>      If bit 0 is on, block mode 0 (BLOCK_NONE) is supported,
>      if bit 1 is on, block mode 1 (BLOCK_READ) is supported,
>      and so on.
> 
> 

I like the idea of naming the flags BLOCK_, as it states the
actual intention and removes some of the confusion that is out
there in the names that various operating systems use.  It will
also make implementors work to understand what they need to do
rather than just match it to the similar sounding name on the
operating system (and possibly have it wrong).

Richard Whalen
Process Software

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Mon, 04 Apr 2005 16:30:34 -0400

> operating system (and possibly have it wrong).

Me too.

I am still a little concerned about the specification of only mandatory 
locking, when there are common server platforms on which mandatory locking 
is used infrequently, if at all, while advisory locking is commonplace.

-- Jeff

---

From: Joseph Galbraith <[email-masked]>
Date: Mon, 04 Apr 2005 16:01:33 -0600

Jeffrey Hutzelman wrote:
> locking, when there are common server platforms on which mandatory 
> locking is used infrequently, if at all, while advisory locking is 
> commonplace.

Hmmm...

I'm not terribly familar with the advisory locking implemenated
on most unix platforms (I could dig, I suppose, but that would
take me some time.)

I presume it has basically three levels of locking,
none, read (don't allow writers) and write (exclusive.)

I presume the lock request fails if there is already
a violation?

What happens when a new access request comes along
that violates the advisory lock?  Is the original
lock holder notified in some fashion?

Is it appropraite to lock a open time, or should it
be a seperate request?

Thanks,

Joseph

---

From: Derek Fawcus <[email-masked]>
Date: Mon, 4 Apr 2005 23:28:51 +0100

> is used infrequently, if at all, while advisory locking is commonplace.

Agreed.

It also seems rather pointless to mandate mandatory locking.  Some unices
do not support it at all.  Which means if they support the protocol locking
commands,  they'll simply be lying about the ability of the lock to exclude
local (non SFTP) access.

However (depending upon the use) this may actually be valid.  I do tend to
prefer advisory lock,  simply because then I can peek under the covers to
check what running apps are doing.

DF

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Mon, 04 Apr 2005 18:39:05 -0400

> none, read (don't allow writers) and write (exclusive.)

Correct.  The difference is pretty simple.  An advisory lock prevents other 
processes from obtaining conflicting locks.  A mandatory lock also prevents 
other processes from performing conflicting file accesses.

On most Unix systems, an SFTP server can guarantee to its client that I 
can't get a lock on a file, but it can't guarantee that I won't just write 
to the file without bothering to get a lock.

-- Jeff

---

From: Bill Sommerfeld <[email-masked]>
Date: Mon, 04 Apr 2005 18:54:18 -0400

> to the file without bothering to get a lock.

unix systems which implement mandatory file locking (typically
SVR4-derived) typically do so on a file-by-file basis -- one particular
combination of file mode bits was recast to mean that  normally-advisory
byte-range locks become mandatory, blocking attempts to do I/O in
conflict with the lock.  this prevents you from mixing advisory and
mandatory locks on the same file.

							- Bill

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Mon, 04 Apr 2005 19:07:07 -0400

On Monday, April 04, 2005 06:54:18 PM -0400 Bill Sommerfeld 
<[email-masked]> wrote:

>> can't get a lock on a file, but it can't guarantee that I won't just
>> write  to the file without bothering to get a lock.
>

Correct.  It also means that changing the locking mode requires changing 
the file mode bits, which normally can be done only by the file's owner or 
by a superuser.

-- Jeff

---

From: Joseph Galbraith <[email-masked]>
Date: Mon, 04 Apr 2005 17:10:30 -0600

Jeffrey Hutzelman wrote:
> Correct.  The difference is pretty simple.  An advisory lock prevents 
> other processes from obtaining conflicting locks.  A mandatory lock also 
> prevents other processes from performing conflicting file accesses.
> 

I was just locking at the advisory locks under linux (I think I was
looking at the advisory locks.)

They appear to allow locking of a byte range, not just the whole
file?

It looks like it would make more sense to support this as a seperate
request rather than as part of open?

Thanks,

Joseph

---

From: Joseph Galbraith <[email-masked]>
Date: Mon, 04 Apr 2005 17:25:58 -0600

Jeffrey Hutzelman wrote:
> the file mode bits, which normally can be done only by the file's owner 
> or by a superuser.

But the server can tell what locking mode is available, and
respond correctly?  Though it may have to advertise both modes
and responde with UNSUPPORTED for some files.

Thanks,

Joseph

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Mon, 04 Apr 2005 19:31:47 -0400

> request rather than as part of open?

Like Windows, UNIX provides both whole-file and byte-range locking 
interfaces.  Both types are generally advisory; as Bill pointed out, the 
decision as to whether to use advisory or mandatory locking is made on a 
file-by-file basis, and is controlled by the file mode bits (attributes). 
The interface is exactly the same in either case.

I agree it would be appropriate to support byte-range locking, if it is 
desirable for sftp to be used as the back-end for something that looks like 
a filesystem.  In particular, our experiences with AFS indicate that while 
relatively few UNIX programs use byte-range locking at all, some software, 
particularly on Windows, simply will not function correctly without it.

-- Jeff

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Mon, 04 Apr 2005 19:33:51 -0400

> and responde with UNSUPPORTED for some files.

This requires platform-specific knowledge, but yes, the server can 
generally tell what sort of locking is available.  If sftp is to support 
both types of locking, then the spec should be written such that a server 
can claim to support advisory locking even when locks it sets are actually 
mandatory.  Put another way, an advisory lock should not require the server 
to prohibit other processes from accessing the file, but it should not 
preclude it from doing so, either.

-- Jeff

---

From: Damien Miller <[email-masked]>
Date: Tue, 05 Apr 2005 10:27:20 +1000

With all this talk of adding yet more complexity to sftp/filexfer, I
have to ask: what is the purpose?

If people want nfs-like functionality in filexfer, then they should just
specify a subsystem for using NFS over ssh. If filexfer is supposed to
be simple and lightweight, then the boat has already been missed.

Sure, much of the functionality mentioned in the draft is optional, but
there is a cost to implementors in implementing fallbacks, etc. If an
implementation needs multiple revisions of the draft (all do), then the
cost is much higher.

Because of this, we (OpenSSH) don't have any plans to implement filexfer
 > 03. The two or three things that 03 doesn't do that our users ask for
can easily be done through the extension mechanism.

-d

---

From: Joseph Galbraith <[email-masked]>
Date: Mon, 04 Apr 2005 19:28:57 -0600

> be simple and lightweight, then the boat has already been missed.

I believe it is still simpler and lighter weight than NFS;
in particular, filexfer excels in the same area that SSH
does: ad-hoc secure networks (or in this case ad-hoc secure
file-sharing.)

Adding NFS to the mix, at least in my opinion, makes it a lot
less ad-hoc in nature.

> there is a cost to implementors in implementing fallbacks, etc.

Well, if you don't use any optional functionality, you don't need to
implement any fallbacks.

Take for example the locking functionality that has just been discussed;
if you don't ever request any locks, mandatory or advisory, you never
even need to worry about whether the server supports them or not.  No
fallback behavior needed.

 > If an

The current draft allows you to skip implementating intermediate
versions.  (You have to implement at least version three so extensions
work.)

> can easily be done through the extension mechanism.

That would be a shame (both for us who still have to interoperate with
openssh, and for you customers), but I don't expect I'll change your mind.

I guess I understand.  To be honest, 03 supported unix secure file 
transfers pretty well.

It isn't a problem to generate the long listings under unix.
And as long as you don't get to wild in the variations, you
don't care that clients are parsing the string to get the
data that isn't in the attributes.

You don't really have need of most of the new attribute fields,
because unix doesn't support them.

The extra open control doesn't buy you anything, because unix
doesn't really support any of it anyway, and the original open
was modelled after the unix open semantics.

Mode bits express the access control on the file.

And so on.

On the other hand, on windows, we do have real problems, even
with plain-old ftp applications.

For example, we have problems with customers wanting to be able
to control the locking mode.  Under windows, opening files for
exclusive access is customary; have the server do so is what
customers expect in most cases.  But in some cases, for example,
when reading a active log file, customers want an exception to
that rule.

Customers doing a listing expect to see create time in the
listing, not ctime.

uid/gid is meaningless-- windows uses a variables sized structure
to express this information.

If a customer wants information about the files access control,
it can't be expressed as mode bits.

In most cases, the added complexities of the draft are similar
in nature-- they support operating systems that aren't unix in
nature while striving to enable unix platforms to continue
to have complete coverage and pay the minimal price in mandatory
complexity.

There are, admittedly, some new features in the draft do exist
solely to support file-sharing better... but, they aren't the
majority of the complexity.

Thanks,

Joseph

---

From: der Mouse <[email-masked]>
Date: Mon, 4 Apr 2005 23:53:20 -0400 (EDT)

> I'm not terribly familar with the advisory locking implemenated on
> most unix platforms [...].

> I presume it has basically three levels of locking, none, read (don't
> allow writers) and write (exclusive.)

Right.

> I presume the lock request fails if there is already a violation?

Depends.  With all interfaces I know of, when you request a lock, you
specify (typically by setting or not setting an optional bit) what to
do if the lock requested cannot be acquired immediately, one behaviour
being to sleep until it can be acquired and the other being to return
immediately with a distinctive error.

> What happens when a new access request comes along that violates the
> advisory lock?  Is the original lock holder notified in some fashion?

By "access request" you mean I/O attempt?  Nothing.  That's the
"advisory" bit: the locks have nothing whatever to do with actually
reading or writing the file; all they really affect is other lock
attempts on the file.

> Is it appropraite to lock a open time, or should it be a seperate
> request?

Recent BSD has the O_SHLOCK and O_EXLOCK flags to open(2), which take a
shared or exclusive lock atomically with opening the file.  Since
opening an existing file is not an operation with semantic significance
to anyone else, the atomicity is irrelevant except when the open
actually creates the file.  Similar remarks apply to opening and
locking over sftp, as far as I can see.

/~\ The ASCII				der Mouse
\ / Ribbon Campaign
 X  Against HTML	       [email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

---

From: der Mouse <[email-masked]>
Date: Tue, 5 Apr 2005 00:25:11 -0400 (EDT)

>> If an implementation needs multiple revisions of the draft (all do),
>> then the cost is much higher.
> versions.  (You have to implement at least version three so
> extensions work.)

Except version three isn't documented, is it?  (Unless you're lucky
enough to have a saved copy of the I-D for it from back when, that is.)

/~\ The ASCII				der Mouse
\ / Ribbon Campaign

---

From: Bill Sommerfeld <[email-masked]>
Date: Tue, 05 Apr 2005 00:44:08 -0400

> enough to have a saved copy of the I-D for it from back when, that is.)

old versions of working group drafts are available from a multitude of
archives, and have recently been made available officially via an ietf
site.

in this case, look at:

http://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/

						- Bill

---

From: der Mouse <[email-masked]>
Date: Tue, 5 Apr 2005 00:54:56 -0400 (EDT)

>> enough to have a saved copy of the I-D for it [...].)
> old versions of working group drafts are available from a multitude
> of archives, and have recently been made available officially via an
> ietf site.

> http://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/

Oh, good.  Thank you.  I didn't know they were now available.

What about the implications for standardization?  Won't the eventual
standard need to have an explicit description of at least some of
version 3?  Surely referring to an expired I-D won't cut it for a
standard?

/~\ The ASCII				der Mouse
\ / Ribbon Campaign