Subject: Ftp-WG: MFXX questions

From: Wolfgang Zekoll <[email-masked]>
Date: Sat, 7 Feb 2004 10:08:17

David,

after having read the MFXX-draft I started to implement the features 
mentioned there into my own server.  This brought me some some questions.

1. mlst-16 states that the MDTM command is only defined on files, not on 
non-files like directories.  What about MFMT and MFF?  Sometimes the draft 
mentions "file" and sometimes "object in the NVFS", see e.g. section 4.1.

2. Your draft shows (allows?) only the modification of the traditional UNIX 
mode bits, what about the sticky-bit set?

3. I can imagine situations where I don't want remote users to set the 
file's execution bit for uploaded files, even if changing permissions in 
general is allowed.  Such a server receives then a "MFF UNIX.mode=775 
some.file" which is not allowed or, depending on the point of view, not 
supported.  It can now be argued that

   (A) the client's permissions are not enough for the operation which 
results in "550 no access",

   (B) the MFF command is not implemented for this particular argument: 
"504 not implemented for this parameter".

On the other hand, the "no-x-bit" configuration could be seen as some kind 
of "missing granularity" (as it could happen for timestamps) which whould 
then return a

   (C) "213 UNIX.mode=664 some.file"

showing the client the file's real mode after the operation.

Please take these as "questions only", I have no preferences for their 
answers.  It's just that, since we are
discussing this draft, it should perhaps be defined in some way.

Regarding the current modification/creation time discussion, (3C) could be 
also interesting.  As said above, (3C) is already in effect for the MFMT 
and MFCT commands.  But time resolution is mentioned only as one 
example.  Another example could be the OS's limitation that creation time 
can never be newer than a file modification time.  Generalizing such 
OS-dependent cases, the meaning of the 213 response code would become 
something like

    Even if the client receives a 213 response it MUST|SHOULD parse
    the return value given in the response to see if this is (or
    comes near to) what the client wants.

This is not very difficult since the server's response lists the file's 
real attributes (or facts) in the same format as they were requested by the 
client.

But to make the create/modify time thing even more difficult, the question 
if such things are allowed or not, could be filesystem- and not 
OS-dependent (as the time granularity already is: even a Linux system 
doesn't have time resolution down to a second on a real FAT 
filesystem).  So depending on the FTP server's file space (the filesystem 
mounts) operations might work or fail to some degree.  And in the case of 
the MFMT time resolution even a partial failure could (has to be) good 
enough for the client.  So what I'm suggesting here is

   C> MFF Create=20040204105005;Modify=20030926053600; some.file
   S> 213 Create=20030926053600;Modify=20030926053600; some.file

in case the server is not able to set a file's creation time after the 
modification time.  It's now left to the client if this is ok or 
not.  Perhaps a new response code should be defined:

    219 approximate operation: the operation might not have been
        carried out to the client's full satisfaction, the client
        should examine the values listed in the response.

Regards

Wolfgang Zekoll

P.S.: Anyway, now that I'm thinking about it, why should an FTP draft/RFC 
define that a file's creation time is always newer than it's modification 
time?  Even if there are OSes or filesystems that could (thinking of Alun's 
archive example: must or simply does) allow this.  I see this as OS or 
client business, but not as something for an FTP specification.

---

From: "David Somers" <[email-masked]>
Date: Sat, 7 Feb 2004 16:15:54

Greetings,

> mentioned there into my own server.  This brought me some some questions.

I hope I can give some answers.

> mentions "file" and sometimes "object in the NVFS", see e.g. section 4.1.

OK. In my document (draft-somers-ftp-mfxx) that's an editorial mistake... it
should only refer to objects... files and directories are just types of
objects and they should be treated the same. Hopefully I'll get it right for
draft -02 :-)

> mode bits, what about the sticky-bit set?

Let's be clear about this: draft-somers-ftp-mfxx only specifies two facts
for use in MFF, 'Create' and 'Modify'. Any others mentioned are purely
examples of what could be possible - not what is possible - see section
5.2.1.

I'm thinking that it should also specify a fact for the perms, since that is
pretty neutral. Perhaps by doing something like:

perms=xxx to *set* the permissions to xxx
perms=+xxx to *add* permissions xxx to the existing permissions
perms=-xxx to *remove* permissions xxx from the existing permissions

The OS-specific facts should be documented in a separate I-D/RFC/whatever.
The idea is that somers-ftp-mfxx  meerly provides the framework to modify
the facts... draft-ietf-ftpext-mlst-16.txt provides the framework to read
the facts; the two are complimentary.

My Unix knowledge isn't that great, but I thought the sticky bit could only
be modified if you have root priviledge. Notwithstanding root issues, sure,
you could implement a fact to twiddle the sticky-bit: 'UNIX.sticky' .. don't
forget to document it somewhere as an I-D/RFC/whatever :-)

> supported.

If not supported, then UNIX.mode shouldn't have been returned as part of the
MFF FEAT response.

If the client-PI sends it and the server-PI doesn't support it, 504 MUST be
returned.

> It can now be argued that
>

Good point. I think A or C would be the ideal response; B to me seems wrong.

I think that the exact behavior is out of the scope of my document... it
should be covered by a document that details the facts for a Unix-type OS,
which then details the behavior required. Any volunteers to write such a
document? If nobody feels like writing such a document I could be persuaded
to edit/prepare such a document if I'm fed the content (beware: my unix
knowledge is a bit rusty!)

> discussing this draft, it should perhaps be defined in some way.

They are very good questions, and many thanks for asking them. They got me
thinking :-) I hope I gave some good answers.

>     comes near to) what the client wants.

That is implicit in the current draft... it says the response MAY not be the
same as that in the request. I'll add a note to implementors that they are
RECOMMENED to parse 213 to see what was actually set.

I've had some further thoughts on the create/modify debate... I'll post
another message when these thoughts can be put into writing.

> real attributes (or facts) in the same format as they were

That was why it was done that way.

>         should examine the values listed in the response.

Not sure I agree with that 219. I think that a 213 is returned, the
client-PI MUST parse the response and can then at its discretion issues
warnings/messages to the user if the requested and actioned times are very
different.

> client business, but not as something for an FTP specification.

I think it was an oversight. But I'm not sure if it can be corrected before
draft-ietf-ftpext-mlst-16 becomes an RFC. KRE, any comments on this?

(I seem to have missed Alun's archive example... my mail provider had some
problems and a few messages went missing.)

Regards,

David

---

From: Wolfgang Zekoll <[email-masked]>
Date: Sun, 8 Feb 2004 06:53:16

>objects and they should be treated the same. Hopefully I'll get it right for

So it would be possible to set the modification time of a directory with 
MFMT but not to query it with MDTM because MDTM excludes directories?

>examples of what could be possible - not what is possible - see section

Well, now that you mention it, yes your draft doesn't deal with UNIX modes 
(just as side-note: setting sticky-bits is possible for normal users).

>If the client-PI sends it and the server-PI doesn't support it, 504 MUST be

This example was perhaps not exact enough.  I'm talking of an FTP-server 
running on a UNIX box, being able to modify file mode bits.  But the 
administrator decides for some reasons that he does not like remote FTP 
clients to set the x-bit of files, anything else, like e.g. 666, is ok for 
files, but not setting any x-bit as in 775, even if the user owns the file 
and has all the permissions he or she needs to do it (the same chmod would 
succeed if the user would be logged in local with a shell).

>Good point. I think A or C would be the ideal response; B to me seems wrong.

If the server would deny, I think 504 is more exact, because it's not 
really a permission thing but a set x-bit in the parameter.

But anyway, I prefer variant C.  Why?  In the no-x-bit server setup the 
server's "permission granularity" is artificially degraded by the admin 
while modifying UNIX permissions in general is allowed (that's why the 
server offers UNIX.mode in it's FEAT response).

>...

To be more precise, I suggest that the client parses the 213 server 
response for any of the commands MFMT, MFCT and MFF because the server may 
not have exactly done what the client asked for.  The no-x-bit is just 
another example of such a situation.

>warnings/messages to the user if the requested and actioned times are very

The return code 213 is already in use for MDTM and SIZE.  And in these 
contexts the return data is always exact (the modification time of a file 
has an exact value) and the return data is exactly what the client asked 
for (the modification time).  With the MFXX commands is still exact but 
it's value may not be exactly what the client asked for.  Because of this 
difference a new return code could be used to make it clear for 
everyone.  But this is perhaps simply a question of my personal taste.

Regards

Wolfgang Zekoll

P.S.: David, you mentioned that someone could write something about UNIX 
permissions and FTP.  If you (or someone else) could point me to the "Draft 
Editors Guide and How-To for Beginners" I would consider doing the job.

---

From: "David Somers" <[email-masked]>
Date: Sun, 8 Feb 2004 09:34:04

> MFMT but not to query it with MDTM because MDTM excludes directories?

I think MFMT has been implemented a few times to work with both files and
directories ;-)

It could be argued that on some systems that a directory is just a special
kindof file, so its not really breaking the rules for MFMT to work with
both. YMMV.

I've realized that MFxx is probably a bit misleading in that the 'file' part
of the name should really be 'object'... but I'm not sure whether its worth
changing the name now (to MOxx)... it depends if I'll get lynched by people
who have already implemented MFxx - although a nice backdoor could be to say
that MFxx is an alias for MOxx :-)

> Well, now that you mention it, yes your draft doesn't deal with

Yep.. that is something for another document... especially as the facts
should work with both MFxx and MLSx.

> (just as side-note: setting sticky-bits is possible for normal users).

OK. I'll take your word for it.

> server offers UNIX.mode in it's FEAT response).

I think variant C is probably the way to go.

This is very implementation specific, and should be covered by whatever
document details UNIX.xxx facts (hint!).

One solution could be something like this:

UNIX.mode - to set the bits, does best effort (MAY be artificially degraded
by the admin), so client-PI MUST parse the 213 response to determine exactly
what did or did not happen to the bits (requested bits and the response bits
could be different).

UNIX.strictmode - to set the bits, does NOT do best effort, NO artificial
degredation allowed. Returns 550 if the exact bits can not be set/cleared,
otherwise 213 (requested bits and the response bits are identical).

> another example of such a situation.

OK. Again it was implicit in the draft (the response MAY be different from
the request). I'll add that its RECOMMENDED for the client to parse the 213
response. (I think most clients would do that anyway because they cache the
facts).

> everyone.  But this is perhaps simply a question of my personal taste.

I don't see a reason to use a response other than 213.

Yes, the response may not be what was requested, but it is exact.

> Editors Guide and How-To for Beginners" I would consider doing the job.

Separate email sent to you with the details.

Cheers,

David.

---

From: Robert Elz <[email-masked]>
Date: Mon, 9 Feb 2004 03:26:27

Date:        Sun, 8 Feb 2004 06:53:16 
    From:        Wolfgang Zekoll <[email-masked]>
    Message-ID:  <[email-masked]>

  | So it would be possible to set the modification time of a directory with 
  | MFMT but not to query it with MDTM because MDTM excludes directories?

Note that MDTM doesn't "exclude directories", it says that the
behaviour there is undefined.   That's because different real
filesystems implement directories in vastly different ways, and
there is/was no desire to constrain them to necessarily have the
concept of modification time for a directory, or force one on top
of the particular server's filesystem.

On the other hand, I'd expect the typical unix server's MDTM
implementation to return the standard unix st_mtime for a directory
when queried (that's permissible as undefined behaviour).   It
simply isn't worth the bother to specifically check for the name
being a directory, and exclude it.

Servers on systems where there is no st_mtime, or anything like it,
for a directory, can return an error, or return 00000000000000, or the
current time, or do just about anything else they like (provided only
that they stay within the protocol).

kre

ps: I haven't had time to read the MFXX draft yet, but if it is going to
go anywhere as an IETF work item, it will need to be specified in a way
such that it is implementable on anything which could reasonably implement
an FTP server.

---

From: "Mark Symons" <[email-masked]>
Date: Mon, 9 Feb 2004 09:09:53

> I've realized that MFxx is probably a bit misleading in
> that the 'file' part of the name should really be
> 'object'... but I'm not sure whether its worth changing
> the name now (to MOxx)... it depends if I'll get lynched
> by people who have already implemented MFxx - although a
> nice backdoor could be to say that MFxx is an alias for
> MOxx :-)

Don't worry about it.  You've got a name...  stick with it.  Opening a discussion on naming would only waste WG time. Put it this way... most people don't have a clue what the "M stands for in MLST, but that has not prevented anyone from implementing  support for it.

Mark Symons
Ipswitch, Inc
Augusta GA

---

From: "David Somers" <[email-masked]>
Date: Mon, 9 Feb 2004 10:45:52

> Don't worry about it.  You've got a name...  stick with it.

Being the sneaky and devious person that I am ;-), in the next draft, I've
changed it from File to Fact, so we have:

MFCT - Modify Fact Creation Time
MFMT - Modify Fact Modification Time
MFF  - Modify Facts

This has two benefits:

- The command names stay the same
- Fact is more accurate than File.. its twiddling the facts and not the
file/dir per se.

> Opening a discussion on naming would only waste WG time. Put it
> this way... most people don't have a clue what the "M stands for
> in MLST, but that has not prevented anyone from implementing
> support for it.

:-)

David Somers

btw, do any of the Ipswitch products implement MFxx?

---

From: "Alun Jones" <[email-masked]>
Date: Mon, 9 Feb 2004 13:34:09

> -----Original Message-----
> From: ftp-wg-owner [mailto:[email-masked]] On 
> Behalf Of David Somers
> Sent: Monday, February 09, 2004 10:46 AM
> 

To avoid people thinking that facts themselves have a creation time or
modification time, how about "MFCT - Modify Fact: Creation Time" and the
like, to prevent idiots from reading something into the name.

Alun.
~~~~
-- 
Texas Imperial Software   | Find us at http://www.wftpd.com or email
1602 Harvest Moon Place   | [email-masked].
Cedar Park TX 78613-1419  | WFTPD, WFTPD Pro are Windows FTP servers.
Fax/Voice +1(512)258-9858 | Try our NEW client software, WFTPD Explorer.

---

From: Wolfgang Zekoll <[email-masked]>
Date: Mon, 9 Feb 2004 16:00:03

Robert,

>of the particular server's filesystem.

As David pointed already out, your draft mentions "files" as MDTM 
parameters (section 3).  This is followed by some reasoning with the REST 
command.  But now that I read your draft again (more detailed) I see that 
section 3.1 states

    The "pathname" specifies an object in the NVFS which may
    be the object of a RETR command.

while section 3.2 uses the term "entity".

To make things even more worrying in the examples (section 3.4) I find

    C> MDTM D
    S> 550 D is not retrievable

where "D" is a directory.

>being a directory, and exclude it.

This is one way to see it.  I'll explain you my point of view (coming from 
a real-life programming experience).

Once I decided to write an FTP syncer program.  Ok, what do I have to 
do?  The program has to automatically identify the files on the remote 
filesystem (1), get their modification times (2) do some magic and so on 
(this in not important for my example).  Part (1) is the interesting 
thing.  As far as I know there is no FTP command that tells the client if a 
remote object is a file or directory or whatever (I don't now if MLSD/MLST 
is already common practice).  Then how can I solve task (1)?  Parsing 
directory listings?  Then I recognized that MDTM is my friend.  I have to 
do it anyway for step (2) but since it's only defined on files (and in 
contrast to MLSX it is common practice) I get error responses for every 
non-file on which I try MDTM.  If something which is not a file is really a 
directory is then later checked with a CWD and ... yes I have I want.  I 
did it by reading the MLST draft and, to be honest, I can't see my mistake.

Yes, I recognize that on the server side the additional check if the MDTM 
parameter is a file or not.  But it's only a "stat(filename, &sbuf); if 
(S_ISREG(sbuf.st_mode) ..." and the command combination "TYPE A; SIZE 
some.file" implies more additional work.  And on the other hand, the 
additional MDTM check gives the "isfile-function" as side effect.

Please notice that the reason why I'm doing this discussion (and why I want 
to continue asking question which might bring you or others to a "Oh no 
Wolfgang, not again!") is that things should be defined as exact as 
possible.  As soon as your draft becomes an RFC it's "the 
law".  Implementors will read and rely on it (this is what RFCs are about, 
isn't it?) and unclear things may then become the programmer's "nightmare".

Coming back to MDTM.  In this special case I obviously have a preference 
how it should work.  But anyway, if you would like to define it different, 
then please do it.  Then it's clear which I find better that an uncertain 
specification which might be this or that way.

Regards

Wolfgang Zekoll

---

From: "Mark Symons" <[email-masked]>
Date: Tue, 10 Feb 2004 09:23:27

> btw, do any of the Ipswitch products implement MFxx?

No they don't.  But, in the words of our WS_FTP Development Manager, having control over Modify Time is one of the biggest bugbears in file synchronisation.  I have given him a copy of draft-somers-ftp-mfxx-01 and he is very interested.

That does not mean that you'll see anything straight away as this was an individual opinion. But it's a good sign.

Mark Symons
Ipswitch, Inc
Augusta GA