Subject: [lemonade] Comments on reconnect-01

From: Randall Gellens <[email-masked]>
Date: Mon, 11 Oct 2004 14:15:02 -0700

The Abstract and Introduction make a number of very sweeping 
assertions ("notoriously slow", "nearly unusable due to the 
inordinate amount of time spent waiting", "users tend to give up on 
the email client on their mobile terminal") -- are these backed up by 
data?

The assumption that nothing has changed on the mail store in a short 
time is suspect, and likely to vary widely per user.

More importantly, it seems to me that the benefits proposed are 
rather limited and are only useful when a client reconnects after a 
brief disconnect.  A mechanism that allowed a client to more 
efficiently determine what has changed since it was last connected 
would, it seems to me, be more broadly useful for all clients, and 
confer much greater benefits to mobile clients than what is proposed 
here.  For example, something such as the ACAP DELETEDSINCE command 
and ability to ask the server for items changed since a certain time. 
We are already moving towards this with CONDSTORE, UIDNEXT, etc.

In Section 2 (Introduction), use of ANNOTATE would allow flags to be 
searched using a modtime, which would avoid the 'UID fetch 1:* 
(FLAGS)' command in the example.  Also, note that the example fecthes 
flags, not headers (the text says twice that headers are fetched).

Preserving sessions during short periods of dropped connectivity is a 
TCP property, and does not need separate IMAP functionality.  Perhaps 
a discussion of client and server TCP implementations would help.

Allowing clients to more efficiently resynchronize is a worthy goal, 
but should have broader applicability.  It seems to me that what 
clients need to know is: have any new messages arrived or existing 
messages been changed or deleted since last time?  It also would 
benefit from knowing if a message moved, but that is much trickier.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Most of our so-called reasoning consists in finding arguments for going
on believing as we already do.                  --James Harvey Robinson

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 12 Oct 2004 12:48:30 +0100

Randall Gellens wrote:

> assertions ("notoriously slow", "nearly unusable due to the inordinate 
> amount of time spent waiting", "users tend to give up on the email 
> client on their mobile terminal") -- are these backed up by data?

I will let Corby to comment on this ;-).

> time is suspect, and likely to vary widely per user.

This is not a requirement.

> More importantly, it seems to me that the benefits proposed are rather 
> limited and are only useful when a client reconnects after a brief 
> disconnect.

This is not the case and if the document mislead you to believe this, I 
think it should be fixed.

>   A mechanism that allowed a client to more efficiently determine what 
> has changed since it was last connected would, it seems to me, be more 
> broadly useful for all clients, and confer much greater benefits to 
> mobile clients than what is proposed here.  For example, something 
> such as the ACAP DELETEDSINCE command and ability to ask the server 
> for items changed since a certain time.

Look at section 3.3, it is very much what you are asking for. I would 
like to give a warning that the section is missing some text and 
examples and the syntax is more likely to change.

> We are already moving towards this with CONDSTORE, UIDNEXT, etc.

Actually, the RECONNECT extension depends on CONDSTORE and provides 
something similar to DELETEDSINCE functionality.

> In Section 2 (Introduction), use of ANNOTATE

Did you mean CONDSTORE?

> would allow flags to be searched using a modtime, which would avoid 
> the 'UID fetch 1:* (FLAGS)' command in the example.  Also, note that 
> the example fetches flags, not headers (the text says twice that 
> headers are fetched).

I've fixed that in my copy, thanks.

> a discussion of client and server TCP implementations would help.

Sure. Any suggested text :-)?

> messages been changed or deleted since last time?

See above.

>   It also would benefit from knowing if a message moved, but that is 
> much trickier.

Maybe. I am not sure how one would detect "moving" a message, as there 
is no IMAP MOVE command, only COPY + STORE \Deleted. Does this mean that 
the server should report locations of all copies of a message?

Alexey

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 12 Oct 2004 14:28:16 +0200

Alexey Melnikov writes:
>> the email client on their mobile terminal") -- are these backed up 
>> by data?
>

I want to comment.

The last mobile client I used started life by sending something very 
much like this:

1 LIST "" "*"
2 SELECT INBOX
3 FETCH 1:* BODY.PEEK[HEADER FLAGS]

A commercial product. The thing cost several hundred euro. I don't need 
data to call that "notoriously slow", "nearly unusable" or worse.

I wanted to comment because I have questions: Are there any mobile 
clients that use IMAP well? If so, which? And do its/their authors 
think IMAP needs new extensions for mobile use?

Arnt

---

From: Randall Gellens <[email-masked]>
Date: Tue, 12 Oct 2004 06:34:31 -0700

At 2:28 PM +0200 10/12/04, Arnt Gulbrandsen wrote:

>>> inordinate amount of time spent waiting", "users tend to give up 
>>> on the email client on their mobile terminal") -- are these 
>>> backed up by data?
>>
>  The last mobile client I used started life by sending something 
> very much like this:
>
>  1 LIST "" "*"
>  2 SELECT INBOX
>  A commercial product. The thing cost several hundred euro. I don't 
> need data to call that "notoriously slow", "nearly unusable" or 
> worse.
>

So should we perhaps produce an Informational or BCP that gives 
advice on good use of IMAP?  Perhaps this document should include an 
example of good use of IMAP and then show which problems still remain?
--
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
In the beginning was the word...and it was the wrong length.
                 --Bob Barton (Burroughs computer architect)

---

From: Randall Gellens <[email-masked]>
Date: Tue, 12 Oct 2004 06:32:32 -0700

At 12:48 PM +0100 10/12/04, Alexey Melnikov wrote:

>> on the email client on their mobile terminal") -- are these backed 
>> up by data?
>
>>  The assumption that nothing has changed on the mail store in a 
>> short time is suspect, and likely to vary widely per user.
>
>> rather limited and are only useful when a client reconnects after 
>> a brief disconnect.
>
>  This is not the case and if the document mislead you to believe 
> this, I think it should be fixed.

I thought the document was proposing a mechanism where sessions could 
continue to live for 30 minutes (or perhaps longer) after a client 
accidently or intentionally disconnects.

>>    A mechanism that allowed a client to more efficiently determine 
>> what has changed since it was last connected would, it seems to 
>> me, be more broadly useful for all clients, and confer much 
>> greater benefits to mobile clients than what is proposed here. 
>> For example, something such as the ACAP DELETEDSINCE command and 
>> ability to ask the server for items changed since a certain time.
>
>  Look at section 3.3, it is very much what you are asking for. I 
> would like to give a warning that the section is missing some text 
> and examples and the syntax is more likely to change.

But the after 30 minutes or so, the client must do a full resynch 
anyway. What if a client reconnects once or twice per day, or three 
times per week?

>>  In Section 2 (Introduction), use of ANNOTATE

No, I was suggesting that perhaps ANNOTATE could be used as an 
alternative to the example shown in Section 2 which is only fetching 
the flags, after all.

>
>>  would allow flags to be searched using a modtime, which would 
>> avoid the 'UID fetch 1:* (FLAGS)' command in the example.  Also, 
>> note that the example fetches flags, not headers (the text says 
>> twice that headers are fetched).
>
>>  Preserving sessions during short periods of dropped connectivity 
>> is a TCP property, and does not need separate IMAP functionality. 
>> Perhaps a discussion of client and server TCP implementations 
>> would help.
>

Let me think about some.

>
>>  Allowing clients to more efficiently resynchronize is a worthy 
>> goal, but should have broader applicability.  It seems to me that 
>> what clients need to know is: have any new messages arrived or 
>> existing messages been changed or deleted since last time?
>
>  See above.

But the above expires after 30 minutes or so.

>
>>    It also would benefit from knowing if a message moved, but that 
>> is much trickier.
>
>  Maybe. I am not sure how one would detect "moving" a message, as 
> there is no IMAP MOVE command, only COPY + STORE \Deleted. Does 
> this mean that the server should report locations of all copies of 
> a message?

I said it was much trickier.  When it was last discussed we ended up 
with UIDPLUS, as I recall.  Which helps the client doing the move, 
but not a client disconnected during the move.

-------------- Randomly-selected tag: ---------------
There are some micro-organisms that exhibit characteristics of
both plants and animals.  When exposed to light they undergo
photosynthesis; and when the lights go out, they turn into
animals.  But then again, don't we all?

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 12 Oct 2004 14:41:30 +0100

Randall Gellens wrote:

>> need data to call that "notoriously slow", "nearly unusable" or worse.
>>
> So should we perhaps produce an Informational or BCP that gives advice 
> on good use of IMAP?  Perhaps this document should include an example 
> of good use of IMAP and then show which problems still remain?

Yes!

I think RFC 2683 ("IMAP4 Implementation Recommendations") addresses this 
to some extend.
Also, draft-melnikov-imap-disc-05.txt has detailed instructions for 
disconnected clients (and examples), however many recommendations apply 
to other types of IMAP clients.

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 12 Oct 2004 16:27:58 +0100

Randall Gellens wrote:

>>> brief disconnect.

Well, that is true, but CONDSTORE is a dependency of RECONNECT and the 
client can use CONDSTORE to avoid full resync.

> alternative to the example shown in Section 2 which is only fetching 

Firstly, flags are a common case. Secondly, I would rather not have 
examples with annotations, as RECONNECT has no dependency on ANNOTATE.

---

From: Randall Gellens <[email-masked]>
Date: Tue, 12 Oct 2004 09:17:51 -0700

At 4:27 PM +0100 10/12/04, Alexey Melnikov wrote:

>>>> rather limited and are only useful when a client reconnects 
>>>> after a brief disconnect.
>>>
>>  I thought the document was proposing a mechanism where sessions 
>> could continue to live for 30 minutes (or perhaps longer) after a 
>> client accidently or intentionally disconnects.
>
>  Well, that is true, but CONDSTORE is a dependency of RECONNECT and 
> the client can use CONDSTORE to avoid full resync.

But CONDSTORE doesn't get the client all the way there, does it?  If 
so, why do we need 30-minute sessions?

>>>    Actually, the RECONNECT extension depends on CONDSTORE and 
>>> provides something similar to DELETEDSINCE functionality.
>>>
>> alternative to the example shown in Section 2 which is only 
>> fetching the flags, after all.
>

But ANNOTATE goes a long way towards helping clients with flags, 
which is, as you say, a common case (and a very important one).

-------------- Randomly-selected tag: ---------------
To determine how long it will take to write and debug a
program, take your best estimate, multiply that by two, add
one, and convert to the next higher units.

---

From: Mark Crispin <[email-masked]>
Date: Tue, 12 Oct 2004 09:07:17 -0700 (Pacific Daylight Time)

On Tue, 12 Oct 2004, Randall Gellens wrote:
> So should we perhaps produce an Informational or BCP that gives advice on 
> good use of IMAP?  Perhaps this document should include an example of good 
> use of IMAP and then show which problems still remain?

My experience of trying to do this has not been positive.

The problem is that many (most?) clients are designed with a model of how 
the access mechanism should work, and then shoehorn the access mechanism 
into fitting into that model.  If the application has not considered the 
facilities of the access mechanism from inception, often what we end up 
using is a "lowest common denominator".  It doesn't help to tell the 
developer "use the protocol this way" if "this way" makes no sense in the 
design of the application.

I don't know how to resolve this issue.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 12 Oct 2004 17:36:11 +0100

Randall Gellens wrote:

> But CONDSTORE doesn't get the client all the way there, does it?

It doesn't send expunged messages to the client, which I think is
a). not a big deal and the client should just do a SEARCH
b). sending EXPUNGE responses is more chatty than just a single response 
from SEARCH (see #1).

>   If so, why do we need 30-minute sessions? 

1). We save some round trips by combining LOGIN/AUTHENTICATE with SELECT.
2). We avoid sending untagged responses required on SELECT.
3). A future extension may associate additional data with an opened 
mailbox, RECONNECT will keep this as well.

Maybe 30 minutes is too little?

> But ANNOTATE goes a long way towards helping clients with flags, 

I am not sure what do you mean.

> which is, as you say, a common case (and a very important one).

Ok, if you right an example, we will consider it for inclusion.

Alexey

---

From: Randall Gellens <[email-masked]>
Date: Tue, 12 Oct 2004 10:00:11 -0700

> mailbox, RECONNECT will keep this as well.

Perhaps an extension that had nothing to do with time would be in 
order.  Consider, for example, an extension to allow a client to set 
a default folder.  The client could pass some token (other than SID) 
to LOGIN/AUTHENTICATE to indicate that the default folder should be 
selected.  Would that give the same benefits?  An advantage is that 
it could always be used, not just within 30 minutes.
--
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Coding is "90% finished" for half of the total coding time. Debugging
is "99% complete" most of the time.                     --Fred Brooks

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 13 Oct 2004 22:17:15 +0100

Randall Gellens wrote:

> order.  Consider, for example, an extension to allow a client to set a 
> default folder.  The client could pass some token (other than SID) to 
> LOGIN/AUTHENTICATE to indicate that the default folder should be 
> selected.  Would that give the same benefits?  An advantage is that it 
> could always be used, not just within 30 minutes.

In your proposal: do you still want the server to keep some per-client 
mailbox state?

If the answer is "yes": I really want to avoid the requirement on 
servers to store additional per-client state in some persistent storage. 
What if the client never comes back? I would rather allow for server 
implementations that can keep all per-client state in memory. If the 
server get restarted - the state would be lost and I think this is just 
fine.
If there is some kind of timeout, than your proposal effectively becomes 
what RECONNECT is now. I frankly don't care if timeout is 30 minutes. It 
can be a couple of days, as long as it is not infinity. We can extend 
RECONNECT to advertise timeout, but I am not sure if this is that useful.

Alexey

---

From: Randall Gellens <[email-masked]>
Date: Thu, 14 Oct 2004 14:44:11 -0700

At 10:17 PM +0100 10/13/04, Alexey Melnikov wrote:

>>>   3). A future extension may associate additional data with an 
>>> opened mailbox, RECONNECT will keep this as well.
>>>
>> order.  Consider, for example, an extension to allow a client to 
>> set a default folder.  The client could pass some token (other 
>> than SID) to LOGIN/AUTHENTICATE to indicate that the default 
>> folder should be selected.  Would that give the same benefits?  An 
>> advantage is that it could always be used, not just within 30 
>> minutes.
>>
>  In your proposal: do you still want the server to keep some 
> per-client mailbox state?

No, I am trying to avoid that.  The server might keep some per-user 
preferences, but that scales much better than per-mailbox state.

> servers to store additional per-client state in some persistent 
> storage. What if the client never comes back? I would rather allow 
> for server implementations that can keep all per-client state in 
> memory. If the server get restarted - the state would be lost and I 
> think this is just fine.

What I'd like to do is make the basic process better, so we don't 
need get lucky with a request coming in during the time window and 
while the server has the data in memory.

>  If there is some kind of timeout, than your proposal effectively 
> becomes what RECONNECT is now. I frankly don't care if timeout is 
> 30 minutes. It can be a couple of days, as long as it is not 
> infinity. We can extend RECONNECT to advertise timeout, but I am 
> not sure if this is that useful.

I'd like to avoid anything with a timeout, if possible.

-------------- Randomly-selected tag: ---------------
I loathe people who keep dogs.  They are cowards who haven't got the
guts to bite people themselves.                  --August Strindberg