Subject: [lemonade] Channel updates for content conversion

From: Lyndon Nerenberg <[email-masked]>
Date: Tue, 09 Nov 2004 23:59:51 -0800

Folks, sorry for being so late with this. I've been dealing with
electrical problems on the boat, which took priority. I wasn't able
to do the update to the draft, but here is a description of how I'm
planning to add support for content conversion.

A new CHANNELCONVERSIONS command is added. It takes the form

  tag CHANNELCONVERSIONS nstring

This command returns a list of
supported conversion target mime types. If the nstring is NIL
the full list of conversion targets is returned. If the string
is non-nil it contains space separated list of top-level MIME
types to restrict the response list to.

The CHANNELCONVERSIONS response takes the form

  * CHANNELCONVERSIONS astring

where the astring contains the MIME type. There is one untagged
response for each matching type. E.g.

  1 CHANNELCONVERSIONS NIL
  * CHANNELCONVERSIONS "image/gif"
  * CHANNELCONVERSIONS "image/jpeg"
  * CHANNELCONVERSIONS "audio/gsm"
  * CHANNELCONVERSIONS "audio/g723"
  * CHANNELCONVERSIONS "audio/basic"
  * CHANNELCONVERSIONS "application/pdf"
  * CHANNELCONVERSIONS "application/postscript"
  * CHANNELCONVERSIONS "application/x-gzip"
  1 OK CHANNELCONVERSIONS completed

  2 CHANNELCONVERSIONS "audio"
  * CHANNELCONVERSIONS "audio/gsm"
  * CHANNELCONVERSIONS "audio/g723"
  * CHANNELCONVERSIONS "audio/basic"
  2 OK CHANNELCONVERSIONS completed

  3 CHANNELCONVERSIONS "image audio foo"
  * CHANNELCONVERSIONS "audio/gsm"
  * CHANNELCONVERSIONS "audio/g723"
  * CHANNELCONVERSIONS "audio/basic"
  * CHANNELCONVERSIONS "image/gif"
  * CHANNELCONVERSIONS "image/jpeg"
  3 OK CHANNELCONVERSIONS completed

  4 CHANNELCONVERSIONS "foo"
  4 OK CHANNELCONVERSIONS completed

The CHANNEL command is extended by adding a new parameter that specifies
a conversion:

  tag CHANNEL channel-uri-list SP channel-set SP conversion-spec

conversion-spec is an nstring that specifies a single MIME type to
convert the content to. If NIL, no conversion is performed.

If the conversion cannot be performed, a NIL response is returned for
that item (the same behaviour as if the requested scheme failed).

  5 CHANNEL (rtsp) (1 2) audio/mpeg
  * 1 CHANNEL 2 NIL
  5 OK done but nothing was sent because audio/mpeg is not a valid 
target

I'm still working out issues with IMAPURL. Also, the syntax here
gives the general sense, but the details will change when the grammar
gets an overhaul. I'm also working on the syntax and semantics for
how to specify the data is to be returned in-band.

--lyndon

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: "Ringel, Rick" <[email-masked]>
Date: Fri, 12 Nov 2004 13:24:57 -0700

All,

I'm not familiar with all the work that has been completed, and the
discussions that have led up to this thread.  But, I wanted to suggest
something, on the outside chance it has not been discussed and rejected.
Forgive my late arrival.

Rather than have the client explicitly request a conversion for a
specific attachment, has the group considered the addition of a session
variable that contains a list of 'preference' formats for audio, image,
etc.   The server could take preferences into account before rendering
the headers in the FETCH response.  In other words, the client would
establish an abstracted 'view' of the message store.

For example, if a low-rate device wants to see all audio files in G.729,
then it would set this preference session variable to contain
audio/G.729.  Then, when the server is processing FETCH commands, it can
determine if the native format matches G.729, or if not, determine if a
conversion is possible.  In either case, it can publish audio/G.729 as
the content type, and thus avoid any explicit conversion requests.  Any
subsequent FETCH of the attachment is served in G.729 format.  This
would stay in scope during use of the CHANNEL and BINARY commands as
well.  

>From an implementation standpoint, it might simplify the client side to
deal with conversion/format issues subsequent to the login phase, but
before initiating any mailbox/message/attachment navigation.

-Rick Ringel

---

From: "Vaudreuil, Greg M (Greg)" <[email-masked]>
Date: Fri, 12 Nov 2004 15:32:30 -0600

This approach was discussed, in part because it is the model used by MMS, using UAProf.  The group discussing it preferred to see the decision-making intelligence pushed down to the client which was in a better position to evaluate whether conversion is desirable.  For example, the client is in a better position to determine if a given audio part, in your example, was suitable for conversion or not.  It is not clear that a client would necessarily want MP3 audio converted to G.729.  It is also clear that sometimes the client will want the content in full form and sometimes not, depending on circumstances.  Is the client in a CDMA 1X or a EVDO network region?  WiFi connected or dial-up?  Is the content "valuable" enough to pay the extra price in getting full fidelity.  In the example of the music, maybe it is worth downloading the MP3 file, or waiting to download the MP3 file when in a WiFi hotspot.  This probably applies even more to visual media.  A scanned text document re!
 duced to a thumbnail size is worthless, but a reduced size picture may still be useful.
 
Also, limited devices are getting more flexible.  They are gaining the ability to fetch new codecs on demand if the user determines that the content is valuable enough. 
 
The approach you describe is an important option, I just wanted to state where I think the previous discussions on this point have been.
 
Greg V.

-----Original Message-----
From: Ringel, Rick [mailto:[email-masked]]
Sent: Friday, November 12, 2004 3:25 PM
To: [email-masked]
Subject: RE: [lemonade] Channel updates for content conversion

All, 

I'm not familiar with all the work that has been completed, and the discussions that have led up to this thread.  But, I wanted to suggest something, on the outside chance it has not been discussed and rejected.  Forgive my late arrival.

Rather than have the client explicitly request a conversion for a specific attachment, has the group considered the addition of a session variable that contains a list of 'preference' formats for audio, image, etc.   The server could take preferences into account before rendering the headers in the FETCH response.  In other words, the client would establish an abstracted 'view' of the message store.

For example, if a low-rate device wants to see all audio files in G.729, then it would set this preference session variable to contain audio/G.729.  Then, when the server is processing FETCH commands, it can determine if the native format matches G.729, or if not, determine if a conversion is possible.  In either case, it can publish audio/G.729 as the content type, and thus avoid any explicit conversion requests.  Any subsequent FETCH of the attachment is served in G.729 format.  This would stay in scope during use of the CHANNEL and BINARY commands as well.  

>From an implementation standpoint, it might simplify the client side to deal with conversion/format issues subsequent to the login phase, but before initiating any mailbox/message/attachment navigation.

-Rick Ringel

---

From: [email-masked]
Date: Sat, 13 Nov 2004 11:14:01 -0600

There has been much talk about CHANNEL doing stream conversion.
I think we are getting bogged down with the most complex case and
ignoring the majority case for content conversion: simple documents.

While both are desired, I can see a server implementer choosing to
implement only the simple case for the enterprise to allow document and
image conversion.

The streaming case (audio, video, etc.) seems to be geared more toward
the consumer market and requires additional complexity and network
resources to deploy.

What I was thinking was to separate the CHANNEL draft into the two use
cases we identified in Vancouver: (A) for RTP streaming, and (B) for
simple file conversion and download.  This would allow us to quickly
generate a draft for the 80% use case while tackling the complexities of
the 20% use case.

On the issue of content transformation on the IMAP server:
There are quite a few vendors out there that provide libraries and
engines which convert content on the users' behalf (eg. ImageMagic).
The transcoding can be done either on another server or in another
process on the same machine, with a defined API or socket interface to
convert the content before download.
The problem of in-machine transcoding is that it doesn't scale well at
all.  Transcoding is VERY expensive and can cripple a server if too many
requests are handled at the same time.  However, for small businesses it
is a perfect solution.
I agree with Lyndon that adding the extensions to FETCH would
overcomplicate the command.  I'm not sure if we are allowed (according
to the charter) to create new commands (CHANNEL || TRANSFORM ||
TRANSCODE), but in this case I can see the need for it simply to handle
the variable parameter list better.

Corby Wilson

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of ext Lyndon Nerenberg
> Sent: Wednesday, November 10, 2004 11:05 PM
> To: Eric Burger
> Cc: [email-masked]
> --On 2004-11-10 10:36 PM -0500 Eric Burger <[email-masked]>
> wrote:
> 
> > That is easy for me to say.  I don't write servers, so I'll "go with
> > the flow."
> 
> Hey, us server authors get away easy. We don't have to maintain the
> client state machine that resolves the n*m capability graph to figure
> out how to do 'foo' with a given server instance :-)
> 
> Seriously, though, the issue of multiple capabilities and code paths
is
> much more of a nightmare on the client side. I will strongly oppose
any
> proposal that supports multiple paths leading to exactly the same
> result, unless there is a very good reason for it (from a protocol and
> software engineering standpoint).
> 
> --lyndon
>

---

From: Stephane Maes <[email-masked]>
Date: Sat, 13 Nov 2004 12:03:32 -0700

Lyndon's approach is fine.

Decision to implement within imap server to have the server delegate is an implementation / deployment choice supported by the proposal.

Stephane
_____
Stephane H. Maes, PhD,
Director of Architecture - Mobile, Oracle Corporation.
Ph: +1-203-300-7786 (mobile/SMS); Fax / UM: +1-650-607-6296.
e-mail: [email-masked]
IM: shmaes (AIM, Y!) or [email-masked] (MSN Messenger) 

-----Original Message-----
From: [email-masked] <[email-masked]>
To: [email-masked] <[email-masked]>; [email-masked] <[email-masked]>
CC: [email-masked] <[email-masked]>
Sent: Sat Nov 13 10:14:01 2004
Subject: RE: [lemonade] Channel updates for content conversion

Corby Wilson

is
any

---

From: [email-masked]
Date: Tue, 23 Nov 2004 09:41:23 -0600

> From: [email-masked] [mailto:[email-masked]]On
> Sent: Wednesday, November 10, 2004 9:30 PM
...
> --On 2004-11-10 9:01 PM -0500 Eric Burger <[email-masked]> 
> wrote:
> 
> > The feeling in the room today is that server-based would be FETCH
> > driven. What do you think of that?
> 
> The issue I see with FETCH is the lack of an 8bit clean path. The 
> converted data would still most likely require base64 
> encoding. I would 
> like to hear Mark's opinion on this. We could structure 
> conversion such 
> that it would work with FETCH, BINARY, and CHANNEL. My concern with 
> this is that it would require adding yet another capability, and as 
> Alexey mentioned, IMAP is growing *way* too many 
> capabilities. From an 
> engineering standpoint I think it's better to implement in-band 
> conversion in one place only. This eliminates the problems associated 

I prefer not to use FETCH as well. 

Another inelegance with using FETCH is that within the same 
FETCH command you could request MIME headers that say the 
Content-Type is one thing and then the conversion is returning
another. I know the syntax could be adjusted to clarify, but 
I still find it unappealing.

Mike

---

From: [email-masked]
Date: Wed, 24 Nov 2004 18:02:54 -0600

> From: ext Lyndon Nerenberg [mailto:[email-masked]]
> Sent: Tuesday, November 23, 2004 11:13 PM
...
> > I would like to see a mechanism that allowed for partial 
> retrieval in
> > the case of in-band.
> 
> Where would this be useful? Channel was instigated in part because 
> Fetch with partials was not sufficient for some clients. 
> Partial fetch 
> was being misused to fake out streaming downloads. That didn't work. 
> Channel is an attempt to fix that. So in the context of Channel, why 
> (and how) would a partial fetch be useful.

When Channel is used as a mechanism for streaming it wouldn't be 
useful, but you've overloaded CHANNEL to do conversions. A 
conversion does not necessarily imply a streaming use case. So 
the usefulness of partial retrieval would be the same as in 
FETCH.

Maybe I'm missing something from the discussion had in Vancouver. I 
read the minutes. I understood the CHANNEL conversion proposal as 
meeting option A)?

Mike

---

From: [email-masked]
Date: Mon, 29 Nov 2004 07:27:28 -0600

> Sent: Thursday, November 25, 2004 12:49 PM
...
> > I think of conversion as a function with a Domain and Range.
> > CHANNELCONVERSIONS returns the Range. You seem to be assuming
> > the Domain to be the universe of all content types, which is
> > reasonable. How do I know what the mapping is?
> 
> The client is assumed to know about the conversions it is performing. 
> E.g. one assumes that a reasonable client won't ask a server 
> to convert 
> audio/basic to text/plain.

Agreed. I'm assuming a client is completely intelligent from it's 
perspective. The problem is not the clients ability to guess a 
reasonable mapping. 

The problem is that the current definition of CHANNELCONVERSIONS
may "tease" a client into thinking a mapping is supported and 
then require a NO response to find out. This does not seem like
the best way to solve the problem. You end up with the following 
unfortunate exchange:

C: 1 CHANNELCONVERSIONS text
S: * CHANNELCONVERSIONS "text/plain"
S: 1 OK CHANNELCONVERSIONS Now guess the legal domain!

C: 2 CHANNEL (inband) (1 2) "text/plain"
S: 2 NO CHANNEL Sorry, no cigar. Try some other Content-Type.

In this case 1.2 may have been application/pdf, but the server only
supports conversions from text/html -> text/plain, for example.

> 
> Enumerating the list of all possible transformations is potentially 
> very verbose. 

It is verbose because you have allowed it to be verbose. I see no 
value in allowing a NIL argument to get the entire Range. This is a 
very simple problem to solve. You change the syntax to have 
CHANNELCONVERSIONS take an element of the Domain and return the 
range for that element only.

C: 1 CHANNELCONVERSIONS application/pdf
S: * CHANNELCONVERSIONS "text/plain"
S: 1 OK CHANNELCONVERSIONS This is entire range

This is more in line with how a mobile client will want to use it.

For example, let's say a mobile client has a Word viewer but no PDF
viewer. It will be interested in asking what are it's options
specifically for PDF, not the fact that the server might support
conversion to plain text.

> is a strong hint that the conversion will work. If the client asks to 
> convert that image/jpeg to a text/plain, it deserves what it gets. (A 
> NO response.)

It does not deserve a NO response if it is asked to convert 
application/pdf to text/plain.

Mike

---

From: [email-masked]
Date: Mon, 29 Nov 2004 09:53:41 -0600

> From: ext Eric Burger [mailto:[email-masked]]
> Sent: Monday, November 29, 2004 10:32 AM
...
> Sorry, but the conversion
> 
>    application/pdf -> audio/basic
> 
> makes sense.  We call it "Text-to-Speech".  Mail readers will 
> render the pdf

Agreed. I thought of that after I sent the response. There are
many like this.

> What's the point of this?  One cannot make an algorithmic or protocol
> mandate of what conversions make sense.  Conversions that 
> don't seem to make
> sense today may make sense tomorrow.

The point is made in my second response.

The point is not that a client needs to be told what are sensible 
conversions. Not at all!

The client should be told what are the legal mappings supported 
by the server so it does not ask for pointless, but sensible, 
conversions. Pointless because the server does not support the 
mapping. See my other post for an example.

Mike