Subject: Distributed Multimodal Synchronization Protocol [dmsp]

From: Chris Cross <[email-masked]>
Date: Tue, 7 Mar 2006 13:48:12 -0500

Hi Everyone,
I'm sending a note to your list to make the members aware of this activity.
The I-D "Distributed Multimodal Synchronization Protocol " (dmsp) can be
found at http://www.ietf.org/internet-drafts/draft-engelsma-dmsp-01.txt. We
have a mailing list at [email-masked] and a twice monthly phone conference.

Dmsp is being developed to enable distributed multimodal systems. I've been
invited to present an overview at both the apps and rai general meetings in
Dallas. We welcome your attendance at these presentations and interested
individuals' participation in our mail list and conference calls.

At the bottom of this note is a draft charter we've discussed on the list
that gives a more detailed description of dmsp.

thanks,
chris

Chris Cross
Multimodal Browser Architect
_________________________
IBM Boca Raton
[email-masked]
voice 561.862.2102
mobile 561.317.0700
fax 501.641.6727

The convergence of wireless communications with information technology and
the miniaturization of computing platforms have resulted in advanced mobile
devices that offer high resolution displays, application programs with
graphical user interfaces, and access to the internet through full function
web browsers.

Mobile phones now support most of the functionality of a laptop computer.
However the miniaturization that has made the technology possible and
commercially successful also puts constraints on the user interface. Tiny
displays and keypads significantly reduce the usability of application
programs.

Multimodal user interfaces, UIs that offer multiple modes of interaction,
have been developed that greatly improve the usability of mobile devices.
In particular multimodal UIs that combine speech and graphical interaction
are proving themselves in the marketplace.

However, not all mobile devices provide the computing resources to perform
speech recognition and synthesis locally on the device. For these devices
it is necessary to distribute the speech modality to a server in the
network.

The Distributed Multimodal Working Group will develop the protocols
necessary to control, coordinate, and synchronize distributed modalities in
a distributed Multimodal system. There are several protocols and standards
necessary to implement such a system including DSR and AMR speech
compression, session control, and media streaming. However, the DM WG will
focus exclusively on the synchronization of modalities being rendered
across a network, in particular Graphical User Interface and Voice Servers.

The DM WG will develop an RFC for a Distributed Multimodal Synchronization
Protocol that defines the logical message set to effect synchronization
between modalities and enough background on the expected multimodal system
architecture (or reference architecture defined elsewhere in W3C or OMA) to
present a clear understanding of the protocol. It will investigate existing
protocols for the transport of the logical synchronization messages and
develop an RFC detailing the message format for commercial alternatives,
including, possibly, HTTP and SIP.

While not being limited to these, for simplicity of the scope the protocol
will assume RTP for carriage of media, SIP and SDP for session control, and
DSR and AMR for speech compression. The working group will not consider the
authoring of applications as it will be assumed that this will be done with
existing W3C markup standards such as XHTML and VoiceXML and commercial
programming languages like Java and C/C++.

It is expected that we will coordinate our work in the IETF with the W3C
Multimodal Interaction Work Group.

The following are our goals for the Working Group.

Date Milestone
TBD Submit Internet Draft Describing DMSP (standards track)
TBD Submit Drafts to IESG for publication
TBD 2006 Submit DMSP specification to IESG

_______________________________________________
Dmsp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/dmsp

---

From: Graham Klyne <[email-masked]>
Date: Wed, 08 Mar 2006 17:48:56 +0000

Chris Cross wrote:
> Hi Everyone,
> I'm sending a note to your list to make the members aware of this
> activity. The I-D "Distributed Multimodal Synchronization Protocol "
> (dmsp) can be found at
> http://www.ietf.org/internet-drafts/draft-engelsma-dmsp-01.txt.

Chris,

Very interesting to see this pop up... I'm thinking about the web view
synchronization problem in a couple of completely different contexts, and I note
that the diagram on page 7 of your Internet draft is almost exactly what I have
been proposing in another context [1].

But this draft seems to combine the basic event propagation model with details
of the specific events used for media synchronization, and I'd like to suggest
that, if this work goes ahead, that the event distribution protocol be defined
separately from the details of the events themselves (e.g. like SMTP/RFC2822 for
email).  Then other applications could use the same event distribution model for
completely different purposes.  (E.g. I have some similar event distribution
requirements for web-based home-control applications.) I also note that existing
work on publish/subscribe systems and/or instant messaging presence protocols
might be used for the protocol.

Turning to the events themselves, I think that to deploy in a Web context it
would be sensible for event types to be identified by URIs rather than binary
codes.  In my own private work to date, I am using a simple event model that
consists of event type and event source, both URIs, and arbitrary additional
payload based on the event type.

Unfortunately, I shall not be present in Dallas to participate in the debate.  I
do think that the requirement that this proposal addresses is symptomatic of a
wider gap in Web/Internet application architecture, and it would be good to see
a simple common solution emerge.  I seem to recall that, some time ago, Lisa
Dussault was discussing some thoughts about notification systems for Internet
applications, which may well have a part to play in this debate.

#g
--

[1] http://wiki.oss-watch.ac.uk/InterPortletCommunicationConsideredHarmful

Chris Cross wrote:
> Hi Everyone,
> I'm sending a note to your list to make the members aware of this
> activity. The I-D "Distributed Multimodal Synchronization Protocol "
> (dmsp) can be found at
> http://www.ietf.org/internet-drafts/draft-engelsma-dmsp-01.txt. We have
> a mailing list at [email-masked] and a twice monthly phone conference.
> 
> Dmsp is being developed to enable distributed multimodal systems. I've
> been invited to present an overview at both the apps and rai general
> meetings in Dallas. We welcome your attendance at these presentations
> and interested individuals' participation in our mail list and
> conference calls.
> 
> At the bottom of this note is a draft charter we've discussed on the
> list that gives a more detailed description of dmsp.
> 
> thanks,
> chris
> 
> 
> Chris Cross
> The convergence of wireless communications with information technology
> and the miniaturization of computing platforms have resulted in advanced
> mobile devices that offer high resolution displays, application programs
> with graphical user interfaces, and access to the internet through full
> function web browsers.
> 
> Mobile phones now support most of the functionality of a laptop
> computer. However the miniaturization that has made the technology
> possible and commercially successful also puts constraints on the user
> interface. Tiny displays and keypads significantly reduce the usability
> of application programs.
> 
> Multimodal user interfaces, UIs that offer multiple modes of
> interaction, have been developed that greatly improve the usability of
> mobile devices. In particular multimodal UIs that combine speech and
> graphical interaction are proving themselves in the marketplace.
> 
> However, not all mobile devices provide the computing resources to
> perform speech recognition and synthesis locally on the device. For
> these devices it is necessary to distribute the speech modality to a
> server in the network.
> 
> necessary to control, coordinate, and synchronize distributed modalities
> in a distributed Multimodal system. There are several protocols and
> standards necessary to implement such a system including DSR and AMR
> speech compression, session control, and media streaming. However, the
> DM WG will focus exclusively on the synchronization of modalities being
> rendered across a network, in particular Graphical User Interface and
> Voice Servers.
> 
> The DM WG will develop an RFC for a Distributed Multimodal
> Synchronization Protocol that defines the logical message set to effect
> synchronization between modalities and enough background on the expected
> multimodal system architecture (or reference architecture defined
> elsewhere in W3C or OMA) to present a clear understanding of the
> protocol. It will investigate existing protocols for the transport of
> the logical synchronization messages and develop an RFC detailing the
> message format for commercial alternatives, including, possibly, HTTP
> and SIP.
> 
> While not being limited to these, for simplicity of the scope the
> protocol will assume RTP for carriage of media, SIP and SDP for session
> control, and DSR and AMR for speech compression. The working group will
> not consider the authoring of applications as it will be assumed that
> this will be done with existing W3C markup standards such as XHTML and
> VoiceXML and commercial programming languages like Java and C/C++.
> 

-- 
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

---

From: Chris Cross <[email-masked]>
Date: Thu, 9 Mar 2006 11:29:58 -0500

Graham,
Its a good suggestion to look at other publish/subscribe models for
guidance. Can you point to other RFCs in this domain?

I'm interested in the details of your URI event model. Have you published
anything? Its important to note that in a multimodal interaction,
synchronization of distributed modalities must take place with  sub-second
latency for the usability of the application to be worth the effort. We
concentrated first on a binary message set to reduce the latency of a
distributed multimodal system implemented over a relatively low bandwidth
cellular network. In that context I would be concerned that a URI scheme
would both inflate the message size and introduce additional turns in a
system that is very sensitive to network latency.

thanks,
chris

Chris Cross
Multimodal Browser Architect
_________________________
IBM Boca Raton
[email-masked]
voice 561.862.2102  t/l 975.2102
mobile 561.317.0700
fax 501.641.6727

             Graham Klyne                                                  
             <[email-masked]                                             
             g>                                                         To 
                                       Chris Cross/West Palm               
             03/08/2006 12:48          Beach/IBM@IBMUS                     
             PM                                                         cc 
                                       [email-masked],                  
                                       [email-masked],              
                                       [email-masked]                       
                                                                   Subject 
                                       Re: Distributed Multimodal          
                                       Synchronization Protocol [dmsp]     

                                                                           

Chris,

Very interesting to see this pop up... I'm thinking about the web view
synchronization problem in a couple of completely different contexts, and I
note
that the diagram on page 7 of your Internet draft is almost exactly what I
have
been proposing in another context [1].

But this draft seems to combine the basic event propagation model with
details
of the specific events used for media synchronization, and I'd like to
suggest
that, if this work goes ahead, that the event distribution protocol be
defined
separately from the details of the events themselves (e.g. like
SMTP/RFC2822 for
email).  Then other applications could use the same event distribution
model for
completely different purposes.  (E.g. I have some similar event
distribution
requirements for web-based home-control applications.) I also note that
existing
work on publish/subscribe systems and/or instant messaging presence
protocols
might be used for the protocol.

Turning to the events themselves, I think that to deploy in a Web context
it
would be sensible for event types to be identified by URIs rather than
binary
codes.  In my own private work to date, I am using a simple event model
that
consists of event type and event source, both URIs, and arbitrary
additional
payload based on the event type.

Unfortunately, I shall not be present in Dallas to participate in the
debate.  I
do think that the requirement that this proposal addresses is symptomatic
of a
wider gap in Web/Internet application architecture, and it would be good to
see
a simple common solution emerge.  I seem to recall that, some time ago,
Lisa
Dussault was discussing some thoughts about notification systems for
Internet
applications, which may well have a part to play in this debate.

#g
--