Subject: [REMOTEUI] scoping to synchronizing distributed XML DOM's

From: Dave Raggett <[email-masked]>
Date: Thu, 4 Aug 2005 18:58:04 +0100 (BST)

This is a follow up to the talk I presented at the Remote UI BoF
on 2nd August, see:

   http://www.w3.org/2005/Talks/0802-dsr-ui/

I am assuming that everyone on this list is familiar with the
benefits of using markup for representing user interfaces and
the importance of a clean separation between presentation, data
and control for developing applications. XML has been applied to
both the user interface and to the data. and as such it makes
sense to generalize the synchronization mechanism proposed in the
BoF to cover both cases.

In short, I believe that it would be valuable to develop protocol
support for synchronising distributed XML DOM's. This would
complement existing W3C work on multimodal interaction and device 
independence. For MMI, it would support the synchronization of 
shared data models as well as distributed multimodal user 
interfaces. For DI, it would support the representation of device 
capabilities and configuration as DOM nodes when these nodes reflect 
properties of remote devices, e.g. remaining paper supplies in a 
printer, and also enable thin clients to take part in complex web 
applications through a re-factoring of the markup between devices 
and servers.

As was discussed in the BoF, it makes sense to factor out work
on establishing the session context and other aspects related to
device coordination, and to instead focus on just what is needed to
synchronize distributed XML DOM's. The basic scenario is where two
devices hold copies of the same XML document, which could represent
the user interface or a shared data model. DOM events can be raised
in either device and need to be propagated to the other device.
Similarly changes to the XML in one device needs to be propagated
to the other device. The fact that the DOM is distributed is
hidden from the code that is manipulating the DOM. This code could
be a markup language interpreter or scripting engine.

This is summarised in slide form at:

   http://www.w3.org/2005/Talks/0802-dsr-ui/#[12]

The DOM is defined by a suite of W3C Recommendations that specify
platform and language neutral interfaces in IDL with bindings to
EMAScript and other languages. A DOM document is a hierarchy of
DOM Nodes which occur as elements, attributes, strings and other
derived types. The DOM Node interface provides the means to traverse
the node hierarchy and to mutate it through the addition and removal
of nodes. There is also a rich model of events with the means to
handle events prior to their delivery to the target node, or
afterwards, in a capture/buble model. For more details see:

   http://www.w3.org/DOM/

DOM Events are formally defined in IDL and should thus be amenable
to marshalling as messages in a message exchange protocol. More
recently, W3C has been looking at extending the DOM to support
interpreted user input expressed in XML in application dependent
markup and annotated with annotations for such things as timestamps,
confidence scores, input modality, alternative recognition hypotheses
and other information of interest, see:

   http://www.w3.org/TR/emma/

EMMA is expected to progress to Last Call Working Draft status
within a few weeks from now, and is intended for a wide variety
of uses, e.g. speech recognition, pen gestures, interpreted
keystrokes, etc.

A protocol to support the distribution of DOM events between
synchronized DOM's will need to take the DOM event model into
account. This includes such things as how to identify the event
target (a node in the node hierarchy), and the means for event
listeners to stop propagation in either the capture or bubble
phases. This is simple when all of the event listeners are in
one device, but when some are on one device and some are on
another, things get a little more complicated.

For DOM updates, I would like to alert people to the work on
"XUpdate" done several years ago. See:

   http://xmldb-org.sourceforge.net/xupdate/

XUpdate serializes DOM Mutation operations as XML messages, and
uses XPath to identify where in the node hierarchy a particular
operation should be applied. The use of XML to serialize DOM
operations makes for a declarative representation and can be
straightforwardly applied to IDL datatypes. The semantics are
well defined compared with shipping program fragments and this
should also simplify security considerations.

In applications involving multiple devices, or multiple modality
components, such as proposed in the W3C Multimodal Interaction
Architecture, there may be more than two instances of the DOM
requiring synchronization. Different parts of the DOM tree may
be "owned" by different devices. It make sense to consider what
kind of access rights model is needed. One possibility is for
annotating nodes as being unmutable, or mutable only by the owner,
or mutable by all participants.

A more complicated approach would be to implement some kind of
transactional model with commits and roll-backs. This is probably 
overkill for the intended purpose of supporting remote user 
interfaces, but it may be possible for the application to support 
something like this at a higher level without burdening the DOM 
synchronization protocol itself.

In addition to incremental changes, it will also be necessary to
enable to replacement of an entire DOM tree or subtree by another.
This is clearly important when starting up. Although session
initiation is out of scope for the synchronisation primitives,
it is worth working through a simple scenario to provide a fuller
context. One possibility is for a client device to use SIP to
establish an application session. Once a control path has been
established to a server, the server can then send control messages
to the client to load the user interface and shared data model.
synchronization messages can then be exchanged as the user
interacts with the remote application. Finally control messages
outside of the scope of the synchronization protocol can be used
to terminate the application and release resources.

Some assumptions include:

   a) the fact that the DOM is distributed can be hidden from
      the code (markup interpreter or script) operating on the DOM

   b) the synchronization protocol doesn't need to be aware of
      the semantics of the markup that is being synchronized.

   c) the synchronization occurs at a fairly loose level that
      allows for a simple approach to propagating changes.

   d) latency considerations are important and the user interface
      should not block on waiting for replies from other devices

- -- 
  Dave Raggett <[email-masked]>  W3C lead for multimodal interaction
  http://www.w3.org/People/Raggett +44 1225 866240 (or 867351)

_______________________________________________
REMOTEUI mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/remoteui

---

From: "Eric Burger" <[email-masked]>
Date: Sun, 7 Aug 2005 16:49:33 -0400

I would also like to highlight some of Dave's points below.  Namely, the
protocol must take into account the DOM event model.

The reason I consider this important is because XML DOM synchronization
is a well characterized problem space.  It puts on a number of important
AND USEFUL constraints on the solution space for the synchronization
protocol.  These constraints give us the opportunity to create an
efficient, usable protocol.

Were we to focus on an abstract synchronization protocol, I fear we
would (1) never finish and (2) if we did finish, no one would be able to
implement it.

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Dave Raggett
Sent: Thursday, August 04, 2005 1:58 PM
To: [email-masked]
Subject: [REMOTEUI] scoping to synchronizing distributed XML DOM's

   http://www.w3.org/DOM/

- --