Subject: [dispatch] Multi-streams for telepresence description of work

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Tue, 18 May 2010 11:30:02 -0700

Hi Folks,

Many of you are aware of telepresence systems (video conferencing on
steroids..:), fewer probably are aware of the fact that the current
systems are not interoperable with each other. After talking with Mary
and Cullen, we have a rough draft of a charter to work on this problem
in DISPATCH,  for discussion on the mailing list.

We don't yet know whether a working group would need to be spun out or
not - that will be clarified as we go.

We think there is enough background info in the work description to
start a discussion, and plan to send out use cases for discussion
shortly.

Who is the we? Several makers and providers of telepresence systems have
gotten together to define the main problem in interoperability - the
handling of multi-streams.

Thanks for your thoughts-

Allyn

Multi-streams for Telepresence Description of Work

Background

One branch of video conferencing has evolved that is focused on
immersive "being there" experience.  Referred to in various ways such as
virtual conferencing, telepresence or media spaces, early systems were
mainly research projects or business systems with limited deployments.
In recent years telepresence systems have seen considerable market
success.  Following the model of early systems, the first wave of
commercial systems have been typically located in specially designed
single-purpose rooms with multiple relatively large displays permitting
life size image reproduction, multiple cameras, encoders, decoders and
microphones.  These systems have several important characteristics that
are different from more traditional video conferencing systems.  

The first difference concerns controlling the visual viewpoint in order
to improve participant nonverbal communication. These systems preserve
essential group meeting characteristics such as eye contact, group
gestures, seating order and spatial audio by carefully orchestrating the
miking and camera angles at each of the sites . This is distinct from
the more traditional approach where the geometric relationship between
media streams is not used to preserve inter-stream communication aspects
such as eye contact and group dynamics.  

A second difference is manipulation of the environment to improve
immersion.  With telepresence systems, cinematographic aspects of the
local environment reproduction are carefully planned including color,
table shape, seating and lighting so that when combined with large high
quality displays, a strong sense of a "trompe l'oeil" or "being there"
immersive experience is created.  Typical video conference systems do
not include these considerations.

As telepresence video systems have become successful in the market,
manufacturers have started exploring delivery of the nonverbal
communication and immersive values of telepresence via smaller, less
expensive and more flexible video conferencing systems for a variety of
venues, such as individual offices, homes and kiosks. These are also
telepresence systems, since the audio and video quality is high enough
to allow clear image reproduction for nonverbal communication, they are
able to send and receive multiple media streams, and large coordinated
multi image displays are available for immersive installations.   As the
industry develops, the line between telepresence and video conferencing
may become blurred as nonverbal communication and immersive
installations become broadly available.

Problem

Although telepresence systems are based on open standards such as RTP,
SIP, H.264, H.323 suite, they cannot easily interoperate with each other
without operator assistance and expensive additional equipment that
translates from one vendor to another. It would be like having to make
sure all parties are on the same equipment (and network) when making a
telephone call.  A major factor in the inability of Telepresence systems
to work with each other is that there is no standard description of the
multiple streams that comprise the media flows. 

For example, in a multiple screen conference, the video and audio
streams sent from remote participants must be understood by receivers so
that they can be presented in a coherent and life-like manner. This
includes the ability to present the remote participants at their true
size for their apparent distance, while maintaining correct eye contact,
gesticular cues, and simultaneously providing a spatial audio sound
stage consistent with the video presentation.  The receiving device that
decides how to display the incoming information needs to understand a
number of variables such as the spatial position of the speaker, the
field of view of the cameras, the camera zoom, which media stream is
related to each of the displays, etc. 

Charter

The Telepresence Multi-Streams work item in DISPATCH is chartered to
define and specify the content of media multi-stream messages and the
way these will be transported. 

This work is aimed at providing a standard for the exchange of media
semantics information that will foster interoperable end stations and
conference bridges. This work item will specify the variables that
describe the semantics of the media streams and the recommended behavior
to achieve interoperability.  

This requires considering current widely deployed use cases, such as
single and multiple screens, multi-point, Scalable Video Coding (SVC),
as well as cases that are expected to be implemented using the protocol
framework produced by this work item.  The methodology for describing
the variables must allow extensibility of the variables, since
telepresence is still a young technology and may have use cases that are
not currently considered.

The work item will identify use cases, define requirements, and define a
method for describing and transporting information about multiple media
streams, including a specification of variables required to support the
use cases. This work item will consider the reuse of existing IETF
protocols and produce an architecture/protocol framework document
describing the protocols required to be implemented to support this
functionality.  The document will identify any enhancements required to
existing protocols as well as describing new protocol(s) for
interoperable multi-streams negotiation that may be required.

Scope

The scope includes both systems that provide a fully immersive
experience, and systems that interwork with them and therefore need to
understand the same multiple stream semantics.

The focus of this work is on audio and video multiple streams, however
other media types may be considered.  Definition of new application
protocols is not within the scope of this work

[Is there anything else we need to explicitly say is out of scope?]

The group will produce

- Requirements and use cases

- Architectural Framework describing the protocols required to be
implemented to support this functionality and identifying existing
protocol enhancements and new protocol functionality required

- Specification of a new protocol to support telepresence multi-streams
[if required]

Goals and Milestones

Nov 2010 

Use Cases and Requirements to IESG as Informational RFC 

March 2011 

Architecture to IESG as Informational RFC 

March 2011

Revise Charter [IF new protocol is not required]

Nov 2011 

Submit protocol draft to IESG as Proposed Standard RFC

---

From: Peter Musgrave <[email-masked]>
Date: Wed, 19 May 2010 06:21:46 +0200

Hi Allyn,

I am interested in having this work start. I suspect it will likely warrant
a working group at the appropriate time.

The charter as written is VERY general. I would suspect that SIP/SDP is
going to be the common ground. Is it worth making this explicit in the
charter? If not perhaps the reasons why not could be stated in the charter.
(My understanding is that most of these systems do use SIP but that they
differ in number of sessions per endpoint and how the media is packaged into
RTP streams etc.)

Is there other IETF work which we can see has relevance and should be
mentioned as tools which can be brought to bear? (xcon? mediactrl?)

Is there interest in ensuring interop of TP with legacy SIP video
conferencing systems as part of this work? Should that be part of the
charter?

Regards,

Peter Musgrave

On Tue, May 18, 2010 at 8:30 PM, Allyn Romanow (allyn) <[email-masked]>wrote:

> steroids..:), fewer probably are aware of the fact that the current systems
> are not interoperable with each other. After talking with Mary and Cullen,
> we have a rough draft of a charter to work on this problem in DISPATCH,  for
> discussion on the mailing list.
>
>
>
> We don’t yet know whether a working group would need to be spun out or not
> – that will be clarified as we go.
>
>
>
> We think there is enough background info in the work description to start a
> discussion, and plan to send out use cases for discussion shortly.
>
>
>
> gotten together to define the main problem in interoperability – the
> *Multi-streams for Telepresence Description of Work*
>
> * *
>
> * *
>
> * *
>
> *Background*
>
> One branch of video conferencing has evolved that is focused on immersive
> “being there” experience.  Referred to in various ways such as virtual
> conferencing, telepresence or media spaces, early systems were mainly
> research projects or business systems with limited deployments.  In recent
> years telepresence systems have seen considerable market success.  Following
> the model of early systems, the first wave of commercial systems have been
> typically located in specially designed single-purpose rooms with multiple
> relatively large displays permitting life size image reproduction, multiple
> cameras, encoders, decoders and microphones.  These systems have several
> important characteristics that are different from more traditional video
> conferencing systems.
>
>
>
> The first difference concerns controlling the visual viewpoint in order to
> improve participant nonverbal communication. These systems preserve
> essential group meeting characteristics such as eye contact, group gestures,
> seating order and spatial audio by carefully orchestrating the miking and
> camera angles at each of the sites . This is distinct from the more
> traditional approach where the geometric relationship between media streams
> is not used to preserve inter-stream communication aspects such as eye
> contact and group dynamics.
>
>
>
> immersion.  With telepresence systems, cinematographic aspects of the local
> environment reproduction are carefully planned including color, table shape,
> seating and lighting so that when combined with large high quality displays,
> a strong sense of a "trompe l'oeil" or "being there" immersive experience is
> created.  Typical video conference systems do not include these
> considerations.
>
>
>
> manufacturers have started exploring delivery of the nonverbal communication
> and immersive values of telepresence via smaller, less expensive and more
> flexible video conferencing systems for a variety of venues, such as
> individual offices, homes and kiosks. These are also  telepresence systems,
> since the audio and video quality is high enough to allow clear image
> reproduction for nonverbal communication, they are able to send and receive
> multiple media streams, and large coordinated multi image displays are
> available for immersive installations.   As the industry develops, the line
> between telepresence and video conferencing may become blurred as nonverbal
> communication and immersive installations become broadly available.
>
>
>
> *Problem*
>
> Although telepresence systems are based on open standards such as RTP, SIP,
> H.264, H.323 suite, they cannot easily interoperate with each other without
> operator assistance and expensive additional equipment that translates from
> one vendor to another. It would be like having to make sure all parties are
> on the same equipment (and network) when making a telephone call.  A major
> factor in the inability of Telepresence systems to work with each other is
> that there is no standard description of the multiple streams that comprise
> the media flows.
>
>
>
> For example, in a multiple screen conference, the video and audio streams
> sent from remote participants must be understood by receivers so that they
> can be presented in a coherent and life-like manner. This includes the
> ability to present the remote participants at their true size for their
> apparent distance, while maintaining correct eye contact, gesticular cues,
> and simultaneously providing a spatial audio sound stage consistent with the
> video presentation.  The receiving device that decides how to display the
> incoming information needs to understand a number of variables such as the
> spatial position of the speaker, the field of view of the cameras, the
> camera zoom, which media stream is related to each of the displays, etc.
>
>
>
> *Charter*
>
> The Telepresence Multi-Streams work item in DISPATCH is chartered to define
> and specify the content of media multi-stream messages and the way these
> will be transported.
>
> describe the semantics of the media streams and the recommended behavior to
> achieve interoperability.
>
>
>
> This requires considering current widely deployed use cases, such as single
> and multiple screens, multi-point, Scalable Video Coding (SVC), as well as
> cases that are expected to be implemented using the protocol framework
> produced by this work item.  The methodology for describing the variables
> must allow extensibility of the variables, since telepresence is still a
> young technology and may have use cases that are not currently considered.
>
>
>
> streams, including a specification of variables required to support the use
> cases. This work item will consider the reuse of existing IETF protocols and
> produce an architecture/protocol framework document describing the protocols
> required to be implemented to support this functionality.  The document will
> identify any enhancements required to existing protocols as well as
> describing new protocol(s) for interoperable multi-streams negotiation that
> may be required.
>
>
>
> *Scope*
>
> The scope includes both systems that provide a fully immersive experience,
> and systems that interwork with them and therefore need to understand the
> same multiple stream semantics.
>
>
>
> [Is there anything else we need to explicitly say is out of scope?]**
>
>
>
> *The group will produce*
>
> implemented to support this functionality and identifying existing protocol
> enhancements and new protocol functionality required
>
>
>
> - Specification of a new protocol to support telepresence multi-streams [if
> required]
>
>
>
> *Goals and Milestones*
>
> Nov 2010
>
> _______________________________________________
> dispatch mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dispatch
>
>

---

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Tue, 18 May 2010 21:50:23 -0700

Hi Peter,
You've raised excellent points. See inline for a few brief comments, and
I'll look to others to elaborate (or disagree).
 
best,
Allyn

________________________________

	From: Peter Musgrave [mailto:[email-masked]] 
	Sent: Tuesday, May 18, 2010 9:22 PM
	To: Allyn Romanow (allyn)
	Cc: IETF DISPATCH list
	Subject: Re: [dispatch] Multi-streams for telepresence
description of work

	Hi Allyn,  

	I am interested in having this work start. I suspect it will
likely warrant a working group at the appropriate time.  

	The charter as written is VERY general. I would suspect that
SIP/SDP is going to be the common ground. Is it worth making this
explicit in the charter? If not perhaps the reasons why not could be
stated in the charter. (My understanding is that most of these systems
do use SIP but that they differ in number of sessions per endpoint and
how the media is packaged into RTP streams etc.)
	[alr] Certainly RTP is used without question, and there has been
discussion about limiting to RTP streams only, not other types of data.
This is still an open issue to be discussed.
	 
	Although I (and some others) initially assumed that we would use
SDP for signaling, after further thought Steve Botsko and  others
suggested we might want to consider another protocol such as XMPP in
light of the nature of the messages we want to send. I think that it's
premature to specify this in the charter as I consider it part of the
the solution, and think it will become clear what protocol to use as we
specify requirements.

	Is there other IETF work which we can see has relevance and
should be mentioned as tools which can be brought to bear? (xcon?
mediactrl?)
	[alr] Good point. We definitely want to consider existing work,
and look to the experts in DISPATCH to help with this. As above, I'm not
sure if we can go there yet without requirements first- but centainly we
can be begin to look at potential tools.

	Is there interest in ensuring interop of TP with legacy SIP
video conferencing systems as part of this work? Should that be part of
the charter?
	[alr] My personal view is not too much, that it's more a matter
of going forward with a  common technology that allows us to
interoperate. I don't want us to be in the position of needing to tie
together systems that we already have, as that would too much limit our
approach. That said, people will want to incorporate the best of their
current technology. What do folks think? 

	Regards, 

	Peter Musgrave

	On Tue, May 18, 2010 at 8:30 PM, Allyn Romanow (allyn)
<[email-masked]> wrote:

		Hi Folks,

		Many of you are aware of telepresence systems (video
conferencing on steroids..:), fewer probably are aware of the fact that
the current systems are not interoperable with each other. After talking
with Mary and Cullen, we have a rough draft of a charter to work on this
problem in DISPATCH,  for discussion on the mailing list.

		We don't yet know whether a working group would need to
be spun out or not - that will be clarified as we go.

		We think there is enough background info in the work
description to start a discussion, and plan to send out use cases for
discussion shortly.

		Who is the we? Several makers and providers of
telepresence systems have gotten together to define the main problem in
interoperability - the handling of multi-streams.

		Allyn

		 

		Background

		One branch of video conferencing has evolved that is
focused on immersive "being there" experience.  Referred to in various
ways such as virtual conferencing, telepresence or media spaces, early
systems were mainly research projects or business systems with limited
deployments.  In recent years telepresence systems have seen
considerable market success.  Following the model of early systems, the
first wave of commercial systems have been typically located in
specially designed single-purpose rooms with multiple relatively large
displays permitting life size image reproduction, multiple cameras,
encoders, decoders and microphones.  These systems have several
important characteristics that are different from more traditional video
conferencing systems.  

		The first difference concerns controlling the visual
viewpoint in order to improve participant nonverbal communication. These
systems preserve essential group meeting characteristics such as eye
contact, group gestures, seating order and spatial audio by carefully
orchestrating the miking and camera angles at each of the sites . This
is distinct from the more traditional approach where the geometric
relationship between media streams is not used to preserve inter-stream
communication aspects such as eye contact and group dynamics.  

		A second difference is manipulation of the environment
to improve immersion.  With telepresence systems, cinematographic
aspects of the local environment reproduction are carefully planned
including color, table shape, seating and lighting so that when combined
with large high quality displays, a strong sense of a "trompe l'oeil" or
"being there" immersive experience is created.  Typical video conference
systems do not include these considerations.

		As telepresence video systems have become successful in
the market, manufacturers have started exploring delivery of the
nonverbal communication and immersive values of telepresence via
smaller, less expensive and more flexible video conferencing systems for
a variety of venues, such as individual offices, homes and kiosks. These
are also  telepresence systems, since the audio and video quality is
high enough to allow clear image reproduction for nonverbal
communication, they are able to send and receive multiple media streams,
and large coordinated multi image displays are available for immersive
installations.   As the industry develops, the line between telepresence
and video conferencing may become blurred as nonverbal communication and
immersive installations become broadly available.

		Problem

		Although telepresence systems are based on open
standards such as RTP, SIP, H.264, H.323 suite, they cannot easily
interoperate with each other without operator assistance and expensive
additional equipment that translates from one vendor to another. It
would be like having to make sure all parties are on the same equipment
(and network) when making a telephone call.  A major factor in the
inability of Telepresence systems to work with each other is that there
is no standard description of the multiple streams that comprise the
media flows. 

		For example, in a multiple screen conference, the video
and audio streams sent from remote participants must be understood by
receivers so that they can be presented in a coherent and life-like
manner. This includes the ability to present the remote participants at
their true size for their apparent distance, while maintaining correct
eye contact, gesticular cues, and simultaneously providing a spatial
audio sound stage consistent with the video presentation.  The receiving
device that decides how to display the incoming information needs to
understand a number of variables such as the spatial position of the
speaker, the field of view of the cameras, the camera zoom, which media
stream is related to each of the displays, etc. 

		Charter

		The Telepresence Multi-Streams work item in DISPATCH is
chartered to define and specify the content of media multi-stream
messages and the way these will be transported. 

		This work is aimed at providing a standard for the
exchange of media semantics information that will foster interoperable
end stations and conference bridges. This work item will specify the
variables that describe the semantics of the media streams and the
recommended behavior to achieve interoperability.  

		This requires considering current widely deployed use
cases, such as single and multiple screens, multi-point, Scalable Video
Coding (SVC), as well as cases that are expected to be implemented using
the protocol framework produced by this work item.  The methodology for
describing the variables must allow extensibility of the variables,
since telepresence is still a young technology and may have use cases
that are not currently considered.

		The work item will identify use cases, define
requirements, and define a method for describing and transporting
information about multiple media streams, including a specification of
variables required to support the use cases. This work item will
consider the reuse of existing IETF protocols and produce an
architecture/protocol framework document describing the protocols
required to be implemented to support this functionality.  The document
will identify any enhancements required to existing protocols as well as
describing new protocol(s) for interoperable multi-streams negotiation
that may be required.

		Scope

		The scope includes both systems that provide a fully
immersive experience, and systems that interwork with them and therefore
need to understand the same multiple stream semantics.

		The focus of this work is on audio and video multiple
streams, however other media types may be considered.  Definition of new
application protocols is not within the scope of this work

		[Is there anything else we need to explicitly say is out
of scope?]

		The group will produce

		- Architectural Framework describing the protocols
required to be implemented to support this functionality and identifying
existing protocol enhancements and new protocol functionality required

		- Specification of a new protocol to support
telepresence multi-streams [if required]

		Goals and Milestones

Nov 2010 

March 2011 

March 2011

Nov 2011

---

From: [email-masked]
Date: Wed, 19 May 2010 13:33:10 +0800

Hi,

I have two level of comments,

1. Feeling for the topic
Technically, I am quit glad to see such discussion about telepresence, 
which is trailblazer for VR(Virtual Reality) production. 

2. About the charter WG
One revolutionary application, such telepresence, would cover basic 
protocol level, application control level, and interworking issue. 

If the work is just focused on pure interworking issue, then the topics 
can be divided into smaller one, which might be in other basic protocols' 
WG's scope. But if the application itself is new, and has some new 
characteristic, more work should be proposed. 

If the work is also focused on telepresence application, which is a 
conferencing system, the xcon WG might be proper one for application level 
definition.

If the charter also want to talk the esential aspect of telepresence(or 
more abstractly, about VR), such as how to using SDP, or relative protocol 
to describe the streams and it's collaboration, it would be quite new 
topic. One new WG might be better.

I find the charter seems covers all the three thing. So, I suggest you 
classify such different level requirement. Then, we may find which part is 
proper for which WG, and which part should be covered by new WG.

Thanks,

Gao

===================================
 Zip    : 210012
 Tel    : 87211
 Tel2   :(+86)-025-52877211
 e_mail : [email-masked]
===================================

"Allyn Romanow (allyn)" <[email-masked]> 
发件人:  [email-masked]
2010-05-19 02:30

收件人
"IETF DISPATCH list" <[email-masked]>
抄送

主题
[dispatch] Multi-streams for telepresence description of work

Hi Folks,
 
Many of you are aware of telepresence systems (video conferencing on 
steroids..:), fewer probably are aware of the fact that the current 
systems are not interoperable with each other. After talking with Mary and 
Cullen, we have a rough draft of a charter to work on this problem in 
DISPATCH,  for discussion on the mailing list.
 
We don’t yet know whether a working group would need to be spun out or 
not – that will be clarified as we go.
 
We think there is enough background info in the work description to start 
a discussion, and plan to send out use cases for discussion shortly.

 
Background
One branch of video conferencing has evolved that is focused on immersive 
“being there” experience.  Referred to in various ways such as virtual 
conferencing, telepresence or media spaces, early systems were mainly 
research projects or business systems with limited deployments.  In recent 
years telepresence systems have seen considerable market success. 
Following the model of early systems, the first wave of commercial systems 
have been typically located in specially designed single-purpose rooms 
with multiple relatively large displays permitting life size image 
reproduction, multiple cameras, encoders, decoders and microphones.  These 
systems have several important characteristics that are different from 
more traditional video conferencing systems. 
 
The first difference concerns controlling the visual viewpoint in order to 
improve participant nonverbal communication. These systems preserve 
essential group meeting characteristics such as eye contact, group 
gestures, seating order and spatial audio by carefully orchestrating the 
miking and camera angles at each of the sites . This is distinct from the 
more traditional approach where the geometric relationship between media 
streams is not used to preserve inter-stream communication aspects such as 
eye contact and group dynamics. 
 
A second difference is manipulation of the environment to improve 
immersion.  With telepresence systems, cinematographic aspects of the 
local environment reproduction are carefully planned including color, 
table shape, seating and lighting so that when combined with large high 
quality displays, a strong sense of a "trompe l'oeil" or "being there" 
immersive experience is created.  Typical video conference systems do not 
include these considerations.
 
As telepresence video systems have become successful in the market, 
manufacturers have started exploring delivery of the nonverbal 
communication and immersive values of telepresence via smaller, less 
expensive and more flexible video conferencing systems for a variety of 
venues, such as individual offices, homes and kiosks. These are also 
telepresence systems, since the audio and video quality is high enough to 
allow clear image reproduction for nonverbal communication, they are able 
to send and receive multiple media streams, and large coordinated multi 
image displays are available for immersive installations.   As the 
industry develops, the line between telepresence and video conferencing 
may become blurred as nonverbal communication and immersive installations 
become broadly available.
 
Problem
 
For example, in a multiple screen conference, the video and audio streams 
sent from remote participants must be understood by receivers so that they 
can be presented in a coherent and life-like manner. This includes the 
ability to present the remote participants at their true size for their 
apparent distance, while maintaining correct eye contact, gesticular cues, 
and simultaneously providing a spatial audio sound stage consistent with 
the video presentation.  The receiving device that decides how to display 
the incoming information needs to understand a number of variables such as 
the spatial position of the speaker, the field of view of the cameras, the 
camera zoom, which media stream is related to each of the displays, etc. 
 
Charter
The Telepresence Multi-Streams work item in DISPATCH is chartered to 
define and specify the content of media multi-stream messages and the way 
these will be transported. 
This work is aimed at providing a standard for the exchange of media 
semantics information that will foster interoperable end stations and 
conference bridges. This work item will specify the variables that 
describe the semantics of the media streams and the recommended behavior 
to achieve interoperability. 
 
This requires considering current widely deployed use cases, such as 
single and multiple screens, multi-point, Scalable Video Coding (SVC), as 
well as cases that are expected to be implemented using the protocol 
framework produced by this work item.  The methodology for describing the 
variables must allow extensibility of the variables, since telepresence is 
still a young technology and may have use cases that are not currently 
considered.
 
The work item will identify use cases, define requirements, and define a 
method for describing and transporting information about multiple media 
streams, including a specification of variables required to support the 
use cases. This work item will consider the reuse of existing IETF 
protocols and produce an architecture/protocol framework document 
describing the protocols required to be implemented to support this 
functionality.  The document will identify any enhancements required to 
existing protocols as well as describing new protocol(s) for interoperable 
multi-streams negotiation that may be required.
 
Scope

 
The group will produce

 
Goals and Milestones

Nov 2010 

--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail is solely property of the sender's organization. This mail communication is confidential. Recipients named above are obligated to maintain secrecy and are not permitted to disclose the contents of this communication to others.
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the originator of the message. Any views expressed in this message are those of the individual sender.
This message has been scanned for viruses and Spam by ZTE Anti-Spam system.

---

From: stephen botzko <[email-masked]>
Date: Wed, 19 May 2010 06:48:02 -0400

Hi Peter

I am one of the "we" that Allyn refers to.  See inline for a comment.

Regards.
Stephen Botzko

On Wed, May 19, 2010 at 12:50 AM, Allyn Romanow (allyn) <[email-masked]>wrote:

>  ------------------------------
> *From:* Peter Musgrave [mailto:[email-masked]]
> *Sent:* Tuesday, May 18, 2010 9:22 PM
> *To:* Allyn Romanow (allyn)
> *Cc:* IETF DISPATCH list
> *Subject:* Re: [dispatch] Multi-streams for telepresence description of
> work
>
> Hi Allyn,
>
> [alr] Certainly RTP is used without question, and there has been discussion
> about limiting to RTP streams only, not other types of data. This is still
> an open issue to be discussed.
>
> Although I (and some others) initially assumed that we would use SDP for
> signaling, after further thought Steve Botsko and  others suggested we might
> want to consider another protocol such as XMPP in light of the nature of the
> messages we want to send. I think that it's premature to specify this in the
> charter as I consider it part of the the solution, and think it will become
> clear what protocol to use as we specify requirements.
>
> [alr] Good point. We definitely want to consider existing work, and look to
> the experts in DISPATCH to help with this. As above, I'm not sure if we can
> go there yet without requirements first- but centainly we can be begin to
> look at potential tools.
>
> [alr] My personal view is not too much, that it's more a matter of going
> forward with a  common technology that allows us to interoperate. I don't
> want us to be in the position of needing to tie together systems that we
> already have, as that would too much limit our approach. That said, people
> will want to incorporate the best of their current technology. What do folks
> think?
>
> [scb] Personally I think it is essential that TP devices be able to use
SIP/SDP to interoperate with traditional videoconferencing systems.  So I
think it is a requirement that this work *not preclude* SIP/SDP
interoperability, either with videoconferencing or ordinary phones.

Also, I think that TP calls will need to be be carried on existing [possibly
upgraded] infrastructure, including SIP proxies, SBCs, etc, as it is
undesirable to deploy a parallel infrastructure for TP.

I do not think it is a requirement for this work to fully accommodate
existing TP systems. While it would be a nice outcome if these systems could
be easily retrofitted to support this new work, it is much more important to
have a solid basis to go forward.

>> **
>> **
>> **
>> translates from one vendor to another. It would be like having to make sure
>> all parties are on the same equipment (and network) when making a telephone
>> call.  A major factor in the inability of Telepresence systems to work with
>> each other is that there is no standard description of the multiple streams
>> that comprise the media flows.
>>
>>
>>

---

From: <[email-masked]>
Date: Wed, 19 May 2010 13:05:01 +0200

Hi Allyn,

Cisco has recently open sourced under Apache license a protocol called Telepresence Interoperability Protocol. Also IMTC (International Multimedia Telecommunications Consortium) has setup an activity around TIP and Telepresence [1]. Would this proposed IETF work be overlapping or orthogonal to TIP and what IMTC is doing?

It's natural that IETF RAI area is a good fit for this type of standards, especially if we can take RTP, SIP, XMPP as the baseline. On the other hand, if a reasonable solution that is backed up by some major vendors is already being developed elsewhere, that will significantly shrik the succes chances of IETF being able to do something that will get deployed.

Regards,
    Markus

[1] http://blog.imtc.org/index.php/2010/03/09/announcing-the-creation-of-imtc-telepresence-multi-streaming-activity-group/
<http://blog.imtc.org/index.php/2010/03/09/announcing-the-creation-of-imtc-telepresence-multi-streaming-activity-group/>

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of ext Allyn Romanow (allyn)
Sent: 18 May, 2010 21:30
To: IETF DISPATCH list
Subject: [dispatch] Multi-streams for telepresence description of work

Hi Folks,

Many of you are aware of telepresence systems (video conferencing on steroids..:), fewer probably are aware of the fact that the current systems are not interoperable with each other. After talking with Mary and Cullen, we have a rough draft of a charter to work on this problem in DISPATCH,  for discussion on the mailing list.

We don't yet know whether a working group would need to be spun out or not - that will be clarified as we go.

We think there is enough background info in the work description to start a discussion, and plan to send out use cases for discussion shortly.

Who is the we? Several makers and providers of telepresence systems have gotten together to define the main problem in interoperability - the handling of multi-streams.

Background
One branch of video conferencing has evolved that is focused on immersive "being there" experience.  Referred to in various ways such as virtual conferencing, telepresence or media spaces, early systems were mainly research projects or business systems with limited deployments.  In recent years telepresence systems have seen considerable market success.  Following the model of early systems, the first wave of commercial systems have been typically located in specially designed single-purpose rooms with multiple relatively large displays permitting life size image reproduction, multiple cameras, encoders, decoders and microphones.  These systems have several important characteristics that are different from more traditional video conferencing systems.

The first difference concerns controlling the visual viewpoint in order to improve participant nonverbal communication. These systems preserve essential group meeting characteristics such as eye contact, group gestures, seating order and spatial audio by carefully orchestrating the miking and camera angles at each of the sites . This is distinct from the more traditional approach where the geometric relationship between media streams is not used to preserve inter-stream communication aspects such as eye contact and group dynamics.

A second difference is manipulation of the environment to improve immersion.  With telepresence systems, cinematographic aspects of the local environment reproduction are carefully planned including color, table shape, seating and lighting so that when combined with large high quality displays, a strong sense of a "trompe l'oeil" or "being there" immersive experience is created.  Typical video conference systems do not include these considerations.

As telepresence video systems have become successful in the market, manufacturers have started exploring delivery of the nonverbal communication and immersive values of telepresence via smaller, less expensive and more flexible video conferencing systems for a variety of venues, such as individual offices, homes and kiosks. These are also  telepresence systems, since the audio and video quality is high enough to allow clear image reproduction for nonverbal communication, they are able to send and receive multiple media streams, and large coordinated multi image displays are available for immersive installations.   As the industry develops, the line between telepresence and video conferencing may become blurred as nonverbal communication and immersive installations become broadly available.

Problem
Although telepresence systems are based on open standards such as RTP, SIP, H.264, H.323 suite, they cannot easily interoperate with each other without operator assistance and expensive additional equipment that translates from one vendor to another. It would be like having to make sure all parties are on the same equipment (and network) when making a telephone call.  A major factor in the inability of Telepresence systems to work with each other is that there is no standard description of the multiple streams that comprise the media flows.

For example, in a multiple screen conference, the video and audio streams sent from remote participants must be understood by receivers so that they can be presented in a coherent and life-like manner. This includes the ability to present the remote participants at their true size for their apparent distance, while maintaining correct eye contact, gesticular cues, and simultaneously providing a spatial audio sound stage consistent with the video presentation.  The receiving device that decides how to display the incoming information needs to understand a number of variables such as the spatial position of the speaker, the field of view of the cameras, the camera zoom, which media stream is related to each of the displays, etc.

Charter
The Telepresence Multi-Streams work item in DISPATCH is chartered to define and specify the content of media multi-stream messages and the way these will be transported.
This work is aimed at providing a standard for the exchange of media semantics information that will foster interoperable end stations and conference bridges. This work item will specify the variables that describe the semantics of the media streams and the recommended behavior to achieve interoperability.

This requires considering current widely deployed use cases, such as single and multiple screens, multi-point, Scalable Video Coding (SVC), as well as cases that are expected to be implemented using the protocol framework produced by this work item.  The methodology for describing the variables must allow extensibility of the variables, since telepresence is still a young technology and may have use cases that are not currently considered.

The work item will identify use cases, define requirements, and define a method for describing and transporting information about multiple media streams, including a specification of variables required to support the use cases. This work item will consider the reuse of existing IETF protocols and produce an architecture/protocol framework document describing the protocols required to be implemented to support this functionality.  The document will identify any enhancements required to existing protocols as well as describing new protocol(s) for interoperable multi-streams negotiation that may be required.

Scope
The scope includes both systems that provide a fully immersive experience, and systems that interwork with them and therefore need to understand the same multiple stream semantics.

The focus of this work is on audio and video multiple streams, however other media types may be considered.  Definition of new application protocols is not within the scope of this work

The group will produce

- Architectural Framework describing the protocols required to be implemented to support this functionality and identifying existing protocol enhancements and new protocol functionality required

- Specification of a new protocol to support telepresence multi-streams [if required]

Goals and Milestones
Nov 2010

March 2011

March 2011

Nov 2011

---

From: Peter Musgrave <[email-masked]>
Date: Wed, 19 May 2010 13:17:50 +0200

Hi Stephen,

I agree that backwards compatibility with existing TP systems should not be
required. It seems we agree that SIP/SDP to talk to legacy
video-conferencing is required, as is RTP for media transport.

If you are thinking in terms of XMPP - would this be based on Jingle? Or is
XMPP just for 'meta-data'?

I struggle to see how the answer for the transport stuff could not be SIP
based - since that allows the solution to build on the NAT traversal, TLS,
SRTP stuff etc. which comes with that (unless you're thinking along the
lines of ViPR type NAT-avoidance and p2p SIP). However, I don't see a need
to restrict the charter to that if there are strong beliefs that other
approaches need to be considered. However, the more the scope can be
constrained at the start - the sooner protocol work would get done.

I agree that there is a bunch of meta-data which may require dedicated
connections. In general I'm finding that adding e.g. TCP in SDP for this
fails since most SBCs do not relay TCP media. So some kind of conveyance for
high rate metadata (speaker changes, view changes etc., collaboration
material) needs to get solved. Maybe XMPP can fit here? (There is a charter
floating around for a WG to address UAs which do both SIP and XMPP - not
sure if this can be used here).

Peter Musgrave

[trimmed previous part of thread]

---

From: stephen botzko <[email-masked]>
Date: Wed, 19 May 2010 07:19:23 -0400

Allyn and I co-chair the IMTC multi-streaming group.  The IMTC would be
delighted if the IETF takes on this work item.

Regards
Stephen Botzko

On Wed, May 19, 2010 at 7:05 AM, <[email-masked]> wrote:

>  Hi Allyn,
>
> Cisco has recently open sourced under Apache license a protocol called
> Telepresence Interoperability Protocol. Also IMTC (International Multimedia
> Telecommunications Consortium) has setup an activity around TIP and
> Telepresence [1]. Would this proposed IETF work be overlapping or orthogonal
> to TIP and what IMTC is doing?
>
> It's natural that IETF RAI area is a good fit for this type of standards,
> especially if we can take RTP, SIP, XMPP as the baseline. On the other hand,
> if a reasonable solution that is backed up by some major vendors is already
> being developed elsewhere, that will significantly shrik the succes chances
> of IETF being able to do something that will get deployed.
>
> Regards,
>     Markus
>
> [1]
> http://blog.imtc.org/index.php/2010/03/09/announcing-the-creation-of-imtc-telepresence-multi-streaming-activity-group/
> *From:* [email-masked] [mailto:[email-masked]] *On
> Behalf Of *ext Allyn Romanow (allyn)
> *Sent:* 18 May, 2010 21:30
> *To:* IETF DISPATCH list
> *Subject:* [dispatch] Multi-streams for telepresence description of work
>
>  Hi Folks,
>
>
>

---

From: <[email-masked]>
Date: Wed, 19 May 2010 13:26:00 +0200

That should make things smooth :)

It would be good to explain the relationship and worksplit (between IETF and IMTC) explicitly in the charter.

Markus

________________________________
From: ext stephen botzko [mailto:[email-masked]]
Sent: 19 May, 2010 14:19
To: Isomaki Markus (Nokia-CIC/Espoo)
Cc: [email-masked]; [email-masked]
Subject: Re: [dispatch] Multi-streams for telepresence description of work

Allyn and I co-chair the IMTC multi-streaming group.  The IMTC would be delighted if the IETF takes on this work item.

Regards
Stephen Botzko

On Wed, May 19, 2010 at 7:05 AM, <[email-masked]<mailto:[email-masked]>> wrote:
Hi Allyn,

Regards,
    Markus

________________________________
From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of ext Allyn Romanow (allyn)
Sent: 18 May, 2010 21:30
To: IETF DISPATCH list
Subject: [dispatch] Multi-streams for telepresence description of work

Hi Folks,

Background

Problem

Charter

Scope

The group will produce

Goals and Milestones
Nov 2010

March 2011

March 2011

Nov 2011

_______________________________________________
dispatch mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dispatch

---

From: stephen botzko <[email-masked]>
Date: Wed, 19 May 2010 07:38:33 -0400

Personally I agree that the transport for media should be SIP based, since I
think it is problematic

One challenge for multi-way streaming is that in multi-way calls the
endpoints (and perhaps the intermediate infrastructure) are not receiving
all the streams.  It is desirable for these devices to be able to "see" the
stream semantics, especially if this or future work includes the ability to
request streams.

Also in a multi-way call you have the late joiner use case.  If you signal
the stream semantics over SDP, then each source needs to be able to re-send
its stream semantics (or there needs to be a lot of state in the
infrastructure).

This led me to wonder about the use of a presence protocol for signaling the
stream semantics. I have not fully analyzed its implications, and it may
turn out to be a bad idea.

Regards,
Stephen Botzko

---

From: stephen botzko <[email-masked]>
Date: Wed, 19 May 2010 07:55:46 -0400

Sorry about the first sentence, I accidentally hit send before I was done...

It should have been: Personally I agree that the solution should be based on
SIP.

Regards,
Stephen Botzko

On Wed, May 19, 2010 at 7:38 AM, stephen botzko <[email-masked]>wrote:

> Personally I agree that the transport for media should be SIP based, since
> I think it is problematic
>
> This led me to wonder about the use of a presence protocol for signaling
> the stream semantics. I have not fully analyzed its implications, and it may
>> I agree that backwards compatibility with existing TP systems should not
>> be required. It seems we agree that SIP/SDP to talk to legacy
>> If you are thinking in terms of XMPP - would this be based on Jingle? Or
>> is XMPP just for 'meta-data'?
>>

---

From: "Elwell, John" <[email-masked]>
Date: Wed, 19 May 2010 15:23:18 +0200

Allyn,

I think this work has at least to support the co-existence of legacy video endpoints and TP endpoints in the same conference.

I assume the work will cover point-to-point (two party) calls also, not just conferences.

I think it is premature to include a new protocol document in the charter at this stage. It could be that work will only require extensions to existing protocols, which would better be done in existing groups such as MMUSIC, AVT, XCON, XMPP. I think we need some progress with the architecture before making that determination.

John

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Allyn Romanow (allyn)
> Sent: 18 May 2010 19:30
> To: IETF DISPATCH list
> conferencing on steroids..:), fewer probably are aware of the 
> fact that the current systems are not interoperable with each 
> other. After talking with Mary and Cullen, we have a rough 
> draft of a charter to work on this problem in DISPATCH,  for 
> We don't yet know whether a working group would need to be 
> spun out or not - that will be clarified as we go.
> 
>  
> 
> description to start a discussion, and plan to send out use 
> cases for discussion shortly.
> 
>  
> 
> Who is the we? Several makers and providers of telepresence 
> systems have gotten together to define the main problem in 
> One branch of video conferencing has evolved that is focused 
> on immersive "being there" experience.  Referred to in 
> various ways such as virtual conferencing, telepresence or 
> media spaces, early systems were mainly research projects or 
> business systems with limited deployments.  In recent years 
> telepresence systems have seen considerable market success.  
> Following the model of early systems, the first wave of 
> commercial systems have been typically located in specially 
> designed single-purpose rooms with multiple relatively large 
> displays permitting life size image reproduction, multiple 
> cameras, encoders, decoders and microphones.  These systems 
> have several important characteristics that are different 
> from more traditional video conferencing systems.  
> 
>  
> 
> viewpoint in order to improve participant nonverbal 
> communication. These systems preserve essential group meeting 
> characteristics such as eye contact, group gestures, seating 
> order and spatial audio by carefully orchestrating the miking 
> and camera angles at each of the sites . This is distinct 
> from the more traditional approach where the geometric 
> relationship between media streams is not used to preserve 
> inter-stream communication aspects such as eye contact and 
> group dynamics.  
> 
>  
> 
> A second difference is manipulation of the environment to 
> improve immersion.  With telepresence systems, 
> cinematographic aspects of the local environment reproduction 
> are carefully planned including color, table shape, seating 
> and lighting so that when combined with large high quality 
> displays, a strong sense of a "trompe l'oeil" or "being 
> there" immersive experience is created.  Typical video 
> conference systems do not include these considerations.
> 
>  
> 
> As telepresence video systems have become successful in the 
> market, manufacturers have started exploring delivery of the 
> nonverbal communication and immersive values of telepresence 
> via smaller, less expensive and more flexible video 
> conferencing systems for a variety of venues, such as 
> individual offices, homes and kiosks. These are also  
> telepresence systems, since the audio and video quality is 
> communication, they are able to send and receive multiple 
> media streams, and large coordinated multi image displays are 
> available for immersive installations.   As the industry 
> develops, the line between telepresence and video 
> conferencing may become blurred as nonverbal communication 
> and immersive installations become broadly available.
> 
>  
> 
> Problem
> 
> Although telepresence systems are based on open standards 
> such as RTP, SIP, H.264, H.323 suite, they cannot easily 
> interoperate with each other without operator assistance and 
> expensive additional equipment that translates from one 
> vendor to another. It would be like having to make sure all 
> parties are on the same equipment (and network) when making a 
> telephone call.  A major factor in the inability of 
> Telepresence systems to work with each other is that there is 
> no standard description of the multiple streams that comprise 
> the media flows. 
> 
>  
> 
> For example, in a multiple screen conference, the video and 
> audio streams sent from remote participants must be 
> understood by receivers so that they can be presented in a 
> coherent and life-like manner. This includes the ability to 
> present the remote participants at their true size for their 
> apparent distance, while maintaining correct eye contact, 
> gesticular cues, and simultaneously providing a spatial audio 
> sound stage consistent with the video presentation.  The 
> receiving device that decides how to display the incoming 
> information needs to understand a number of variables such as 
> the spatial position of the speaker, the field of view of the 
> cameras, the camera zoom, which media stream is related to 
> each of the displays, etc. 
> 
>  
> 
> Charter
> 
> chartered to define and specify the content of media 
> multi-stream messages and the way these will be transported. 
> 
> This work is aimed at providing a standard for the exchange 
> of media semantics information that will foster interoperable 
> end stations and conference bridges. This work item will 
> specify the variables that describe the semantics of the 
> media streams and the recommended behavior to achieve 
> interoperability.  
> 
>  
> 
> This requires considering current widely deployed use cases, 
> such as single and multiple screens, multi-point, Scalable 
> Video Coding (SVC), as well as cases that are expected to be 
> implemented using the protocol framework produced by this 
> work item.  The methodology for describing the variables must 
> allow extensibility of the variables, since telepresence is 
> still a young technology and may have use cases that are not 
> currently considered.
> 
>  
> 
> The work item will identify use cases, define requirements, 
> and define a method for describing and transporting 
> information about multiple media streams, including a 
> specification of variables required to support the use cases. 
> This work item will consider the reuse of existing IETF 
> protocols and produce an architecture/protocol framework 
> document describing the protocols required to be implemented 
> to support this functionality.  The document will identify 
> any enhancements required to existing protocols as well as 
> describing new protocol(s) for interoperable multi-streams 
> negotiation that may be required.
> 
>  
> 
> Scope
> 
> immersive experience, and systems that interwork with them 
> and therefore need to understand the same multiple stream semantics.
> 
>  
> 
> streams, however other media types may be considered.  
> Definition of new application protocols is not within the 
> scope of this work
> 
>  
> 
> - Architectural Framework describing the protocols required 
> to be implemented to support this functionality and 
> identifying existing protocol enhancements and new protocol 
> functionality required
> 
>  
>

---

From: Marshall Eubanks <[email-masked]>
Date: Wed, 19 May 2010 09:45:09 -0400

On May 19, 2010, at 9:23 AM, Elwell, John wrote:

> Allyn,
>
> I think this work has at least to support the co-existence of legacy  
> video endpoints and TP endpoints in the same conference.

Do you mean that it should support interop between purely H.323 units  
and SIP based units ?

Regards
Marshall

> I think it is premature to include a new protocol document in the  
> charter at this stage. It could be that work will only require  
> extensions to existing protocols, which would better be done in  
> existing groups such as MMUSIC, AVT, XCON, XMPP. I think we need  
> some progress with the architecture before making that determination.
>
> John
>
>

---

From: "Elwell, John" <[email-masked]>
Date: Wed, 19 May 2010 16:04:27 +0200

No, I meant co-existence of TP-aware and basic SIP video endpoints in the same conference. If that is supported, then presumably existing SIP-H.323 gateways would be able to accommodate legacy H.323 endpoints too, but that was not my primary concern.

John 

> From: Marshall Eubanks [mailto:[email-masked]] 
> Sent: 19 May 2010 14:45
> To: Elwell, John
> Cc: Allyn Romanow (allyn); IETF DISPATCH list
> > I assume the work will cover point-to-point (two party) 
> calls also,  
> > not just conferences.
> >

---

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Wed, 19 May 2010 08:48:33 -0700

Hi Markus,

Thanks for bringing up these telepresence related activities, whose
roles need to be clarified. The bottom line is that Cisco's TIP is not
related to the proposed multi-stream work, and the IMTC Telepresence
Activity Group is related and has had the goal of bringing proposed work
definition, use cases and requirements, but NOT solution, to the IETF.

Cisco has indeed made its TIP protocol freely available. In fact  the
copyright is in the process of being given over to the IMTC, who will
own it, manage it, and have change control. The IMTC has not yet
announced how it will manage TIP. TIP is Cisco's way of doing
telepresence signaling and media control.

At the same time, Cisco is also engaging in the development of a new
open standard which we believe would most appropriately be developed
within the IETF as the technology is based IETF protocols, RTP and SIP
in particular. The IMTC is a multimedia industry consortium that does a
lot of interoperability testing. Many vendors are members and individual
non-members can participate in activities. This was a good place to
begin to pull together thoughts on how existing standards can help
interoperability and what is missing from them. Thus the Telepresence
Activity Group was formed. Steve Botzko and myself  are the co-chairs.
The work of the TP AG has been to define the problem that could be
worked on in the IETF. We have been consulting with Cullen and Mary from
the beginning, and have been quite clear not to address potential
solutions prior to working in the IETF.

Again, on the relationship with TIP, since TIP is Cisco's solution for
media control, people from Cisco will undoubtedly suggest that what they
feel are the most valuable parts of the TIP technology are not discluded
from a standard, but no more so than what people from other existing
solutions are likely to do. People bring to the table their experience
and expertise. 

I hope this clarifies.. it's potentially quite confusing. Please ask any
further questions that come up.

Best regards,

Allyn

From: [email-masked] [mailto:[email-masked]] 
Sent: Wednesday, May 19, 2010 4:05 AM
To: Allyn Romanow (allyn); [email-masked]
Subject: RE: Multi-streams for telepresence description of work

Hi Allyn,

Cisco has recently open sourced under Apache license a protocol called
Telepresence Interoperability Protocol. Also IMTC (International
Multimedia Telecommunications Consortium) has setup an activity around
TIP and Telepresence [1]. Would this proposed IETF work be overlapping
or orthogonal to TIP and what IMTC is doing?

It's natural that IETF RAI area is a good fit for this type of
standards, especially if we can take RTP, SIP, XMPP as the baseline. On
the other hand, if a reasonable solution that is backed up by some major
vendors is already being developed elsewhere, that will significantly
shrik the succes chances of IETF being able to do something that will
get deployed. 

Regards,

    Markus

[1]
http://blog.imtc.org/index.php/2010/03/09/announcing-the-creation-of-imt
c-telepresence-multi-streaming-activity-group/

	From: [email-masked]
[mailto:[email-masked]] On Behalf Of ext Allyn Romanow
(allyn)
	Sent: 18 May, 2010 21:30
	To: IETF DISPATCH list
	Subject: [dispatch] Multi-streams for telepresence description
of work

	Hi Folks,

	We don't yet know whether a working group would need to be spun
out or not - that will be clarified as we go.

	We think there is enough background info in the work description
to start a discussion, and plan to send out use cases for discussion
shortly.

	Allyn

	 

	Background

	The first difference concerns controlling the visual viewpoint
in order to improve participant nonverbal communication. These systems
preserve essential group meeting characteristics such as eye contact,
group gestures, seating order and spatial audio by carefully
orchestrating the miking and camera angles at each of the sites . This
is distinct from the more traditional approach where the geometric
relationship between media streams is not used to preserve inter-stream
communication aspects such as eye contact and group dynamics.  

	A second difference is manipulation of the environment to
improve immersion.  With telepresence systems, cinematographic aspects
of the local environment reproduction are carefully planned including
color, table shape, seating and lighting so that when combined with
large high quality displays, a strong sense of a "trompe l'oeil" or
"being there" immersive experience is created.  Typical video conference
systems do not include these considerations.

market, manufacturers have started exploring delivery of the nonverbal

	Problem

	Although telepresence systems are based on open standards such
as RTP, SIP, H.264, H.323 suite, they cannot easily interoperate with
each other without operator assistance and expensive additional
equipment that translates from one vendor to another. It would be like
having to make sure all parties are on the same equipment (and network)
when making a telephone call.  A major factor in the inability of
Telepresence systems to work with each other is that there is no
standard description of the multiple streams that comprise the media
flows. 

audio streams sent from remote participants must be understood by

	Charter

	This work is aimed at providing a standard for the exchange of
media semantics information that will foster interoperable end stations
and conference bridges. This work item will specify the variables that
describe the semantics of the media streams and the recommended behavior
to achieve interoperability.  

	This requires considering current widely deployed use cases,
such as single and multiple screens, multi-point, Scalable Video Coding
(SVC), as well as cases that are expected to be implemented using the
protocol framework produced by this work item.  The methodology for
describing the variables must allow extensibility of the variables,
since telepresence is still a young technology and may have use cases
that are not currently considered.

	The work item will identify use cases, define requirements, and
define a method for describing and transporting information about
multiple media streams, including a specification of variables required
to support the use cases. This work item will consider the reuse of
existing IETF protocols and produce an architecture/protocol framework
document describing the protocols required to be implemented to support
this functionality.  The document will identify any enhancements
required to existing protocols as well as describing new protocol(s) for
interoperable multi-streams negotiation that may be required.

	Scope

	The focus of this work is on audio and video multiple streams,
however other media types may be considered.  Definition of new
application protocols is not within the scope of this work

	[Is there anything else we need to explicitly say is out of
scope?]

	The group will produce

	- Architectural Framework describing the protocols required to
be implemented to support this functionality and identifying existing
protocol enhancements and new protocol functionality required

	- Specification of a new protocol to support telepresence
multi-streams [if required]

	Goals and Milestones

Nov 2010 

March 2011 

March 2011

Nov 2011

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Wed, 19 May 2010 15:14:28 -0700

Just an FYI, for the more rudimentary level of interoperability along
the lines of what is possible with traditional H.323 based video
conferencing equipment, folks may want to direct their input and
requirements to the SIP Parity group within IMTC
(http://www.imtc.org/activity_groups/SIP.asp). This group is not trying
to create any new standards or protocols, but rather to describe best
practices that promote interoperability among the various vendors.
I view this potential DISPATCH working group as taking things to the
next level.
 
Cheers,
Charles

> From: [email-masked] [mailto:[email-masked]] On
Behalf Of stephen botzko
> Sent: Wednesday, May 19, 2010 3:48 AM
> On Wed, May 19, 2010 at 12:50 AM, Allyn Romanow (allyn)
<[email-masked]> wrote:
> 
> 
> 	Hi Peter,
> 	You've raised excellent points. See inline for a few brief
comments, and I'll look to others to
> elaborate (or disagree).
description of work
> 
> 
> 		Hi Allyn,
> 
> 		I am interested in having this work start. I suspect it
will likely warrant a working
> group at the appropriate time.
> 
> 		The charter as written is VERY general. I would suspect
that SIP/SDP is going to be the
> common ground. Is it worth making this explicit in the charter? If not
perhaps the reasons why not
> could be stated in the charter. (My understanding is that most of
these systems do use SIP but that
> they differ in number of sessions per endpoint and how the media is
packaged into RTP streams etc.)
> 
> 		[alr] Certainly RTP is used without question, and there
has been discussion about limiting
> to RTP streams only, not other types of data. This is still an open
issue to be discussed.
> 
> 		Although I (and some others) initially assumed that we
would use SDP for signaling, after
> further thought Steve Botsko and  others suggested we might want to
consider another protocol such as
> XMPP in light of the nature of the messages we want to send. I think
that it's premature to specify
> this in the charter as I consider it part of the the solution, and
think it will become clear what
> protocol to use as we specify requirements.
> 
> 		Is there other IETF work which we can see has relevance
and should be mentioned as tools
> which can be brought to bear? (xcon? mediactrl?)
> 
> 		[alr] Good point. We definitely want to consider
existing work, and look to the experts in
> DISPATCH to help with this. As above, I'm not sure if we can go there
yet without requirements first-
> but centainly we can be begin to look at potential tools.
> 
> 		Is there interest in ensuring interop of TP with legacy
SIP video conferencing systems as
> part of this work? Should that be part of the charter?
> 
> 		[alr] My personal view is not too much, that it's more a
matter of going forward with a
> common technology that allows us to interoperate. I don't want us to
be in the position of needing to
> tie together systems that we already have, as that would too much
limit our approach. That said,
> people will want to incorporate the best of their current technology.
What do folks think?
> 
> [scb] Personally I think it is essential that TP devices be able to
use SIP/SDP to interoperate with
> traditional videoconferencing systems.  So I think it is a requirement
that this work not preclude
> SIP/SDP interoperability, either with videoconferencing or ordinary
phones.
> 
> Also, I think that TP calls will need to be be carried on existing
[possibly upgraded] infrastructure,
> including SIP proxies, SBCs, etc, as it is undesirable to deploy a
parallel infrastructure for TP.
> I do not think it is a requirement for this work to fully accommodate
existing TP systems. While it
> would be a nice outcome if these systems could be easily retrofitted
to support this new work, it is
> much more important to have a solid basis to go forward.
> 
> 
> 
> 		Regards,
> 
> 		Peter Musgrave
> 
> 
> 
<[email-masked]> wrote:
> 
> 
> 			Hi Folks,
> 
> 
> 
> 			Many of you are aware of telepresence systems
(video conferencing on steroids..:),
> fewer probably are aware of the fact that the current systems are not
interoperable with each other.
> After talking with Mary and Cullen, we have a rough draft of a charter
to work on this problem in
> 			We don't yet know whether a working group would
need to be spun out or not - that
> will be clarified as we go.
> 
> 
> 
telepresence systems have gotten
> together to define the main problem in interoperability - the handling
of multi-streams.
> 			One branch of video conferencing has evolved
that is focused on immersive "being
> there" experience.  Referred to in various ways such as virtual
conferencing, telepresence or media
> spaces, early systems were mainly research projects or business
systems with limited deployments.  In
> recent years telepresence systems have seen considerable market
success.  Following the model of early
> systems, the first wave of commercial systems have been typically
located in specially designed
> single-purpose rooms with multiple relatively large displays
permitting life size image reproduction,
> multiple cameras, encoders, decoders and microphones.  These systems
have several important
> characteristics that are different from more traditional video
conferencing systems.
> 
> 
> 
> 			The first difference concerns controlling the
visual viewpoint in order to improve
> participant nonverbal communication. These systems preserve essential
group meeting characteristics
> such as eye contact, group gestures, seating order and spatial audio
by carefully orchestrating the
> miking and camera angles at each of the sites . This is distinct from
the more traditional approach
> where the geometric relationship between media streams is not used to
preserve inter-stream
> 			A second difference is manipulation of the
environment to improve immersion.  With
> telepresence systems, cinematographic aspects of the local environment
reproduction are carefully
> planned including color, table shape, seating and lighting so that
when combined with large high
> quality displays, a strong sense of a "trompe l'oeil" or "being there"
immersive experience is
considerations.
> 
> 
> 
> 			As telepresence video systems have become
successful in the market, manufacturers
> have started exploring delivery of the nonverbal communication and
immersive values of telepresence
> via smaller, less expensive and more flexible video conferencing
systems for a variety of venues, such
> as individual offices, homes and kiosks. These are also  telepresence
systems, since the audio and
> video quality is high enough to allow clear image reproduction for
nonverbal communication, they are
> able to send and receive multiple media streams, and large coordinated
multi image displays are
> available for immersive installations.   As the industry develops, the
line between telepresence and
> video conferencing may become blurred as nonverbal communication and
immersive installations become
> broadly available.
standards such as RTP, SIP, H.264,
> H.323 suite, they cannot easily interoperate with each other without
operator assistance and expensive
> additional equipment that translates from one vendor to another. It
would be like having to make sure
> all parties are on the same equipment (and network) when making a
telephone call.  A major factor in
> the inability of Telepresence systems to work with each other is that
there is no standard description
> of the multiple streams that comprise the media flows.
> 
> 
> 
> 			For example, in a multiple screen conference,
the video and audio streams sent from
> remote participants must be understood by receivers so that they can
be presented in a coherent and
> life-like manner. This includes the ability to present the remote
participants at their true size for
> their apparent distance, while maintaining correct eye contact,
gesticular cues, and simultaneously
> providing a spatial audio sound stage consistent with the video
presentation.  The receiving device
> that decides how to display the incoming information needs to
understand a number of variables such as
> the spatial position of the speaker, the field of view of the cameras,
the camera zoom, which media
> 			The Telepresence Multi-Streams work item in
DISPATCH is chartered to define and
> specify the content of media multi-stream messages and the way these
will be transported.
> 
> 			This work is aimed at providing a standard for
the exchange of media semantics
> information that will foster interoperable end stations and conference
bridges. This work item will
> specify the variables that describe the semantics of the media streams
and the recommended behavior to
> 			This requires considering current widely
deployed use cases, such as single and
> multiple screens, multi-point, Scalable Video Coding (SVC), as well as
cases that are expected to be
> implemented using the protocol framework produced by this work item.
The methodology for describing
> the variables must allow extensibility of the variables, since
telepresence is still a young
> technology and may have use cases that are not currently considered.
> 
> 
> 
requirements, and define a method for
> describing and transporting information about multiple media streams,
including a specification of
> variables required to support the use cases. This work item will
consider the reuse of existing IETF
> protocols and produce an architecture/protocol framework document
describing the protocols required to
> be implemented to support this functionality.  The document will
identify any enhancements required to
> existing protocols as well as describing new protocol(s) for
interoperable multi-streams negotiation
> that may be required.
> 
> 
> 
> 			Scope
> 
> 			The scope includes both systems that provide a
fully immersive experience, and
> systems that interwork with them and therefore need to understand the
> 
> 
> 
> 			The focus of this work is on audio and video
multiple streams, however other media
> types may be considered.  Definition of new application protocols is
not within the scope of this work
> 			- Architectural Framework describing the
protocols required to be implemented to
> support this functionality and identifying existing protocol
enhancements and new protocol
> 			- Specification of a new protocol to support
telepresence multi-streams [if