Subject: Re: [httpstreaming] QoE feedback support

From: "Ali C. Begen (abegen)" <[email-masked]>
Date: Wed, 10 Nov 2010 04:52:30 -0800

I really find it very unattractive to call this stuff QoE reporting/feedback. Give me one example on what metric would measure my level of pleasure from a streaming service ;)

-acbegen

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Ingemar Johansson S
> Sent: Wednesday, November 10, 2010 8:42 PM
> To: httpstreaming
> Subject: [httpstreaming] QoE feedback support
> 
> FYI
> 
> This document describes how QoE support is proposed for HTTP streaming in 3GPP.
> http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/Ad-hoc_MBS/Docs_AHI/S4-AHI118.zip
> I suspect that there exist a newer version and/or a better description.
> 
> Regards
> Ingemar
> =================================
> INGEMAR JOHANSSON  M.Sc.
> Senior Research Engineer
> 
> Ericsson AB
> Multimedia Technologies
> Labratoriegränd 11
> 971 28, Luleå, Sweden
> Phone +46-1071 43042
> SMS/MMS +46-73 078 3289
> [email-masked]
> www.ericsson.com
> Visit http://labs.ericsson.com !
> =================================
> _______________________________________________
> httpstreaming mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/httpstreaming

---

From: David Singer <[email-masked]>
Date: Wed, 10 Nov 2010 14:09:09 +0100

>> I suspect that there exist a newer version and/or a better description.

we were actually just discussing this area at a 3G meeting, and I realized that there are really two kinds of metrics it is useful to get from clients:

a) truly Qo Experience, that is, experiential metrics -- the mismatch between the user's expectation, and reality (tune-in delay, media they wanted to see but did not, and so on)
b) operational metrics -- amount of data fetched, buffer sizes maintained, and so on

There are some in the middle, like bandwidth experienced, which are both operational and affect experience.

David Singer
Multimedia and Software Standards, Apple Inc.

---

From: "Ali C. Begen (abegen)" <[email-masked]>
Date: Wed, 10 Nov 2010 22:57:27 +0800

Sure but this does not answer my question afaict. 

-acbegen

---

From: Gunnar Heikkilä <[email-masked]>
Date: Wed, 10 Nov 2010 16:26:07 +0100

The acronym QoE is maybe not the best choice of name, but the concept allows the 3GPP operator to get a better understanding on how the streaming service works for the end client. Some more integrated services (like circuit-switched AMR voice) can typically be handled by measuring things within the 3GPP network itself (for instance codec mode adaptation, frame loss etc.), but other services (like HTTP streaming) have a lot more freedom for the end client to decide how to play out the media.

This is a good thing, as clients can choose different strategies to satisfy the end user, but it also means that it is difficult for the operator to understand if his network works good enough for these type of services. The metrics reported back from the client are typically things which are not possible to measure in the network, but which does affect the media "enjoyment" a lot. For instance, the length of the initial buffering, or any re-buffering occurances etc.

Not rocket science, not perfect, but still information which help the operator to see a more complete picture of his network and the services used...

/Gunnar

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Ali C. Begen (abegen)
Sent: on 10 november 2010 15:57
To: David Singer
Cc: Ingemar Johansson S; httpstreaming
Subject: Re: [httpstreaming] QoE feedback support

-acbegen

On Nov 10, 2010, at 9:09 PM, "David Singer" <[email-masked]<mailto:[email-masked]>> wrote:

On Nov 10, 2010, at 13:52 , Ali C. Begen (abegen) wrote:

>> From: [email-masked]<mailto:[email-masked]> [<mailto:[email-masked]>mailto:[email-masked]] On Behalf Of Ingemar Johansson S
>> <http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/Ad-hoc_MBS/Docs_AHI/S4-AHI118.zip> http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/Ad-hoc_MBS/Docs_AHI/S4-AHI118.zip

David Singer

---

From: "Ali C. Begen (abegen)" <[email-masked]>
Date: Wed, 10 Nov 2010 07:32:41 -0800

> From: Gunnar Heikkilä [mailto:[email-masked]]
> Sent: Wednesday, November 10, 2010 11:26 PM
> To: Ali C. Begen (abegen); David Singer
> Cc: Ingemar Johansson S; httpstreaming; Gunnar Heikkilä
> Subject: RE: [httpstreaming] QoE feedback support
> 
> The acronym QoE is maybe not the best choice of name, but the concept allows the 3GPP operator to get a better

It is not the best choice and actually it is pretty misleading.

> understanding on how the streaming service works for the end client. Some more integrated services (like circuit-switched
> AMR voice) can typically be handled by measuring things within the 3GPP network itself (for instance codec mode
> adaptation, frame loss etc.), but other services (like HTTP streaming) have a lot more freedom for the end client to decide
> how to play out the media.

Indeed. That's why we should not call it QoE.
 
> This is a good thing, as clients can choose different strategies to satisfy the end user, but it also means that it is difficult for
> the operator to understand if his network works good enough for these type of services. The metrics reported back from the
> client are typically things which are not possible to measure in the network, but which does affect the media "enjoyment" a
> lot. For instance, the length of the initial buffering, or any re-buffering occurances etc.
> 

I am all for reporting useful stuff from the clients in an extensible and modular way.

-acbegen
 
> 	> I really find it very unattractive to call this stuff QoE reporting/feedback. Give me one example on what metric
> would measure my level of pleasure from a streaming service ;)
> 	>
> 	> -acbegen
> 	>
> 	>> From: [email-masked] [ <mailto:[email-masked]> mailto:httpstreaming-
> [email-masked]] On Behalf Of Ingemar Johansson S
> 	>> <http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/Ad-hoc_MBS/Docs_AHI/S4-AHI118.zip>
> 	a) truly Qo Experience, that is, experiential metrics -- the mismatch between the user's expectation, and reality (tune-
> in delay, media they wanted to see but did not, and so on)

---

From: Mark Watson <[email-masked]>
Date: Wed, 10 Nov 2010 16:56:31 -0800

Ali,

There was push-back on the QoE name in MPEG too an the work item is now called 'Quality Metrics'.

As Dave say there are (at least) two kinds if measurement (i) those that relate only to what the user actually experienced (start-up time, playback stalls which version of the audio/video was actually played, when switches occured) and (ii) metrics to do with the how that experience was actually delivered: bandwidth measurements, buffer sizes, server response times etc.

...Mark 

Sent from my iPhone