Subject: [avtext] framemarking: add frame size info

From: Miguel París Díaz <[email-masked]>
Date: Thu, 25 Aug 2016 16:12:48 +0200

Hello,
it would be great having frame size (width and height) info in the Frame
Marking RTP header extension [1].

Why?
For example, in the case of using simulcast in an SFU, selecting the stream
by the size would ease the application development and improve the
experience of the users.
Application developers don't usually have deep knowledge about media like
bitrate, etc., but they know which video size has to be rendered in the
GUI, which may depend on the client where the app is running: a mobile, a
PC with a 13"· screen, a PC with 27" screen, etc.

In this way and taking a videoconference app as example, if a participant
select another participant to be rendered as main video, the app could ask
the SFU to select the video quality that better matches to 800x600 size.

What do you think about this idea?

Thanks and best regards!!

Refs
[1] https://tools.ietf.org/html/draft-ietf-avtext-framemarking-02

-- 
Miguel París Díaz
------------------------------------------------------------------------
Computer/Software engineer.
Researcher and architect in http://www.kurento.org
http://twitter.com/mparisdiaz
------------------------------------------------------------------------

---

From: "Paul E. Jones" <[email-masked]>
Date: Fri, 26 Aug 2016 16:52:18 +0000

Miguel,

You make the assumption that the media distributor will not see the SDP, 
I suppose.  While certainly a valid model, I'll admit that I had 
personally assumed any media forwarding function would see the SDP (or 
at least be told the PT values and any relevant flow information similar 
to what RFC 6236 provides) and would thus know which PT values 
correspond to what video resolutions if simulcast is employed.

Paul

------ Original Message ------
From: "Miguel París Díaz" <[email-masked]>
To: [email-masked]
Sent: 8/25/2016 10:12:48 AM
Subject: [avtext] framemarking: add frame size info

>Hello,
>it would be great having frame size (width and height) info in the 
>Frame Marking RTP header extension [1].
>
>Why?
>For example, in the case of using simulcast in an SFU, selecting the 
>stream by the size would ease the application development and improve 
>the experience of the users.
>Application developers don't usually have deep knowledge about media 
>like bitrate, etc., but they know which video size has to be rendered 
>in the GUI, which may depend on the client where the app is running: a 
>mobile, a PC with a 13"· screen, a PC with 27" screen, etc.
>
>In this way and taking a videoconference app as example, if a 
>participant select another participant to be rendered as main video, 
>the app could ask the SFU to select the video quality that better 
>matches to 800x600 size.
>

---

From: Jonathan Lennox <[email-masked]>
Date: Fri, 26 Aug 2016 17:07:53 +0000

(As an individual.)

In the latest version of simulcast the media distributor would need the RID values, not the PT values, but the idea is the same — it needs the SDP.

Note that if the media distributor doesn’t have information from the SDP it can’t reliably identify the frame marking header extension at all, since header extension IDs are negotiated. So I’m not sure how much benefit there is to putting the size in the header extension.

That said, if we envision a scenario where encoders might be frequently changing their video size (in response to available network bandwidth, or the like), it might be useful for encoders to be able to indicate the current size they’re encoding without needing to send updated SDP all the time.

On Aug 26, 2016, at 12:52 PM, Paul E. Jones <[email-masked]<mailto:[email-masked]>> wrote:

Miguel,

You make the assumption that the media distributor will not see the SDP, I suppose.  While certainly a valid model, I'll admit that I had personally assumed any media forwarding function would see the SDP (or at least be told the PT values and any relevant flow information similar to what RFC 6236 provides) and would thus know which PT values correspond to what video resolutions if simulcast is employed.

Paul

------ Original Message ------
From: "Miguel París Díaz" <[email-masked]<mailto:[email-masked]>>
To: [email-masked]<mailto:[email-masked]>
Sent: 8/25/2016 10:12:48 AM
Subject: [avtext] framemarking: add frame size info

Hello,
it would be great having frame size (width and height) info in the Frame Marking RTP header extension [1].

Why?
For example, in the case of using simulcast in an SFU, selecting the stream by the size would ease the application development and improve the experience of the users.
Application developers don't usually have deep knowledge about media like bitrate, etc., but they know which video size has to be rendered in the GUI, which may depend on the client where the app is running: a mobile, a PC with a 13"· screen, a PC with 27" screen, etc.

In this way and taking a videoconference app as example, if a participant select another participant to be rendered as main video, the app could ask the SFU to select the video quality that better matches to 800x600 size.

Refs

--
Miguel París Díaz
------------------------------------------------------------------------
Computer/Software engineer.
Researcher and architect in http://www.kurento.org<http://www.kurento.org/>
http://twitter.com/mparisdiaz
------------------------------------------------------------------------
_______________________________________________
avtext mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/avtext

---

From: Miguel París Díaz <[email-masked]>
Date: Tue, 30 Aug 2016 12:37:16 +0200

I assume that the media distributor has the information from the SDP (it
performs the SDP negotiation which each "client"), but the point is that
encoders may change the video size depending on the available bandwidth,
the complexivity of the video source, etc., unless the sender forces the
encoders' configuration with a fix frame size...

2016-08-26 19:07 GMT+02:00 Jonathan Lennox <[email-masked]>:

> (As an individual.)
>
> In the latest version of simulcast the media distributor would need the
> RID values, not the PT values, but the idea is the same — it needs the SDP.
>
> Note that if the media distributor doesn’t have information from the SDP
> it can’t reliably identify the frame marking header extension at all, since
> header extension IDs are negotiated. So I’m not sure how much benefit there
> is to putting the size in the header extension.
>
> That said, if we envision a scenario where encoders might be frequently
> changing their video size (in response to available network bandwidth, or
> the like), it might be useful for encoders to be able to indicate the
> current size they’re encoding without needing to send updated SDP all the
> time.
>
> On Aug 26, 2016, at 12:52 PM, Paul E. Jones <[email-masked]> wrote:
>
> Miguel,
>
> You make the assumption that the media distributor will not see the SDP, I
> suppose.  While certainly a valid model, I'll admit that I had personally
> assumed any media forwarding function would see the SDP (or at least be
> told the PT values and any relevant flow information similar to what RFC
> 6236 provides) and would thus know which PT values correspond to what video
> resolutions if simulcast is employed.
>
> Paul
>
> stream by the size would ease the application development and improve the
> experience of the users.
> [email-masked]