Subject: [core] Fw: New Version Notification for draft-bhattacharyya-core-a-realist-00.txt

From: Abhijan Bhattacharyya <[email-masked]>
Date: Sun, 7 Oct 2018 00:53:24 +0530

Dear list,
A new draft has been submitted which essentially tries to facilitate real-time live FPV feeds with high QoE over the Internet under constrained environments. The technology leverages on some of the advantages of CoAP and proposes few extensions so that CoAP can have an efficient way of real-time streaming for the 'Things' version of Internet (I mean IoT), just like HTTP has HTTP-streaming in conventional Internet. So, we can have a protocol which is equally efficient in sending small sensor data as well as streams which may contain video information. This draft is an outcome of the experiments and implementations that we carried out on the face of some real-life challenges faced in robotic and UAV based AR applications.

Comments are welcome.

Dear chairs,
If you may please allow us to present, then we can try to explain the concept in physical meetings and also show some visuals from the demo that we had and the benchmark results. Looking forward.

Please note: an extended version of this draft has been accepted to be presented in Wi-UAV workshop of Globecom in coming December.

With Best Regards
Abhijan Bhattacharyya
Consultant / Scientist,
{Internet Protocols | 5G | Standardization}, 
TCS Research,
Tata Consultancy Services
Building 1B,Ecospace
Plot -  IIF/12 ,New Town, Rajarhat,
Kolkata - 700160,West Bengal
India
Ph:- +91 33 66884691
Cell:- +919830468972 | +918583875003
Mailto: [email-masked]
Website: http://www.tcs.com
____________________________________________
Experience certainty.	IT Services
Business Solutions
Consulting
____________________________________________

-----Forwarded by Abhijan Bhattacharyya/KOL/TCS on 10/07/2018 12:30AM -----
To: "Abhijan Bhattacharyya" <[email-masked]>, "Balamurali Purushothaman" <[email-masked]>, "Suvrat Agrawal" <[email-masked]>, "Hemant Rath" <[email-masked]>, "Arpan Pal" <[email-masked]>
From: [email-masked]
Date: 10/07/2018 12:29AM
Subject: New Version Notification for draft-bhattacharyya-core-a-realist-00.txt

"External email. Open with Caution"

A new version of I-D, draft-bhattacharyya-core-a-realist-00.txt
has been successfully submitted by Abhijan Bhattacharyya and posted to the
IETF repository.

Name:	draft-bhattacharyya-core-a-realist
Revision:	00
Title:	Adaptive RESTful Real-time Live Streaming for Things (A-REaLiST)
Document date:	2018-10-07
Group:	Individual Submission
Pages:	17
URL:            https://www.ietf.org/internet-drafts/draft-bhattacharyya-core-a-realist-00.txt
Status:         https://datatracker.ietf.org/doc/draft-bhattacharyya-core-a-realist/
Htmlized:       https://tools.ietf.org/html/draft-bhattacharyya-core-a-realist-00
Htmlized:       https://datatracker.ietf.org/doc/html/draft-bhattacharyya-core-a-realist

Abstract:
   This draft presents extensions to Constrained Application Protocol
   (CoAP) to enable RESTful Real-time Live Streaming for improving the
   Quality of Experience (QoE) for delay-sensitive Internet of Things
   (IoT) applications. The overall architecture is termed ''Adaptive
   RESTful Real-time Live Streaming for Things (A-REaLiST)''. It is
   particularly designed for applications which rely on real-time
   augmented vision through live First Person View (FPV) feed from
   constrained remote agents like Unmanned Aerial Vehicle (UAV), etc.
   These extensions provide the necessary hooks to help solution
   designers ensure low-latency transfer of streams and, for contents
   like video, a quick recovery from freeze and corruption without
   incurring undue lag. A-REaLiST is an attempt to provide an
   integrated approach to maintain the balance amongst QoE, resource-
   efficiency and loss resilience. It provides the necessary hooks to
   optimize system performance by leveraging contextual intelligence
   inferred from instantaneous information segments in flight. These
   extensions equip CoAP with a standard for efficient RESTful
   streaming for Internet of Things (IoT) contrary to HTTP-streaming in
   conventional Internet.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain 
confidential or privileged information. If you are 
not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 
please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you

---

From: Christian Amsüss <[email-masked]>
Date: Tue, 9 Oct 2018 09:25:35 +0200

Hello Abhijan,

On Sun, Oct 07, 2018 at 12:53:24AM +0530, Abhijan Bhattacharyya wrote:
> A new draft [...]  of real-time streaming

I've had a brief look at this draft, and have some questions and
comments:

* I assume that an I-Frame is still larger than an MTU, thus transported
  in the stream it would still occupy multiple exchanges. Given that you
  stick with NSTART=1 ("MUST wait to send any further segment"), won't
  the round-trip times for the (how-many?) CON exchanges bring
  throughput to a halt at every I-Frame?

* The stream states look like something that should not be managed in an
  option but can be managed at REST level by using established
  mechanisms: "Create an entity (stream) and give a handle that we can
  use to talk about it" is commonly expressed by POSTing somewhere
  and receiving a URI for that stream as a Location.

  I'll try to re-phrase the exchange on p9 in such terms:

   Client (Producer)                                  Server (Consumer)
   |                                                  |
   | POST: CON;                                       |
   |       URI=/video;                                |
   |       Payload= CBOR or JSON                      |\
   +------------------------------------------------->| |
   |                                                  | |Stream
   | ACK;                                             | |negotiation
   | Response = 2.04 CHANGED                          | |
   | Location: /five-bit-id                           | |
   |<-------------------------------------------------|/
   :                                                  :
   :                                                  :
   |(First segment of an MJPEG frame. Contains        |
   | meta-data. Critical segment needs reliable       |
   | delivery.)                                       |
   |                                                  |
   | PUT: CON;                                        |
   |       URI=/five-bit-id?t=123&pos=0               |
   |       Payload= <Bytes_in_1st segment>            |\
   +------------------------------------------------->| |
   |                                                  | |
   | ACK;                                             | |
   | Response = 2.04 CHANGED                          | |
   |<-------------------------------------------------| |
   |(Second segment of an MJPEG frame. Contains       | |
   | non-meta-data. Non-critical segment- best effort | |
   | transfer.)                                       | |
   |                                                  | | Stream
   | PUT: NON;                                        | | ongoing
   |       URI=/five-bit-id?t=123&pos=1024;           | |
   |       No-response = 127                          | |
   |       Payload= <Bytes_in 2nd _segment>           | |
   +------------------------------------------------->| |
   |                                                  | |
   :                                                  : |

   The re-negotiation information could then go into unsuccessful
   messages (eg. 4.xx Too Many Requests, with a payload indicating to
   increase compression).

   This example moves the stream ID into the URI path and both timestamp
   and position into the URI-Query. I've expressed both timestamp and
   position in decimal format, but that shouldn't be a show stopper; if
   it causes trouble, we'll come up with something (eg. a way of having
   a compressed URI query that's transmitted as int but means a
   decimal-encoded URI-Query component).

   If the offsets and lengths of the messages within a time slice are
   always block-sized, it would be well possible to use blockwise
   transfer for the individual time frames. AFAICT it should be within
   the specification to have some blocks CON transferred, some NON even
   with no-response. As processing of the time frames is not atomic, the
   application could work with blocks that are missing. That'd also give
   us access to other established mechanisms, like sending a Size1
   option with the pos=0 package of a timestamp to indicate how many
   butes there will be for this timestamp.

* In general, video streaming has been my classical example of things
  you would not want to do over CoAP, so please forgive my skepticism.
  I'd be happy to be shown to be wrong, but that might need statistical
  comparison to RTP streaming, and practical demonstration (though I
  only might be saying this because I want to try remote piloting a UAV
  over CoAP myself ;-) )

Best regards
Christian

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

---

From: Abhijan Bhattacharyya <[email-masked]>
Date: Wed, 10 Oct 2018 15:25:27 +0530

Hi Christian,
It is a pleasure to get your valuable comments. Probably the subject 
matter needs to be discussed in the broader context of real-time 
time-series streaming for IoT. Your inputs are important in this regard.

Let us share some ready thoughts in response to some of your comments. We 
can definitely discuss further. 

>   throughput to a halt at every I-Frame?

Yes, the possibility cannot be ignored. However, since I-frames are 
essentially JPEG encoded spatially compressed frames so similar techniques 
used in the MJPEG streaming example can be used. So one can sub-classify 
critical and non-critical information within I-frames and switch the 
semantics accordingly. Anyway, one thing that we need to bear in mind is 
that the I-frames are sparse and separated by the scale of the GoP. P / B 
frames are usually very small in size. So one gets some time to transfer 
the I-frames and then may be serialize the P and B frames in subsequent 
best-effort delivery. Depends on design experiences. May be we can put 
some text regarding this in future versions. 

Having said that, I admit one of your later points where you have 
mentioned about some of the possible optimizations in block wise transfer. 
 These things can be discussed in a broader spectrum. At present we did 
not attempt to disrupt the Block-wise transfer mechanism as it has been 
created with a specific use case of reliable large data delivery. We need 
to define what should be the behaviour if a block option is used with 
No-Response (if at all).

>    decimal-encoded URI-Query component).

What you propose is definitely a sensible approach. 
But we tried to follow the approach in existing streaming standards and 
put the specific header options to help the consuming end relate the 
components of a series-transfer. One thing to note is that in this case 
the header over-head due to stream control information has a deterministic 
limit. Parsing is also straight forward with minimal processing overhead. 
Stream-state in the header helps quick understanding on agreement/ 
disagreement from the other party. In case we put the details in a URI 
query string then that impacts the effective length of the header and we 
need special mechanism to compress the URI. We wanted to avoid these 
complexities. The philosophy is to minimise control level processing to 
cater the hard real time requirements for live feed in AR control 
applications like automated VSLAM (Visual Simultaneous Localization and 
Mapping) for dumb robot terminals and UAVs. So the terminal is not 
manually remote controlled. 

We assume that the time-stamp is presented in epoch time. 

Another point is that we chose to provide autonomy to the producer to 
propose the stream-ID through header option. We consider the typical case 
where a remote (UAV) terminal is depleted of energy and replaced by 
another terminal, which streams as soon as commissioned, using the same 
stream ID (keeping an option for pre-provisioning). It becomes seamless 
for the consumer.

>   only might be saying this because I want to try remote piloting a UAV

RTP is something that will automatically come as a first thought. However, 
according to our experience, commercial IP based streaming implementations 
are using custom protocols on TCP. These protocols are quite in line with 
the usual current practice for RESTful streaming over public Internet 
which uses HTTP-streaming with ABR. So, we tried to create an efficient 
streaming counterpart of HTTP-streaming for IoT. ABR can be an added 
feature to what we proposed in the draft. In future AR the producer may 
not only stream multimedia but can send mulSEmedia (multi SEnsory media) 
with video as a sensory input. So we wanted to have a RESTful protocol 
which is equally efficient in transferring sensor data as well as large 
streams. Also, as we have mentioned in draft, CoAP allows us to create the 
intelligent switch to balance between real-time and reliable delivery 
while optimizing the resource usage and allowing flexibility to react to 
the connection environment. 

Having said that, we have done bench marking experiments to compare with 
RTP. We are not doing bad actually (however, definitely you need to change 
the default time-out limits, etc. for the real-time). 

We can present the approach and use cases in more detail with descriptions 
of the experiments and the bench marking results for further discussion in 
a physical meeting. 

Thank you.
With Best Regards
Abhijan Bhattacharyya
Consultant / Scientist,

"Christian Amsüss" <[email-masked]> wrote on 10/09/2018 12:55:35 
PM:

> From: "Christian Amsüss" <[email-masked]>
> To: "Abhijan Bhattacharyya" <[email-masked]>
> Cc: [email-masked], "Hemant Rath" <[email-masked]>
> Date: 10/09/2018 12:55 PM
> Subject: Re: [core] Fw: New Version Notification for draft-
> bhattacharyya-core-a-realist-00.txt
> 
> Hello Abhijan,
> 
> [attachment "signature.asc" deleted by Abhijan Bhattacharyya/KOL/TCS]

---

From: Mohit Sethi M <[email-masked]>
Date: Wed, 10 Oct 2018 13:00:33 +0000

Hi authors,

I was going through the draft. I looked at section 2.3 that CoAP provides best of both worlds by choosing between Confirmable and Non-confirmable messages. This only true if CoAP is used over UDP. If CoAP is used over TCP/TLS etc. as defined in RFC 8323 then Confirmable/Non-confirmable does not really have any affect. So is the draft trying to do streaming over CoAP+UDP?

Do you have some performance results that show significant benefits of using CoAP instead of RTP or HTTP?

--Mohit
On 10/06/2018 10:23 PM, Abhijan Bhattacharyya wrote:
Dear list,
A new draft has been submitted which essentially tries to facilitate real-time live FPV feeds with high QoE over the Internet under constrained environments. The technology leverages on some of the advantages of CoAP and proposes few extensions so that CoAP can have an efficient way of real-time streaming for the 'Things' version of Internet (I mean IoT), just like HTTP has HTTP-streaming in conventional Internet. So, we can have a protocol which is equally efficient in sending small sensor data as well as streams which may contain video information. This draft is an outcome of the experiments and implementations that we carried out on the face of some real-life challenges faced in robotic and UAV based AR applications.

Comments are welcome.

Dear chairs,

With Best Regards
Abhijan Bhattacharyya
Consultant / Scientist,
Mailto: [email-masked]<mailto:[email-masked]>

-----Forwarded by Abhijan Bhattacharyya/KOL/TCS on 10/07/2018 12:30AM -----
To: "Abhijan Bhattacharyya" <[email-masked]<mailto:[email-masked]>>, "Balamurali Purushothaman" <[email-masked]<mailto:[email-masked]>>, "Suvrat Agrawal" <[email-masked]<mailto:[email-masked]>>, "Hemant Rath" <[email-masked]<mailto:[email-masked]>>, "Arpan Pal" <[email-masked]<mailto:[email-masked]>>
From: [email-masked]<mailto:[email-masked]>
Date: 10/07/2018 12:29AM
Subject: New Version Notification for draft-bhattacharyya-core-a-realist-00.txt

Abstract:

The IETF Secretariat

_______________________________________________
core mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/core

---

From: Abhijan Bhattacharyya <[email-masked]>
Date: Thu, 11 Oct 2018 01:34:44 +0530

Hi Mohit,
Thanks for sparing time to review the draft and showing interest.

> I looked at section 2.3 that CoAP
>provides best of both worlds by choosing between Confirmable and
>Non-confirmable messages. This only true if CoAP is used over UDP. If
>CoAP is used over TCP/TLS etc. as defined in RFC 8323 then
>Confirmable/Non-confirmable does not really have any affect. So is
>the draft trying to do streaming over CoAP+UDP? 

Indeed the draft considers CoAP over UDP. That is why we have put only RFC 7252 in the reference. But, we should explicitly mention this in the draft. Thanks for pointing this out.

>of using CoAP instead of RTP or HTTP?
 Yes we do have. As I mentioned in a previous communication, we have an accepted paper in Globecom workshop on UAV and is due to be presented this December. If we get a chance to present the draft in a physical meeting, we can discuss more with visuals of the experiments and the results under different connection environments.

Thank you.

With Best Regards
Abhijan Bhattacharyya
Consultant / Scientist,

-----"Mohit Sethi M" <[email-masked]> wrote: -----

>To: "Abhijan Bhattacharyya" <[email-masked]>,
>"[email-masked]" <[email-masked]>
>From: "Mohit Sethi M" <[email-masked]>
>Date: 10/10/2018 06:33PM
>Cc: "Hemant Rath" <[email-masked]>
>Subject: Re: [core] Fw: New Version Notification for
>draft-bhattacharyya-core-a-realist-00.txt
>
> I was going through the draft. I looked at section 2.3 that CoAP
>provides best of both worlds by choosing between Confirmable and
>Non-confirmable messages. This only true if CoAP is used over UDP. If
>CoAP is used over TCP/TLS etc. as defined in RFC 8323 then
>Confirmable/Non-confirmable does not really have any affect. So is
>the draft trying to do streaming over CoAP+UDP? 
> 
>A new draft has been submitted which essentially tries to facilitate
>real-time live FPV feeds with high QoE over the Internet under
>constrained environments. The technology leverages on some of the
>advantages of CoAP and proposes few extensions so that CoAP can have
>an efficient way of real-time streaming for the 'Things' version of
>Internet (I mean IoT), just like HTTP has HTTP-streaming in
>conventional Internet. So, we can have a protocol which is equally
>efficient in sending small sensor data as well as streams which may
>contain video information. This draft is an outcome of the
>experiments and implementations that we carried out on the face of
>some real-life challenges faced in robotic and UAV based AR
>applications. 
>
> 
>Comments are welcome. 
>
> 
>Dear chairs, 
>If you may please allow us to present, then we can try to explain the
>concept in physical meetings and also show some visuals from the demo
>that we had and the benchmark results. Looking forward. 
>
> 
>Please note: an extended version of this draft has been accepted to
>be presented in Wi-UAV workshop of Globecom in coming December. 
>
> With Best Regards
> Abhijan Bhattacharyya
> Consultant / Scientist,
> To: "Abhijan Bhattacharyya" <[email-masked]>,
>"Balamurali Purushothaman" <[email-masked]>, "Suvrat Agrawal"
><[email-masked]>, "Hemant Rath" <[email-masked]>, "Arpan Pal"
><[email-masked]>
> Subject: New Version Notification for
>draft-bhattacharyya-core-a-realist-00.txt
> 
> 
> URL:
> https://www.ietf.org/internet-drafts/draft-bhattacharyya-core-a-real
>ist-00.txt
> Status:
>https://datatracker.ietf.org/doc/draft-bhattacharyya-core-a-realist/
> Htmlized:
>https://tools.ietf.org/html/draft-bhattacharyya-core-a-realist-00
> Htmlized:
>https://datatracker.ietf.org/doc/html/draft-bhattacharyya-core-a-real
>ist
> 
> 
> Abstract:
>[email-masked]