Subject: [ipwave] traffic lights status displayed in the car, and the latency problem

From: Alexandre Petrescu <[email-masked]>
Date: Fri, 30 Aug 2019 10:14:47 +0200

Hi,

Has someone else filmed a demo of displaying the traffic lights status 
in the car?  (a communicated status, not a camera recognized status).

We filmed recently several such trials.  We noticed a systematic problem 
with latency.  This problem lies in the difference between the color 
displayed by the traffic light bulb and the color displayed in the car.

Ideally, at no time should a human perceive a difference between what 
the light bulb displays and what the laptop displays in the car.

https://youtu.be/RR5hpL29-vk

At point 3 second, the bulb displays green whereas the laptop displays 
red.  This undesirable situation lasts for 1 second.  It is an enormous 
time lapse.  It is sufficient for the driver to loose confidence in the 
laptop display, and it is ample time for a self-driving car to do many 
undesirable things.  A typical reason for human loosing confidence in 
laptop display is that during such lapse of time (1 second) in many 
places in Paris area one gets honked (klaxonned) if one is first in line 
and does not leave at a moment's notice; because the last in line risks 
having to wait a second cycle - everyone in line knows that and at least 
one will honk (klaxon).  It is forbidden to klaxon in city.

For this video, we did our best to reduce the latency.  The 
communication path between the traffic lights controller and the car was 
set with 4G.  There are two 4G links in sequence: one between the 
traffic lights controller and the VPN server in the cloud, and another 
between the VPN server and the car.  The measured end-to-end latency 
from laptop to traffic lights controller averages 100ms.  The queries to 
obtain the status of lights are sent with a frequency of approx. 20 
Hertz, which is approx. each 50ms. (DIASER on UDP on IPv4).  The laptop 
is a recent thinkpad with python doing queries and display.  The traffic 
lights controller is an Aximum Maestro with a Motorola MC-something.

We could put 802.11-OCB there, to further gain on the communication 
latency.

We could try recent pre-5G chipsets.

We could try the ITS-G5 SPAT-EM technology which relies on DIASER still.

But we are not sure the 1s delay exposed above will get any improvement 
by any of these steps.

This is why I am asking if this situation of latent display of traffic 
lights in car was witnessed elsewhere, and which paths could be explored 
to improve the latency?

Alex

---

From: Chris Shen <[email-masked]>
Date: Fri, 30 Aug 2019 17:52:48 +0900

Hi Alex,

Thanks Alex, it is good to see this experiment.

About the latency, it seems that you use a shared cloud server for both car
and traffic light to access.
That is the traffic light updates its status to the cloud server, and the
car queries the cloud server if there is any update. There is no direct
communication between the traffic light and the car.
So how do you measure the E2E latency from car to traffic light?

Thanks!
Chris.

On Fri, Aug 30, 2019 at 5:15 PM Alexandre Petrescu <
[email-masked]> wrote:

> Hi,
>
> Has someone else filmed a demo of displaying the traffic lights status in
> the car?  (a communicated status, not a camera recognized status).
>
> Ideally, at no time should a human perceive a difference between what the
> light bulb displays and what the laptop displays in the car.
>
> laptop display is that during such lapse of time (1 second) in many places
> in Paris area one gets honked (klaxonned) if one is first in line and does
> not leave at a moment's notice; because the last in line risks having to
> wait a second cycle - everyone in line knows that and at least one will
> honk (klaxon).  It is forbidden to klaxon in city.
>
> For this video, we did our best to reduce the latency.  The communication
> path between the traffic lights controller and the car was set with 4G.
> There are two 4G links in sequence: one between the traffic lights
> controller and the VPN server in the cloud, and another between the VPN
> server and the car.  The measured end-to-end latency from laptop to traffic
> lights controller averages 100ms.  The queries to obtain the status of
> lights are sent with a frequency of approx. 20 Hertz, which is approx. each
> 50ms. (DIASER on UDP on IPv4).  The laptop is a recent thinkpad with python
> doing queries and display.  The traffic lights controller is an Aximum
> Maestro with a Motorola MC-something.
>
> But we are not sure the 1s delay exposed above will get any improvement by
> any of these steps.
>
> lights in car was witnessed elsewhere, and which paths could be explored to
> improve the latency?
>
> Alex
>
>
> _______________________________________________
> its mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/its
>

-- 
Yiwen (Chris) Shen, Ph.D. Candidate

Homepage: https://chrisshen.github.io
IoT Lab: *http://iotlab.skku.edu <http://iotlab.skku.edu/>*
Sungkyunkwan University, Suwon, South Korea
Mobile:+82-(0)10-6871-8103
Email: [email-masked]

---

From: Alexandre Petrescu <[email-masked]>
Date: Fri, 30 Aug 2019 11:17:30 +0200

Hi Chris,

Le 30/08/2019 à 10:52, Chris Shen a écrit :
> About the latency, it seems that you use a shared cloud server for 
> both car and traffic light to access. That is the traffic light 
> updates its status to the cloud server, and the car queries the
> cloud server if there is any update. There is no direct
> communication between the traffic light and the car. So how do you
> measure the E2E latency from car to traffic light?

Thank you for the reply.

Do you consider a cloud server that stores the status of traffic lights?

We measure the E2E latency in two manners: the RTT displayed by ping
command between laptop and traffic lights controller, and the time
difference between the UDP DIASER request and response recorded on the
laptop.  See the points 1 and 2 below.

The car does not query the cloud server, but it queries the traffic
lights controller through VPN.  The traffic lights controller does not
send its status periodically to a server, but it responds to requests
issued by the laptop through VPN.  We do not store the traffic lights
status on the server. If we did this, then it would add up to the latency.

1. the VPN server in the cloud is a rendez-vous point.  It runs OpenVPN
software.  The car laptop opens an openvpn connection to this server.
Also the router-modem of the traffic lights controller opens such a
connection to the same VPN server.  Once the two VPN connections are
open, the car laptop can ping the router-modem of the traffic light
controller and thus note the RTT reported by ping command.  This ping
gets through the VPN server.

2. the laptop originates an UDP DIASER request for status of traffic
lights.  This request is captured by the wireshark in the laptop.  This
UDP DIASER request is sent on the Internet, arrives at VPN server and
gets forwarded to the router-modem attached to the traffic lights
controller.  The router-modem does port forwarding and forwards the
request to the controller.  The controller sends back the UDP DIASER
reply containing the status.  This UDP DIASER reply is captured by the
wireshark in the laptop.  The time difference between those two messages
makes for the E2E latency.

Alex

> 
> Thanks! Chris.
> 
> On Fri, Aug 30, 2019 at 5:15 PM Alexandre Petrescu 
> <[email-masked] <mailto:[email-masked]>> 
> wrote:
> 
> Hi,
> 
> Has someone else filmed a demo of displaying the traffic lights 
> status in the car?  (a communicated status, not a camera recognized 
> status).
> 
> We filmed recently several such trials.  We noticed a systematic 
> problem with latency.  This problem lies in the difference between 
> the color displayed by the traffic light bulb and the color
> displayed in the car.
> 
> Ideally, at no time should a human perceive a difference between
> what the light bulb displays and what the laptop displays in the
> car.
> 
> At point 3 second, the bulb displays green whereas the laptop 
> displays red.  This undesirable situation lasts for 1 second.  It is
>  an enormous time lapse.  It is sufficient for the driver to loose 
> confidence in the laptop display, and it is ample time for a 
> self-driving car to do many undesirable things.  A typical reason
> for human loosing confidence in laptop display is that during such
> lapse of time (1 second) in many places in Paris area one gets
> honked (klaxonned) if one is first in line and does not leave at a
> moment's notice; because the last in line risks having to wait a
> second cycle - everyone in line knows that and at least one will honk
> (klaxon). It is forbidden to klaxon in city.
> 
> communication path between the traffic lights controller and the car
>  was set with 4G.  There are two 4G links in sequence: one between
> the traffic lights controller and the VPN server in the cloud, and 
> another between the VPN server and the car.  The measured end-to-end
>  latency from laptop to traffic lights controller averages 100ms.
> The queries to obtain the status of lights are sent with a frequency
> of approx. 20 Hertz, which is approx. each 50ms. (DIASER on UDP on 
> IPv4).  The laptop is a recent thinkpad with python doing queries
> and display.  The traffic lights controller is an Aximum Maestro with
> a Motorola MC-something.
> 
> But we are not sure the 1s delay exposed above will get any 
> improvement by any of these steps.
> 
> This is why I am asking if this situation of latent display of 
> traffic lights in car was witnessed elsewhere, and which paths could
>  be explored to improve the latency?
> 
> Alex
> 
> 
> _______________________________________________ its mailing list 
> [email-masked] <mailto:[email-masked]> 
> -- Yiwen (Chris) Shen, Ph.D. Candidate
> 
> Homepage: https://chrisshen.github.io IoT Lab: 
> _http://iotlab.skku.edu <http://iotlab.skku.edu/>_ Sungkyunkwan 
> University, Suwon, South Korea Mobile:+82-(0)10-6871-8103 Email: 
> [email-masked] <mailto:[email-masked]>

---

From: "Mr. Jaehoon Paul Jeong" <[email-masked]>
Date: Fri, 30 Aug 2019 18:56:02 +0900

Alex,
The detour path through the VPN server using LTE links experiences such a
long delay.
This is why IPWAVE is required to allow a vehicle and a traffic light
controller to communicate directly with each other
using IEEE 802.11-OCB links.

Did you measure the response time from a laptop to a traffic light
controller using 802.11-OCB?

We need to break down the delay components to analyze the bottleneck in
this traffic light control scenario.

Thanks.

Paul

-- 
===========================
Mr. Jaehoon (Paul) Jeong, Ph.D.
Associate Professor
Department of Software
Sungkyunkwan University
Office: +82-31-299-4957
Email: [email-masked], [email-masked]
Personal Homepage: http://iotlab.skku.edu/people-jaehoon-jeong.php
<http://cpslab.skku.edu/people-jaehoon-jeong.php>

---

From: Alexandre Petrescu <[email-masked]>
Date: Fri, 30 Aug 2019 12:31:07 +0200

Paul,

Le 30/08/2019 à 11:56, Mr. Jaehoon Paul Jeong a écrit :
> Alex,
> The detour path through the VPN server using LTE links experiences such 
> a long delay.

No, we did not yet perform tests with 802.11-OCB between car and traffic 
lights controller.

We can consider that.

> this traffic light control scenario.

I agree.

Alex

> 
> Thanks.
> 
> Paul
> 
> 
> 
> On Fri, Aug 30, 2019 at 6:17 PM Alexandre Petrescu 
> <[email-masked] <mailto:[email-masked]>> wrote:
> 
>     Hi Chris,
> 
>      > <[email-masked]
>     <mailto:[email-masked]>
>     <mailto:[email-masked]
>     <mailto:[email-masked]>>>
>      > wrote:
>      >
>      > Hi,
>      >
>      > [email-masked] <mailto:[email-masked]> <mailto:[email-masked]
>     <mailto:[email-masked]>>
>     <mailto:[email-masked] <mailto:[email-masked]>>
> 
> Email: [email-masked] <mailto:[email-masked]>, 
> [email-masked] <mailto:[email-masked]>

---

From: <[email-masked]>
Date: Fri, 30 Aug 2019 12:33:02 -0400

The 802.11 OCB was never intended to communicate with the traffic controller, the RSE is.
As far as for latency, It is almost impossible to determine the latency of an 802.11 OCB (it is CSMA based).

Please find attached old documents 2011 which may help you understand.

Fygs

Sent from Mail for Windows 10

From: Alexandre Petrescu
Sent: Friday, August 30, 2019 6:31 AM
To: Mr. Jaehoon Paul Jeong
Cc: [email-masked]; Chris Shen; [email-masked]
Subject: Re: [ipwave] traffic lights status displayed in the car, and thelatency problem

Paul,

We can consider that.

I agree.

Alex