Subject: [ippm] draft-fioccola-ippm-rfc6812-alt-mark-ext-00

From: Esteban Carisimo <[email-masked]>
Date: Tue, 26 Jan 2016 10:22:00 -0300

Hi everybody,

I'm Esteban Carisimo, I'm a PhD candidate at University of Buenos Aires 
and I've been working with active measurements on my PhD Thesis.

I've read the draft 
https://tools.ietf.org/id/draft-fioccola-ippm-rfc6812-alt-mark-ext-00.txt and 
I've got a little suggestion about it. At page 5, the draft propose 
using "Average delay", however, the nature of the delay might highly 
affect the average of a certain block. I suggest to use median delay 
rather than mean delay, or at least use both, to reduce the variation of 
the parameter. As I've told before, I've been doing some active 
measurements and I'd like to share an analysis that shows the advantage 
of median over mean.

This analysis consist in analyzing the minimum, maximum, mean and median 
values of a dataset of RTT hour by hour. In order to obtain the RTTs we 
did a traceroute per second to the default gateway during a week. The 
attached graph shows that the mean is very sensitible with the maximum's 
increasement while the median keeps almost flat.

Best regards,

Esteban

---

From: Fioccola Giuseppe <[email-masked]>
Date: Wed, 27 Jan 2016 16:57:24 +0000

Hi Esteban,
Thanks for reading the draft and for your interesting comments.
Regarding your suggestions, I also believe that the median value is a better information, but you need to know the delay for each packet in order to calculate the median value.
Average delay could be a good evaluation for the delay if we choose a short marking interval, so we have less delay variability.
We proposed two choices for active performance measurements:
- Delay for each packet: in this case we can calculate maximum, minimum, average and median.
- Average Delay: in this case we calculate the average delay without exchanging data between the two end points. It is a smart way to evaluate delay with a low computational load on a network element, because you don't need to transmit all the timestamps but only the average timestamp.
I think that we can add your suggestion in the Delay measurement section.
What do you think?
I appreciate your comment and I suggest you to read also the related document draft-tempia-ippm-p3m-02, that is about passive performance measurements.

Regards,

Giuseppe

-----Messaggio originale-----
Da: ippm [mailto:[email-masked]] Per conto di Esteban Carisimo
Inviato: martedì 26 gennaio 2016 14:22
A: [email-masked]
Oggetto: [ippm] draft-fioccola-ippm-rfc6812-alt-mark-ext-00

Hi everybody,

I'm Esteban Carisimo, I'm a PhD candidate at University of Buenos Aires and I've been working with active measurements on my PhD Thesis.

I've read the draft
https://tools.ietf.org/id/draft-fioccola-ippm-rfc6812-alt-mark-ext-00.txt and I've got a little suggestion about it. At page 5, the draft propose using "Average delay", however, the nature of the delay might highly affect the average of a certain block. I suggest to use median delay rather than mean delay, or at least use both, to reduce the variation of the parameter. As I've told before, I've been doing some active measurements and I'd like to share an analysis that shows the advantage of median over mean.

This analysis consist in analyzing the minimum, maximum, mean and median values of a dataset of RTT hour by hour. In order to obtain the RTTs we did a traceroute per second to the default gateway during a week. The attached graph shows that the mean is very sensitible with the maximum's increasement while the median keeps almost flat.

Best regards,

Esteban

Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie.

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks.

---

From: Esteban Carisimo <[email-masked]>
Date: Thu, 28 Jan 2016 12:48:14 -0300

Giussepe,

     Before keep on discussing about mean and median value on the delay, 
I have to say that there is a previous RFC, which was written by this 
group (I mean IPPM), which is RFC 6703 
(https://tools.ietf.org/html/rfc6703). In this work the authors discuss 
about advantages and disadvantages of mean and median value of the 
latency. I think the knowledge of this RFC might solve the issue  of the 
median and mean.

     I have been thinking about what you said that Average delay could 
suffer little variation in small time windows. However, we should define 
what is a small time inteval. For example, I did a few pings from my 
laptop to 8.8.8.8 and check this out:

     64 bytes from 8.8.8.8: icmp_seq=0 ttl=56 time=4.987 ms
     64 bytes from 8.8.8.8: icmp_seq=1 ttl=56 time=4.882 ms
     64 bytes from 8.8.8.8: icmp_seq=2 ttl=56 time=4.933 ms
     64 bytes from 8.8.8.8: icmp_seq=3 ttl=56 time=4.501 ms
     64 bytes from 8.8.8.8: icmp_seq=4 ttl=56 time=5.048 ms
     64 bytes from 8.8.8.8: icmp_seq=5 ttl=56 time=5.283 ms
     64 bytes from 8.8.8.8: icmp_seq=6 ttl=56 time=4.556 ms
     64 bytes from 8.8.8.8: icmp_seq=7 ttl=56 time=4.582 ms
     64 bytes from 8.8.8.8: icmp_seq=8 ttl=56 time=89.366 ms
     64 bytes from 8.8.8.8: icmp_seq=9 ttl=56 time=108.486 ms
     64 bytes from 8.8.8.8: icmp_seq=10 ttl=56 time=101.605 ms
     64 bytes from 8.8.8.8: icmp_seq=11 ttl=56 time=5.737 ms
     ^C
     --- 8.8.8.8 ping statistics ---
     12 packets transmitted, 12 packets received, 0.0% packet loss
     round-trip min/*avg*/max/stddev = 4.501/*28.664*/108.486/41.273 ms

*median=5.0175*

     Doing just 12 pings, with a time window of 12 seconds we figured 
out that mean is 28ms while median is 5 ms.
I know that the delay measurements on this RFC will be done in a 
different way but we should check which might be a proper time window in 
order to get an appropiate mean value.

     However, I agree with mean value is quite simple to be calculated. 
I think we should be concerned about the overload on the routers 
produced by any kind of measurements.

Best regards,

Esteban

---

From: Fioccola Giuseppe <[email-masked]>
Date: Thu, 28 Jan 2016 17:14:31 +0000

"i'm not sure if the whole IPPM WG list received my email so i'm sending a copy"

Hi Esteban,

Regards,

Giuseppe

-----Messaggio originale-----
Da: ippm [mailto:[email-masked]] Per conto di Esteban Carisimo
Inviato: martedì 26 gennaio 2016 14:22
A: [email-masked]
Oggetto: [ippm] draft-fioccola-ippm-rfc6812-alt-mark-ext-00

Hi everybody,

I've read the draft

Best regards,

Esteban

---

From: Fioccola Giuseppe <[email-masked]>
Date: Thu, 28 Jan 2016 17:39:21 +0000

Hi Esteban,
Many thanks for the analysis.
I agree that there is a trade-off between average and median value.
I think we can add the reference to RFC 6703 and we can detail more about the appropriate delay value to choose depending on the case.
Will be great if you could propose some text.

Best Regards,

Giuseppe

Da: Esteban Carisimo [mailto:[email-masked]]
Inviato: giovedì 28 gennaio 2016 16:48
A: Fioccola Giuseppe; [email-masked]
Oggetto: Re: R: [ippm] draft-fioccola-ippm-rfc6812-alt-mark-ext-00

Giussepe,

    Before keep on discussing about mean and median value on the delay, I have to say that there is a previous RFC, which was written by this group (I mean IPPM), which is RFC 6703 (https://tools.ietf.org/html/rfc6703). In this work the authors discuss about advantages and disadvantages of mean and median value of the latency. I think the knowledge of this RFC might solve the issue  of the median and mean.

    I have been thinking about what you said that Average delay could suffer little variation in small time windows. However, we should define what is a small time inteval. For example, I did a few pings from my laptop to 8.8.8.8 and check this out:

    round-trip min/avg/max/stddev = 4.501/28.664/108.486/41.273 ms

    median=5.0175

    Doing just 12 pings, with a time window of 12 seconds we figured out that mean is 28ms while median is 5 ms.
I know that the delay measurements on this RFC will be done in a different way but we should check which might be a proper time window in order to get an appropiate mean value.

    However, I agree with mean value is quite simple to be calculated. I think we should be concerned about the overload on the routers produced by any kind of measurements.

Best regards,

Esteban

Hi Esteban,

What do you think?

Regards,

Giuseppe

-----Messaggio originale-----

A: [email-masked]<mailto:[email-masked]>

Hi everybody,

I've read the draft

Best regards,

Esteban

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Thu, 28 Jan 2016 13:04:25 -0500

Hi Giuseppe,

RFC 6703 recommends to report both median and mean delay,
and that’s partly because the difference between these two
statistics gives additional information about the distribution.

So, there may be no need to provide detail on which to choose.
Al

From: ippm [mailto:[email-masked]] On Behalf Of Fioccola Giuseppe
Sent: Thursday, January 28, 2016 12:39 PM
To: Esteban Carisimo; [email-masked]
Subject: [ippm] R: R: draft-fioccola-ippm-rfc6812-alt-mark-ext-00

Hi Esteban,

Best Regards,

Giuseppe

Da: Esteban Carisimo [mailto:[email-masked]]
Inviato: giovedì 28 gennaio 2016 16:48
A: Fioccola Giuseppe; [email-masked]<mailto:[email-masked]>
Oggetto: Re: R: [ippm] draft-fioccola-ippm-rfc6812-alt-mark-ext-00

Giussepe,

    median=5.0175

Best regards,

Esteban

Hi Esteban,

What do you think?

Regards,

Giuseppe

-----Messaggio originale-----

Hi everybody,

I've read the draft

Best regards,

Esteban

---

From: Fioccola Giuseppe <[email-masked]>
Date: Fri, 29 Jan 2016 13:45:05 +0000

Hi Al,
Many thanks for your feedback,
Absolutely, the knowledge of both median and mean delay is the preferable choice.
In fact the first proposed technique is to calculate the delay for each packet in order to obtain the delay distribution (maximum, minimum, median, mean,…).
Additionally, we introduced the alternative possibility, in case we want a "very light" measurement, to calculate the mean delay by exchanging only the average timestamp for each colored batch (without exchanging all the timestamps) as long as you can choose a proper duration of the colored batch.
We cannot do the same with the median timestamp because the median is not a linear operator.
But I want to highlight that the mean delay is not the main technique, but a secondary option in case we have to save computational resources.
I will review the draft after these significant hints.

I will appreciate if you could take time to read this draft and also the related document draft-tempia-ippm-p3m-02, where the alternatives for delay measurements are explained in details.
Comments are always welcome.

Best Regards,

Giuseppe

Da: MORTON, ALFRED C (AL) [mailto:[email-masked]]
Inviato: giovedì 28 gennaio 2016 19:04
A: Fioccola Giuseppe; Esteban Carisimo; [email-masked]
Oggetto: RE: [ippm] R: R: draft-fioccola-ippm-rfc6812-alt-mark-ext-00

Hi Giuseppe,

From: ippm [mailto:[email-masked]] On Behalf Of Fioccola Giuseppe
Sent: Thursday, January 28, 2016 12:39 PM
To: Esteban Carisimo; [email-masked]<mailto:[email-masked]>
Subject: [ippm] R: R: draft-fioccola-ippm-rfc6812-alt-mark-ext-00

Hi Esteban,

Best Regards,

Giuseppe

Giussepe,

    median=5.0175

Best regards,

Esteban

Hi Esteban,

What do you think?

Regards,

Giuseppe

-----Messaggio originale-----

Hi everybody,

I've read the draft

Best regards,

Esteban