Subject: [ippm] FW: New Version Notification for draft-chen-ippm-coloring-based-ipfpm-framework-05.txt

From: Mach Chen <[email-masked]>
Date: Mon, 25 Jan 2016 03:06:48 +0000

Hi IPPMers,

This update is just to keep the document alive. We received some comments offline, we'd appreciate that there will be more people who could spend some time to review the draft and send the comments.

Thanks,
Mach

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Monday, January 25, 2016 11:01 AM
To: Mach Chen; Greg Mirsky; Vero Zheng; Mach Chen; Giuseppe Fioccola
Subject: New Version Notification for draft-chen-ippm-coloring-based-ipfpm-framework-05.txt

A new version of I-D, draft-chen-ippm-coloring-based-ipfpm-framework-05.txt
has been successfully submitted by Mach(Guoyi) Chen and posted to the IETF repository.

Name:		draft-chen-ippm-coloring-based-ipfpm-framework
Revision:	05
Title:		IP Flow Performance Measurement Framework
Document date:	2016-01-24
Group:		Individual Submission
Pages:		16
URL:            https://www.ietf.org/internet-drafts/draft-chen-ippm-coloring-based-ipfpm-framework-05.txt
Status:         https://datatracker.ietf.org/doc/draft-chen-ippm-coloring-based-ipfpm-framework/
Htmlized:       https://tools.ietf.org/html/draft-chen-ippm-coloring-based-ipfpm-framework-05
Diff:           https://www.ietf.org/rfcdiff?url2=draft-chen-ippm-coloring-based-ipfpm-framework-05

Abstract:
   This document specifies a measurement method, the IP flow performance
   measurement (IPFPM).  With IPFPM, data packets are marked into
   different blocks of markers by changing one or more bits of packets.
   No additional delimiting packet is needed and the performance is
   measured in-service and in-band without the insertion of additional
   traffic.

Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: Gregory Mirsky <[email-masked]>
Date: Mon, 25 Jan 2016 19:30:23 +0000

Dear BIERers (I like its sound),

this work is closely related to Performance Measurement (PM) with Marking Method in Bit Index Explicit Replication (BIER) Layer<https://tools.ietf.org/html/draft-mirsky-bier-pmmm-oam-00>.

Your questions, comments on both or any of the drafts are always welcome and greatly appreciated.

                Regards,

                                Greg

-----Original Message-----
From: ippm [mailto:[email-masked]] On Behalf Of Mach Chen
Sent: Sunday, January 24, 2016 7:07 PM
To: [email-masked]
Subject: [ippm] FW: New Version Notification for draft-chen-ippm-coloring-based-ipfpm-framework-05.txt

Hi IPPMers,

Thanks,

Mach

-----Original Message-----

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]

Revision:              05

Pages:                   16

Abstract:

   traffic.

The IETF Secretariat

_______________________________________________

ippm mailing list

[email-masked]<mailto:[email-masked]>

https://www.ietf.org/mailman/listinfo/ippm

---

From: "J. Ignacio Alvarez-Hamelin" <[email-masked]>
Date: Mon, 25 Jan 2016 18:36:33 -0300 (ART)

Hi IPPMers,

I have just read this draft version, and below are my comments.

Page 4:

   "An additional method consists in taking into account the average 
arrival
    time of the packets within a single block (i.e. the same block of
    markers used for packet loss measurement)." and the following 
sentences...

Since packet delay is highly variable (e.g., self-similar traffic at 
Internet edges), I suggest to use the median instead of average; this 
*estimator* will be less biased by rare events.

Page 5:

   "Type of Service (TOS)"

Should be: "Type of Service (ToS)"

Page 8:

   "This
    requires that the upstream and downstream MAs having a certain time
    synchronization capability (e.g., supporting the Network Time
    Protocol (NTP) [RFC5905], or the IEEE 1588 Precision Time Protocol
    (PTP) [IEEE1588].)  The PN is calculated as the modulo of the local
    time (when the counts or timestamps are read) and the interval of the
    marking time period."

It could be interesting to include a brief comment about precision of both 
methods (because the errors of the one-way measurements depend on 
timestamp precision).
Even though NTP and PTP synchronization methods are widely accepted, this 
document may
consider forthcoming novel methods.

Best regards,

         J.Ignacio

_______________________________________________________________

CONICET and Facultad de Ingeniería, Universidad de Buenos Aires
Av. Paseo Colón 850 - C1063ACV - Buenos Aires - Argentina
+54 (11) 4343-0891 / 4343-2775  ext 299
e-mail: [email-masked]
web: http://cnet.fi.uba.ar/ignacio.alvarez-hamelin/
_______________________________________________________________

On Mon, 25 Jan 2016, Mach Chen wrote:

> [email-masked]

---

From: Tal Mizrahi <[email-masked]>
Date: Tue, 26 Jan 2016 09:41:11 +0000

Hi Mach, 

I have read the draft and I believe it certainly worth pursuing in the IPPM WG.

A couple of comments:

1. Section 6 discusses the need for synchronization. Note that while this is true for one-way delay measurement, *two-way* delay measurement does not require the two MAs to be accurately synchronized. 
However, since the Period Number is derived from the time-of-day, MAs should be synchronized with an accuracy of +/- 0.5*period to guarantee consistent period numbering.

2. Regarding the Security Considerations section, you may want to consider adding: (1) Network reconnaissance. (2) Attacks on the synchronization protocol that may compromise the measurement.

Best regards,
Tal Mizrahi.

>Sent: Monday, January 25, 2016 5:07 AM
>To: [email-masked]
>This update is just to keep the document alive. We received some comments
>offline, we'd appreciate that there will be more people who could spend some
>time to review the draft and send the comments.
>
>Thanks,
>Mach
>
>Please note that it may take a couple of minutes from the time of submission
>until the htmlized version and diff are available at tools.ietf.org.
>
>The IETF Secretariat
>

---

From: Mach Chen <[email-masked]>
Date: Wed, 27 Jan 2016 08:54:34 +0000

Hi Tal,

Thanks for reading the draft and your valuable comments!

Please see my replies inline..

> From: Tal Mizrahi [mailto:[email-masked]]
> Sent: Tuesday, January 26, 2016 5:41 PM
> To: Mach Chen; [email-masked]
> Subject: RE: New Version Notification for
> draft-chen-ippm-coloring-based-ipfpm-framework-05.txt
> 
> Hi Mach,
> 

Thanks!

> 
> A couple of comments:
> 
> 1. Section 6 discusses the need for synchronization. Note that while this is true
> for one-way delay measurement, *two-way* delay measurement does not
> require the two MAs to be accurately synchronized.

Yes.

> However, since the Period Number is derived from the time-of-day, MAs should
> be synchronized with an accuracy of +/- 0.5*period to guarantee consistent
> period numbering.

I am not sure that the "+/- 0.5" will apply to all the case, but I agree that there should be some text to discuss this accuracy related stuff. 

> 
> 2. Regarding the Security Considerations section, you may want to consider
> adding: (1) Network reconnaissance. (2) Attacks on the synchronization
> protocol that may compromise the measurement.

Good idea, that will be great if you could provide some proposed text!

Thanks,
Mach

> >Subject: [ippm] FW: New Version Notification for
> >draft-chen-ippm-coloring- based-ipfpm-framework-05.txt
> >
> >Hi IPPMers,
> >
> >This update is just to keep the document alive. We received some
> >comments offline, we'd appreciate that there will be more people who
> >could spend some time to review the draft and send the comments.
> >
> >Thanks,
> >Mach
> >
> >Subject: New Version Notification for
> >draft-chen-ippm-coloring-based-ipfpm-
> >framework-05.txt
> >
> >
> >URL:
> https://www.ietf.org/internet-drafts/draft-chen-ippm-coloring-
> >based-ipfpm-framework-05.txt
> >Status:
> https://datatracker.ietf.org/doc/draft-chen-ippm-coloring-based-
> >ipfpm-framework/
> >Diff:
> https://www.ietf.org/rfcdiff?url2=draft-chen-ippm-coloring-based-
> >ipfpm-framework-05
> >
> >Abstract:
> >Please note that it may take a couple of minutes from the time of
> >submission until the htmlized version and diff are available at tools.ietf.org.
> >
> >The IETF Secretariat
> >

---

From: Mach Chen <[email-masked]>
Date: Wed, 27 Jan 2016 09:09:02 +0000

Hi J.Ignacio,

Thanks for your review and the comments!

Please see my response inline...

> From: ippm [mailto:[email-masked]] On Behalf Of J. Ignacio
> Alvarez-Hamelin
> Sent: Tuesday, January 26, 2016 5:37 AM
> To: Mach Chen
> Cc: [email-masked]
> Subject: Re: [ippm] FW: New Version Notification for

Thanks!

> 
> 
> Page 4:
> 
>    "An additional method consists in taking into account the average arrival
> Since packet delay is highly variable (e.g., self-similar traffic at Internet edges),
> I suggest to use the median instead of average; this

Good suggestion, but sometime people may just want average, maybe both can be mentioned here. How do you think?
> Should be: "Type of Service (ToS)"

OK.

> methods (because the errors of the one-way measurements depend on

I do not have the figures at hand, we'd appreciate if you or someone else can provide the figures, then we may add this information to document as reference.

> document may consider forthcoming novel methods.

IMHO, NTP and PTP are just two examples, this document does not exclude any other methods. 

Do you have a specific method in mind? Or just want to give an rough description here.  How about this? To add an "etc" at the end.

(e.g., supporting the Network Time
     Protocol (NTP) [RFC5905], or the IEEE 1588 Precision Time Protocol
     (PTP) [IEEE1588], etc.)

Thanks,
Mach

> CONICET and Facultad de Ingeniería, Universidad de Buenos Aires Av. Paseo
> Colón 850 - C1063ACV - Buenos Aires - Argentina
> > A new version of I-D,
> https://www.ietf.org/internet-drafts/draft-chen-ippm-coloring-based-ipfpm-fra
> mework-05.txt
> > Status:
> https://datatracker.ietf.org/doc/draft-chen-ippm-coloring-based-ipfpm-framew
> ork/
> > Htmlized:
> https://tools.ietf.org/html/draft-chen-ippm-coloring-based-ipfpm-framework-0
> 5
> > Diff:
> https://www.ietf.org/rfcdiff?url2=draft-chen-ippm-coloring-based-ipfpm-frame
> work-05
> >
> > Abstract:

---

From: "J. Ignacio Alvarez-Hamelin" <[email-masked]>
Date: Wed, 27 Jan 2016 10:03:47 -0300 (ART)

Hi Math,

also inline answers

> Good suggestion, but sometime people may just want average, maybe both can be mentioned here. How do you think?

Yes, I agree, we can mention both (I know that average seems simpler, but
I was thinking on that the average is a biased estimator while the median 
is not).

> I do not have the figures at hand, we'd appreciate if you or someone 
> else can provide the figures, then we may add this information to 
> document as reference.
>

Yes, I think this should be good. Perhaps we can also add a comment to
get an idea about their precision like:
   Using NTP one can get precision aground milliseconds while PTP can
   achieve microseconds (and nanoseconds while it is used in a LAN).

> Mach

Thanks to you

---

From: Fioccola Giuseppe <[email-masked]>
Date: Wed, 27 Jan 2016 15:56:52 +0000

Hi Tal, All
Thanks for your comments.
Regarding MAs synchronization, I agree with you.
The clock error between the MAs has to be less than  +/- P/2 time units, where P is the Period length. In this way each colored packet can be assigned to the right batch by a MA. This is because the minimum time distance between two packets of the same color but belonging to different batches is P time units. In fact we need to wait P/2 time units after the foreseen color changing in order to read the still counter of the previous color, so we minimize the possibility to read a running counter instead of a still one.
I think your comment is relevant.
We touched on this issue in draft-tempia-ippm-p3m-02, I suggest you to read also draft-tempia-ippm-p3m-02 that is the companion document of draft-chen-ippm-coloring-based-ipfpm-framework-05.

I hope this is helpful.

Regards,

Giuseppe

-----Messaggio originale-----
Da: ippm [mailto:[email-masked]] Per conto di Mach Chen
Inviato: mercoledì 27 gennaio 2016 09:55
A: Tal Mizrahi; [email-masked]
Oggetto: Re: [ippm] New Version Notification for draft-chen-ippm-coloring-based-ipfpm-framework-05.txt

Hi Tal,

Thanks!

>
> A couple of comments:
>
> 1. Section 6 discusses the need for synchronization. Note that while
> this is true for one-way delay measurement, *two-way* delay
> measurement does not require the two MAs to be accurately synchronized.

Yes.

> However, since the Period Number is derived from the time-of-day, MAs
> should be synchronized with an accuracy of +/- 0.5*period to guarantee
> consistent period numbering.

Thanks,
Mach

Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie.

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks.

---

From: Fioccola Giuseppe <[email-masked]>
Date: Wed, 27 Jan 2016 16:47:57 +0000

Hi J.Ignacio, All
many thanks for your comments.
I agree with you: the median value is a better estimator than the average value, but I take the opportunity to clarify why we choose the average delay.
Your understanding is right, the average delay is a smart way to calculate the delay.
It is simple to calculate because the network devices update the average timestamp for each received marking packet and don't need to collect any timestamp, so it greatly reduces the number of timestamps that have to be collected by the management system.  On the other hand, it only gives one measure for the duration of the block and it doesn't give the minimum, maximum and median delay values. This limitation could be overcome by reducing the duration of the block (f.i. from 5 minutes to a few seconds) by means of an highly optimized implementation of the method.
But if we want to know more about the statistic distribution of delay values, the other approach can be considered: you can periodically mark packets in the IP flow, the timestamps have to be collected by the management system, that can calculate the minimum, the maximum and the median delay values.
What do you think?

I suggest you to read also draft-tempia-ippm-p3m-02, that is the companion document of draft-chen-ippm-coloring-based-ipfpm-framework-05, where the different alternatives for the delay calculation are explained.

Best Regards,

Giuseppe

-----Messaggio originale-----
Da: ippm [mailto:[email-masked]] Per conto di J. Ignacio Alvarez-Hamelin
Inviato: mercoledì 27 gennaio 2016 14:04
A: [email-masked]
Oggetto: Re: [ippm] FW: New Version Notification for draft-chen-ippm-coloring-based-ipfpm-framework-05.txt

Hi Math,

also inline answers

> Good suggestion, but sometime people may just want average, maybe both can be mentioned here. How do you think?

Yes, I agree, we can mention both (I know that average seems simpler, but I was thinking on that the average is a biased estimator while the median is not).

Thanks to you

---

From: Fioccola Giuseppe <[email-masked]>
Date: Thu, 28 Jan 2016 17:12:11 +0000

"i'm not sure if the whole IPPM WG list received my email so i'm sending a copy"

Hi Tal, All

I hope this is helpful.

Regards,

Giuseppe

-----Messaggio originale-----
Da: ippm [mailto:[email-masked]] Per conto di Mach Chen
Inviato: mercoledì 27 gennaio 2016 09:55
A: Tal Mizrahi; [email-masked]
Oggetto: Re: [ippm] New Version Notification for draft-chen-ippm-coloring-based-ipfpm-framework-05.txt

Hi Tal,

Thanks!

Yes.

Thanks,
Mach

---

From: Fioccola Giuseppe <[email-masked]>
Date: Thu, 28 Jan 2016 17:13:38 +0000

Hi J.Ignacio, All

Best Regards,

Giuseppe

-----Messaggio originale-----
Da: ippm [mailto:[email-masked]] Per conto di J. Ignacio Alvarez-Hamelin
Inviato: mercoledì 27 gennaio 2016 14:04
A: [email-masked]
Oggetto: Re: [ippm] FW: New Version Notification for draft-chen-ippm-coloring-based-ipfpm-framework-05.txt

Hi Math,

also inline answers

>> It could be interesting to include a brief comment about precision of
>> both methods (because the errors of the one-way measurements depend
>> on timestamp precision).
>
>> Even though NTP and PTP synchronization methods are widely accepted,
>> this document may consider forthcoming novel methods.
>

Yes, I think this should be good. Perhaps we can also add a comment to get an idea about their precision like:
   Using NTP one can get precision aground milliseconds while PTP can
   achieve microseconds (and nanoseconds while it is used in a LAN).

Thanks to you

>> CONICET and Facultad de Ingeniería, Universidad de Buenos Aires Av.
>> Paseo Colón 850 - C1063ACV - Buenos Aires - Argentina
>>> comments
>> offline, we'd appreciate that there will be more people who could
>> spend some time to review the draft and send the comments.
>>>
>>> Thanks,
>>> Mach
>>>
>>> has been successfully submitted by Mach(Guoyi) Chen and posted to
>>> the IETF
>> repository.
>>>
>>> submission

---

From: "J. Ignacio Alvarez-Hamelin" <[email-masked]>
Date: Thu, 28 Jan 2016 17:44:02 -0300 (ART)

Hi Fioccola, All

You are right about the mean and its simplicity. Nevertheless, here it is 
a reference to compute median without memory:

   Jain, Raj, and Imrich Chlamtac. "The P 2 algorithm for dynamic
   calculation of quantiles and histograms without storing observations."
   Communications of the ACM 28.10 (1985): 1076-1085.

Briefly, this article propose to compute min, p/2, p, (1+p)/2, max; where 
for p=0.5 is standard quartile computation. For each new value it makes a 
simple calculation and estimates the p value using just these 5 previous 
values.

Thank you for pointing out the draft-tempia-ippm-p3m-02.

Best regards,

 	J.Ignacio

---

From: Fioccola Giuseppe <[email-masked]>
Date: Fri, 29 Jan 2016 14:22:29 +0000

Hi J. Ignacio, All
Thanks for the reference.
This is very useful for the processing of the data by the MCP. But I think it is out of scope here. 
The mean delay is one of the several alternatives we propose for the delay calculation.
And it is based on the hypothesis that each MA does not know the timestamps of the other MAs, so you cannot apply the algorithm locally without knowing the delay of a segment. 
The mean is convenient because it is a linear operator so we can calculate the mean delay from the average timestamps. But we cannot do the same with the median, because it is not a linear operator so we cannot deduce the median delay from the median timestamps.
Sometimes a MA could not be able to take and transmit all the packet timestamps of real traffic to the MCP, but even if a MA could do that, it may be useless because of out of order issue.
On the other hand, the mean delay calculated for all the packets of a batch is resilient to out of order and it also reduces the computational load and the data that a MA have to transmit to the MCP.

Another option is called double marking methodology. A second marking selects the packets for measuring delay by creating a new set of marked packets within the first marking batches. These second marking packets are chosen at the right "time distance" to avoid out of order, and are fully identified so that a MA can store the timestamps of these packets. These timestamps can be compared on the MCP for the processing and for the calculation of maximum, minimum, median, mean, etc. but just for a few packets within the batches (see draft-tempia-ippm-p3m-02, in draft-chen-ippm-coloring-based-ipfpm-framework-05 a similar technique is presented with only one measured packet per batch).

In brief, because we cannot know the delay for each packet of a batch, may be useful to know the mean delay of the whole batch together with the delay distribution of only a few packets within the batch.
I hope this helps.

Regards,

Giuseppe

-----Messaggio originale-----
Da: J. Ignacio Alvarez-Hamelin [mailto:[email-masked]] 
Inviato: giovedì 28 gennaio 2016 21:44
A: [email-masked]
Cc: Fioccola Giuseppe
Oggetto: Re: [ippm] R: FW: New Version Notification for draft-chen-ippm-coloring-based-ipfpm-framework-05.txt

Hi Fioccola, All

You are right about the mean and its simplicity. Nevertheless, here it is a reference to compute median without memory:

Briefly, this article propose to compute min, p/2, p, (1+p)/2, max; where for p=0.5 is standard quartile computation. For each new value it makes a simple calculation and estimates the p value using just these 5 previous values.

Best regards,

 	J.Ignacio

On Thu, 28 Jan 2016, Fioccola Giuseppe wrote:

> Oggetto: Re: [ippm] FW: New Version Notification for 
>>> It could be interesting to include a brief comment about precision 
>>> of both methods (because the errors of the one-way measurements 
>>> depend on timestamp precision).
>>
>>> https://www.ietf.org/internet-drafts/draft-chen-ippm-coloring-based-
>>> i
>>> pfpm-fra
>>> mework-05.txt
>>>> Status:
>>> https://datatracker.ietf.org/doc/draft-chen-ippm-coloring-based-ipfp
>>> m
>>> -framew
>>> ork/
>>>> Htmlized:
>>> https://tools.ietf.org/html/draft-chen-ippm-coloring-based-ipfpm-fra
>>> m
>>> ework-0
>>> 5
>>>> Diff:
>>> https://www.ietf.org/rfcdiff?url2=draft-chen-ippm-coloring-based-ipf
>>> p
>>> m-frame
>>> work-05
>>>>
>>>> Abstract: