Subject: [ippm] Composition of Metrics

From: Maurizio Molina <[email-masked]>
Date: Wed, 01 Jun 2005 19:47:24 +0100

Dear all,
In RFC 2330 (Framework for IP Performance Metrics), the Spatial and
temporal composition of metrics is defined (see extract below).
However, in none of the RFCs defining the metrics the problem appears to
be tackled. Could somebody clarify me if there is a reason why this
piece of work has not been undertaken?
Thanks,
Maurizio

9.1. Spatial Composition of Metrics
.....
   When the definition of a metric includes a conjecture that the metric
   across the path is related to the metric across the subpaths of the
   path, that conjecture constitutes a claim that the metric exhibits
   spatial composition.  The definition should then include:
 +    the specific conjecture applied to the metric,
 +    a justification of the practical utility of the composition in
      terms of making accurate measurements of the metric on the path,
 +    a justification of the usefulness of the composition in terms of
      making analysis of the path using A-frame concepts more effective,
      and
 +    an analysis of how the conjecture could be incorrect.

9.2. Temporal Composition of Formal Models and Empirical Metrics

......

   When the definition of a metric includes a conjecture that the metric
   across the path at a given time T is related to the metric across the
   path for a set of other times, that conjecture constitutes a claim
   that the metric exhibits temporal composition.  The definition should
   then include:

 +    the specific conjecture applied to the metric,
 +    a justification of the practical utility of the composition in
      terms of making accurate measurements of the metric on the path,
      and
 +    a justification of the usefulness of the composition in terms of
      making analysis of the path using A-frame concepts more effective.

_______________________________________________
ippm mailing list
[email-masked] 
https://www1.ietf.org/mailman/listinfo/ippm

---

From: [email-masked]
Date: Thu, 9 Jun 2005 13:54:06 -0400

the composition of metrics is on everybody's mind ( a trivial statement)
and I guess the solution is elusive.
I have seen two broad approaches  or maybe  three
(1) Cannot be done and violates all known physical and mathematical laws.
(2) must be done and any reasonable method we agree on will do
(3) some approximation to the end to end performance from sub-segments
metrics  for practical purposes can be achieved using
addition ( latency), probability based( packet loss) aggregates that, while
inaccurate, from the SP  perspective will do.

I am somehow leaning towards the option (3). There is in my view an absence
of hard guidelines how to do the recombination
but I guess there is plenty of pragmatic approaches implemented or used.
And considering the nature of the problem
pragmatic appoach is the best we can do. To this point, maybe under  IPPM
WG we should develop the document
with requirements and a summary of pragmatic solutions from different SPs.
In my view such a document would certainly help all of us involved to focus
on some best practices or methods
roman k

                      "Maurizio Molina"                                                                                        
                      <maurizio.molina@da        To:       "STEPHAN Emile RD-CORE-LAN" <[email-masked]>,      
                      nte.org.uk>                 [email-masked]                                                                
                      Sent by:                   cc:       "Lei Liang" <[email-masked]>, "GN-JRA1-list"                  
                      [email-masked]         <[email-masked]>                                                      
                      rg                         Subject:  Re: [ippm] Composition of Metrics                                   

                      06/09/2005 12:21 PM                                                                                      

Hi Emile,

I compared your two drafts, and summarizing I would say (correct me if
I’m wrong) that:

1) Type-P-METRIC-XX-trajectory in the old draft corresponds to the
Type-P-METRIC-stream in the new one.

2) The new draft contains the one-to-group metrics

3) The asynchronous metrics were ruled out.

Regarding the last point, I understand that if you have a set of hosts
<H0, H1, …Hn> along a path and a set of delays <dT1, dT2, …dTn> obtained
by *different* probes (thus asynchronous...) is too naïve to say that
the delay’s sum is a good representation of the singleton one-way-delay
on the path. The path delay and its partitioning in the different
portions could of course be obtained if we had a trajectory metric,
which is a good thing but is not given by any currently widely deployed
measurement protocol.

So, it looks like the IPPM WG preferred just to prepare the container
for something that it is still to come and not to say anything about how
to compose existing metric. What I think would be useful is giving
guidelines about the composition of metric *statistics* (e.g. the mean
-> trivial - the variance or the percentiles -> not trivial because the
independence on the several path portions comes into play). I know that
testing such an independence hypothesis is difficult or impossible, is
it a reason for not saying anything about how to compose metric
statistics (which is something that operators would like to do)?.
Furthermore, in the fourth bullet point of RFC 2330 there was exactly
the statement that the specification of the composition would come along
with some reasoning about its correctness (see below).
Apologies if I'm raising a point that has already been discussed and
buried in te WG. In this case, I'd appreciate pointers to mail archives,
etc.
Regards,
Maurizio

STEPHAN Emile RD-CORE-LAN wrote:

>Dear Maurizio
>
>2 years ago I tried to introduce this point in the IPPM WG, see
http://www.watersprings.org/pub/id/draft-stephan-ippm-spatial-metrics-01.txt
. This ID defines metrics for spatial composition and spatial
decomposition. This point was presented and discussed in the IPPM WG.
>My understanding of the IPPM WG conclusion is that it is quite tricky to
define composition metrics respectful of the RFC2330 because it is possible
to prove that such composition is not deterministic.
>
>Consequently Lei and I are focusing on the definition of spatial metrics
to decompose end-to-end measures and one-to-group metrics in
http://www.ietf.org/internet-drafts/draft-stephan-ippm-multimetrics-00.txt.

>
>
>Regards
>Emile
>
>
>
>>-----Message d'origine-----
>>De : [email-masked] [mailto:[email-masked]] De la part de
>>Maurizio Molina
>>Envoyé : mercredi 1 juin 2005 20:47
>>À : [email-masked]
>>Objet : [ippm] Composition of Metrics
>>
>>Dear all,

---

From: "STEPHAN Emile RD-CORE-LAN" <[email-masked]>
Date: Fri, 10 Jun 2005 14:56:44 +0200

Dear Roman,

In the draft http://www.ietf.org/internet-drafts/draft-stephan-ippm-multimetrics-00.txt we define the decomposition of an end to end metrics. It satisfies the point(3) because the result of a decomposition over one path give the performance of a sub-segment and consequently may be used to compute an approximation of the end to end performance of another path. 

So I agree with your proposal. Nevertheless the main issue of composition will be to define what is accurate enough to be usable.

Regards
Emile

> De : [email-masked] [mailto:[email-masked]]
> Envoyé : jeudi 9 juin 2005 19:54
> À : Maurizio Molina
> Cc : STEPHAN Emile RD-CORE-LAN; GN-JRA1-list; [email-masked]; ippm-
> [email-masked]; Lei Liang
> Objet : Re: [ippm] Composition of Metrics
> 
> 
> 
> 
> 
>                       <maurizio.molina@da        To:       "STEPHAN Emile
> RD-CORE-LAN" <[email-masked]>,
> I'm wrong) that:
> <H0, H1, ...Hn> along a path and a set of delays <dT1, dT2, ...dTn> obtained
> the delay's sum is a good representation of the singleton one-way-delay

---

From: "Fardid, Reza" <[email-masked]>
Date: Fri, 10 Jun 2005 11:19:33 -0700

Dear All,

SPs already cope with measurement inaccuracies, in particular those attributed to sampling and coverage. 

This is useful work to undertake, even if it creates another source of inaccuracy. Tolerance to inaccuracy depends on a number of interrelated factors, from an SP perspective, which appear to be out of scope of IPPM, if I am not mistaken:

1) Use of measurements in network design, SLA management, troubleshooting;
2) Type of application;
3) Network administrative boundaries, when different segments in the end-to-end path are controlled by different providers.

It will be good to define the sources of (de)composition inaccuracies for each measurement and leave it up to SPs to decide how best to cope with them.

Regards,
Reza Fardid

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of STEPHAN Emile RD-CORE-LAN
Sent: Friday, June 10, 2005 5:57 AM
To: [email-masked]
Cc: Lei Liang; [email-masked]
Subject: RE: [ippm] Composition of Metrics

Dear Roman,

Regards
Emile

---

From: "STEPHAN Emile RD-CORE-LAN" <[email-masked]>
Date: Tue, 14 Jun 2005 14:00:23 +0200

Hi all

I will contribute too.

Regards
Emile

> Envoyé : mardi 14 juin 2005 11:34
> À : [email-masked]
> [email-masked] wrote:
> 
> >
> >
> >Henk
> >I would be interested in working on the document... as a part of a team
> >roman krzanowski
> >
> >
> So would I...
> Maurizio Molina
> 
> >
> >
> >                      "Henk Uijterwaal"
> >                      <[email-masked]>          To:       "Al Morton"
> <[email-masked]>, [email-masked], [email-masked]
> >                      Sent by:                 cc:       "Matthew J
> Zekauskas" <[email-masked]>
> >                      ippm-bounces@ietf        Subject:  Re: [ippm]
> Composition of Metrics
> >                      .org
> >
> >
> >                      06/13/2005 05:43
> >                      AM
> >
> >
> >
> >
> >
> >
> >
> >Speaking as chair...
> >
> >
> >
> >>We've heard interest from several people so far...
> >>
> >>
> >
> >Yes, there seems to be interest, so I think we should see if there are
> >sufficient volunteers to work on this.  We should also define the scope
> >of this work.   So, if you are interested and can spend time on this,
> >drop me a note.  I'll also put it on the agenda for Paris,
> >
> >Henk
> >
> >(who has been extremely busy and is neglecting IPPM a bit).
> >
> >
> >-------------------------------------------------------------------------
> -----
> >
> >Henk Uijterwaal                           Email:
> >henk.uijterwaal(at)ripe.net
> >RIPE Network Coordination Centre
> http://www.amsterdamned.org/~henk
> >P.O.Box 10096          Singel 258         Phone: +31.20.5354414
> >1001 EB Amsterdam      1016 AB Amsterdam  Fax: +31.20.5354445
> >The Netherlands        The Netherlands    Mobile: +31.6.55861746
> >-------------------------------------------------------------------------
> -----
> >
> >
> >Look here junior, don't you be so happy.
> >And for Heaven's sake, don't you be so sad.                 (Tom Verlaine)
> >
> >
> --
> ______________________________________________________________________
> 
> Maurizio Molina
> Network Engineer
> 
> DANTE - www.dante.net
> 
> Tel: +44 (0)1223 371 300
> Fax: +44 (0)1223 371 371
> Email: [email-masked]
> PGP Key ID: 3FF58D51
> 
> City House, 126-130 Hills Road
> Cambridge CB2 1PQ
> UK
> _____________________________________________________________________
> 
> 
>

---

From: "Fardid, Reza" <[email-masked]>
Date: Tue, 14 Jun 2005 09:13:27 -0700

Henk,

I will be interested in contributing, too.

Regards,
Reza Fardid

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Henk Uijterwaal
Sent: Monday, June 13, 2005 2:44 AM
To: Al Morton; [email-masked]; [email-masked]
Cc: Matthew J Zekauskas
Subject: Re: [ippm] Composition of Metrics

Speaking as chair...

Henk

------------------------------------------------------------------------
------
Henk Uijterwaal                           Email:
------------------------------------------------------------------------
------

Look here junior, don't you be so happy.
And for Heaven's sake, don't you be so sad.                 (Tom
Verlaine)