Subject: [ippm] Cost metrics in draft-wu-alto-te-metrics

From: Mirja Kühlewind <[email-masked]>
Date: Thu, 21 Jul 2016 15:41:57 +0200

Hi ippm folks, hi alto folks,

cross-posting because draft-wu-alto-et-metrics defines a set of alto cost metrics such as delay or bandwidth which sound to me like IP performance metrics. At the same time IPPM is currently in the process of defining a metric registry (draft-ietf-ippm-metric-registry-07 and draft-ietf-ippm-initial-registry-01). How do these relate to each other and how can we make sure that they are inline with each other?

Mirja

---

From: Qin Wu <[email-masked]>
Date: Thu, 21 Jul 2016 13:54:20 +0000

Thank Mirja to help raise this issue, for sure, we will align with what have been defined in draft-ietf-ippm-metric-registry-07 and draft-ietf-ippm-initial-registry-01.

-Qin
-----邮件原件-----
发件人: ippm [mailto:[email-masked]] 代表 Mirja Kühlewind
发送时间: 2016年7月21日 21:42
收件人: [email-masked]; [email-masked]
主题: [ippm] Cost metrics in draft-wu-alto-te-metrics

cross-posting because draft-wu-alto-etemetrics defines a set of alto cost metrics such as delay or bandwidth which sound to me like IP performance metrics. At the same time IPPM is currently in the process of defining a metric registry (draft-ietf-ippm-metric-registry-07 and draft-ietf-ippm-initial-registry-01). How do these relate to each other and how can we make sure that they are inline with each other?

Mirja
_______________________________________________
ippm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ippm

---

From: Gao Kai <[email-masked]>
Date: Thu, 21 Jul 2016 22:08:54 +0800

Hi Mirja and all,

Personally I think the metrics in draft-wu-alto-te-metrics can be
registered to IPPM following the process introduced in the IPPM drafts. 
If there are already some metrics with similar definitions,
draft-wu-alto-te-metrics can still define the corresponding
cost-metric/cost-mode values uesd in ALTO and have the registeredIPPM
metrics as references.

Regards,
Kai

On 21/07/16 21:41, Mirja Kühlewind wrote:
> alto mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/alto

---

From: Joachim Fabini <[email-masked]>
Date: Thu, 21 Jul 2016 16:46:23 +0200

Hi Mirja,

thanks for bringing this topic to ippm. Some technical notes on this:
RFC2330  does not differentiate at which layer timestamps are acquired.
If the host time is considered to be an application layer timestamp,
alto could immediately adopt/use the base framework and all the metrics
that have been defined so far in ippm - including OWD, RTD, IPDV (alias
Jitter), Loss, Loss Patterns, BTC, etc., all defined in separate
documents and in substantial detail.

So yes, I share your opinion. I recommend alto to consider this
procedure - unless there are specific reasons why this can't be done (in
which case ippm will likely be very interested in feedback on the reasons).

Some more related documents:
- The update to 2330 (RFC7312) has particular focus on uncertainty
factors that measurements at application level will encounter in today's
networks.

- Al and I have written a draft to update RFC2330 to be more specific
wrt timestamp acquisition, even considering virtualization and related
challenges. The draft
(https://datatracker.ietf.org/doc/draft-fabini-ippm-2330-time/ )  has
expired but I plan to rewrite it to fit what we have discussed in
Yokohama (in particular a use-case based discussion of timestamp
acquisition in measurements).

- IPv6 update for 2330 is on the way.
(https://tools.ietf.org/html/draft-ietf-ippm-2330-ipv6-00 )

Any feedback and summary on requirements from alto is very welcome;
likewise some statements on the challenges in adopting the existing ippm
metrics. I guess that this information is valuable feedback for ippm in
evaluating past and guiding future work.

Joachim

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Thu, 21 Jul 2016 11:13:42 -0400

Hi Mirja and Joachim,

Another way to interpret what ALTO needs is to look at 
cost metrics the same as link costs in routing.
Routing link costs are usually static and the values
might be set relative to some real metric like latency
or max link line rate. These involve no measurement at all.

So when they say:
   In this document, we proposes a set of Cost Metrics, derived and
   aggregated from routing protocols with different granularity and
   scope, such as BGP-LS,OSPF-TE and ISIS-TE, or from end to end traffic
   management tool.

only the last phrase implies measurement, with the long list of examples:

   We currently define 11 new Performance Metric to
   measure network delay, jitter, packet loss, hop count, and bandwidth.

None of these are new, and they are already defined in great detail
or ippm is in the process of doing so. The draft seems to assume that a 
measurement system exists to supply all the needed measurements with 
relevant context and scope.

I wonder how much all these "new" metrics actually improve alto cost
accuracy.

Al

> -----Original Message-----
> From: ippm [mailto:[email-masked]] On Behalf Of Joachim Fabini
> Sent: Thursday, July 21, 2016 10:46 AM
> To: Mirja Kühlewind; [email-masked]; [email-masked]
> Subject: Re: [ippm] Cost metrics in draft-wu-alto-te-metrics
> 
> Hi Mirja,
> 
> > cross-posting because draft-wu-alto-et-metrics defines a set of alto
> cost metrics such as delay or bandwidth which sound to me like IP
> performance metrics. At the same time IPPM is currently in the process
> of defining a metric registry (draft-ietf-ippm-metric-registry-07 and
> draft-ietf-ippm-initial-registry-01). How do these relate to each other
> and how can we make sure that they are inline with each other?
> >
> > Mirja

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Thu, 21 Jul 2016 11:23:03 -0400

Maybe I missed it, but the draft "names" many metrics,
it doesn't define them except in the most general way.
No two entities could build comparable measurement systems
that produce equivalent results with the specs in
the draft. This is one main purpose of the registry.

Al

From: ippm [mailto:[email-masked]] On Behalf Of Gao Kai

Hi Mirja and all,

Personally I think the metrics in draft-wu-alto-te-metrics can be registered to IPPM following the process introduced in the IPPM drafts.  If there are already some metrics with similar definitions, draft-wu-alto-te-metrics can still define the corresponding cost-metric/cost-mode values uesd in ALTO and have the registered IPPM metrics as references.

---

From: Qin Wu <[email-masked]>
Date: Thu, 21 Jul 2016 15:29:46 +0000

Thanks Al and Joachim to share the IPPM view. Please see my clarification inline.

-Qin
-----邮件原件-----
发件人: ippm [mailto:[email-masked]] 代表 MORTON, ALFRED C (AL)
发送时间: 2016年7月21日 23:14
收件人: [email-masked]; Mirja Kühlewind; [email-masked]; [email-masked]
主题: Re: [ippm] Cost metrics in draft-wu-alto-te-metrics

Hi Mirja and Joachim,

Another way to interpret what ALTO needs is to look at cost metrics the same as link costs in routing.
Routing link costs are usually static and the values might be set relative to some real metric like latency or max link line rate. These involve no measurement at all.

So when they say:

None of these are new, and they are already defined in great detail or ippm is in the process of doing so. The draft seems to assume that a measurement system exists to supply all the needed measurements with relevant context and scope.

[Qin]:Sure, that's what I underline in the ALTO session, we are not aiming at redefine the metrics. What we try to do in this draft is to derive and aggregate these metrics from various different data source ,e.g., management measurement tool.

I wonder how much all these "new" metrics actually improve alto cost accuracy.

[Qin]: not targeting to improve alto cost accuracy, but Allow applications to determine "where" to connect based on end to end network performance criteria, that's the main motivation to enrich single cost metric (routing cost) defined in ALTO protocol.
Please also refer to Alto Server Network API Use case proposed in RFC7752, section 2.2, this draft is targeting to address that use case and fill the gap.

Al

> alto could immediately adopt/use the base framework and all the 
> metrics that have been defined so far in ippm - including OWD, RTD, 
> IPDV (alias Jitter), Loss, Loss Patterns, BTC, etc., all defined in 
> separate documents and in substantial detail.
> 
> procedure - unless there are specific reasons why this can't be done 
> (in which case ippm will likely be very interested in feedback on the 
> reasons).
> 
> factors that measurements at application level will encounter in 
> today's networks.
> 
> likewise some statements on the challenges in adopting the existing 
> ippm metrics. I guess that this information is valuable feedback for 
> ippm in evaluating past and guiding future work.
> 
> Joachim
> 
> draft-ietf-ippm-initial-registry-01). How do these relate to each 
> other and how can we make sure that they are inline with each other?
> >
> > Mirja

---

From: Qin Wu <[email-masked]>
Date: Thu, 21 Jul 2016 15:44:36 +0000

Al, Your understanding is correct, this draft doesn’t intend to redefine these metrics. The definition of these metrics is not focus of this draft.

-Qin
发件人: alto [mailto:[email-masked]] 代表 MORTON, ALFRED C (AL)
发送时间: 2016年7月21日 23:23
收件人: Gao Kai; Mirja Kühlewind; [email-masked]; [email-masked]
主题: Re: [alto] [ippm] Cost metrics in draft-wu-alto-te-metrics

Maybe I missed it, but the draft “names” many metrics,
it doesn’t define them except in the most general way.
No two entities could build comparable measurement systems
that produce equivalent results with the specs in
the draft. This is one main purpose of the registry.

Al

Hi Mirja and all,

---

From: Leeyoung <[email-masked]>
Date: Thu, 21 Jul 2016 15:53:33 +0000

Hi,

I agree with Qin. There TE-metrics have been widely deployed and used in TE networks. The draft is simply exposing these metrics to the ALTO client.

Regards,
Young

From: alto [mailto:[email-masked]] On Behalf Of Qin Wu
Sent: Thursday, July 21, 2016 10:45 AM
To: MORTON, ALFRED C (AL); Gao Kai; Mirja Kühlewind; [email-masked]; [email-masked]
Subject: Re: [alto] [ippm] Cost metrics in draft-wu-alto-te-metrics

-Qin
发件人: alto [mailto:[email-masked]] 代表 MORTON, ALFRED C (AL)
发送时间: 2016年7月21日 23:23
收件人: Gao Kai; Mirja Kühlewind; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
主题: Re: [alto] [ippm] Cost metrics in draft-wu-alto-te-metrics

Al

Hi Mirja and all,