Subject: Re: [CCAMP] [mpls] ISIS TE Metric Extension Discussion (IETF-85 Followup)

From: "Mcdysan, David E" <[email-masked]>
Date: Thu, 21 Feb 2013 16:48:34 -0500

Some responses in line below related to draft-fuxh-mpls-delay-loss-te-problem-statement and -framework in line below. Cross posting to camp since the mols wg and the AD's decided in Atlanta that these drafts should be discussed in camp since they include multi-layer aspects.
Dave
From: Alia Atlas <[email-masked]<mailto:[email-masked]>>
Date: Thursday, February 21, 2013 3:56 PM
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Cc: Mpls <[email-masked]<mailto:[email-masked]>>, Nabil Bitar <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, Alia Atlas <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] ISIS TE Metric Extension Discussion (IETF-85 Followup)

Hi Curtis,

On Tue, Feb 19, 2013 at 7:56 PM, Curtis Villamizar <[email-masked]<mailto:[email-masked]>> wrote:

Alia,

How many different sets of ISIS and OSPF standards for advertising
delay and jitter should we be aiming for?

I think that the answer should be one.  Not two.  Not three or more.

In general, we should avoid duplication.  Where there is strong disagreement about the requirements and usage, or where there isn't sufficient maturity on all the aspects, then it makes sense to think about the problem.

There's also the difference between trying to make a cup of tea and boiling the ocean.

Finally, it is good to get things done - particularly when drafts are sitting for 1+ years without comments...

Therefore it would be beneficial to look at the set of requirements
and see if using one document as the base - fine with me if
draft-atlas-mpls-te-express-path is used as the base - and then see if
adding just a little to that covers CL requirements, and maybe adding
just a little more covers anything missed in
draft-fuxh-mpls-delay-loss-* that is deemed important.

I think that mpls-te-express-path and the igp drafts cover case 1 of Section 5.3 of draft-fuxh-mpls-delay-loss-te-problem-statement-01

1.  The case of few nodes with large geographic separation, (e.g.,trans-oceanic), where link delay alone would be a good approximation.

Would like to hear from others if the data center and mix of link/node delay for cases 2 and 3 is of strong interest to anyone.

draft-fuxh-mpls-delay-loss-te-framework documented our understanding of what requirements the express path igp and mpls drafts met. Suggest that the IGP and MPLS draft authors review that draft to see if anything in there is covered, or could be easily added/changed.

There are some  requirements in draft-fuxh-mpls-delay-loss which are concerning, such as the desire to
include node queuing delay and use that as a criteria for paths without addressing any of the stability concerns
that started this conversation.

In the -01 version we tried to state requirements that would address these stability concerns, for example in section 5.2, draft-fuxh-mpls-delay-loss-te-problem-statement-01

The solution SHOULD provide a means to dynamically measure and/or estimate the one-way link and node performance parameters for delay, loss and delay variation.
As defined in section 2.2. , the estimation interval for the performance parameters is assumed to be on the order of minutes. The solution MUST not impact stability nor significantly increase convergence time if performance parameters change over a timeframe on the order of minutes.

 I have NO interest in trying to meet that requirement  without someone doing the necessary
convincing studies, analysis & convincing.
In my opinion, the mandatory requirement is to allow configuration of the parameters, which would enable an operator to measure parameters and configure them that could be reported on in an analysis/study as stated in 5.2.

The solution MUST provide a means to configure the one-way link and node performance parameters for delay, loss and delay variation.

 Someone else can define a new sub-TLV to advertise node delays and related information.   In this case, IMHO, it's useful to separate out the fairly static link delay from the traffic-dependent queuing delay.
So - no duplication there.   It needs significant research and analysis to prove it's useful; I don't see a reason to wait on the draft-ietf-ospf-te-metric-extensions or draft-previdi-isis-te-metric-extensions for that.

Agreed, Section 5.3 of draft-fuxh-mpls-delay-loss-te-problem-statement-01 states that node level delay is a separate requirement that is desirable (SHOULD), not mandatory.

As for the RSVP-TE path related requirements, these drafts do go much much farther than draft-atlas-mpls-te-express-path does.   This is the ocean-boiling (perhaps pond) of handling inter-area, inter-as, and LSPs without explicit routes completely computed by the ingress.  Those aspects may be addressing very real use-cases - but that is much more extensive and complicated than a simple informational draft showing how the extended TE metrics can be useful today with minimal effort.

The delay related requirements in CL are spelled out in the CL
framework.  They are (from 7.1.  Brief Review of Requirements):

   delay and delay variation

       FR#7 (expose lower layer measured delay),
       FR#8 (precision of latency reporting),
       FR#9 (limit latency on per LSP basis),
       FR#15 (minimum delay path),
       FR#16 (bounded delay path),
       FR#17 (bounded jitter path)

FR#7 essentially asks for interworking with delay measured at a lower
layer.  Obviously this would *not* include queuing delay.  Maybe no
impact on draft-atlas-mpls-te-express-path since it does not specify
where to get the information.

Right, the intent is that the information could come from many places, and John Drake
can quote the RFC that is a good example.

FR#8 (in CL requirements) is:

   FR#8    The precision of latency reporting SHOULD be configurable.
           A reasonable default SHOULD be provided.  Implementations
           SHOULD support precision of at least 10% of the one way
           latencies for latency of 1 ms or more.

It seems that the remaining four are covered.  So meeting CL
requirements may be a NOOP except perhaps FR#8.

If you look at draft-ietf-ospf-te-metric-extensions, the delay can range from 1 micro-second to
16.777215 seconds.  Do we really need more dynamic range and associated complexity?

Certainly, if draft-ietf-ospf-te-metric-extensions and draft-previdi-isis-te-metric-extensions meet
the composite-link requirements, that is useful; it's always nice to solve multiple problems at the
same time.

Some critical questions which I don't see answered in any draft, are
1) whether queuing delay is measured, 2) if so, can measurements at
different PSC (direct mapping of TC bits to PHB for E-LSP) be handled,
3) if so, how are different PSC handled.  It would be nice if the
precision of the delay measurement was reported, for example reporting
a range for the link (ie: 10-12 msec, rounded to 1 msec).  If delay
was 9.8-11.5 for an interval and 10.2-12.2 the next, it need not be
readvertised but its nice to know through the reported precision that
it could be in the range of 9.5-12.499 and still count as 10-12.

(1),(2), and (3) It's not measuring queuing delay.  I'll work on improved text to clarify this.

Then there is the greater issue of measurement interval and
confidence.  Is the reported value the lowest reading over the
interval, highest, or some statistical norm?  Is it a mean value over
the interval? 98th percentile?  Is it necessary to communicate the
measurement interval?  For example, would a NPO be satisfied for delay
if the measurement was made over the previous hour and any changes
would be reflected in the next measurement an hour later?  My
suggestion was a range and precision.

The draft states that it should be an average.  Why would this information WANT to be
communicated?  It seems the type of thing that would be documented - perhaps
configurable on a router, but I can't see a reason for signaling it.

In my opinion, the mandatory requirement from fuxh-problem-statement in section 5.2 is to allow configuration of the parameters, which would enable an operator to change these based upon static or dynamic measurements.

The solution MUST provide a means to configure the one-way link and node performance parameters for delay, loss and delay variation.
I think that the IGP drafts support this.

 I can see that there might be an argument for using a 98th percentile vs. an average.

I don't think you can just punt on oscillations and stability by
saying for further study.

Why not?  What is being measured and reported does not change as a result of
traffic load.  It's based on characteristics of the link and not queuing.  Perhaps one should
put in some words about future study for stability for virtual links, whose measurements may
include queuing delay in them?  Perhaps we should should say that these values shouldn't
be advertised for such links?  Or have a way of measuring them that assumes maximum
queuing delay at each hop on the virtual link's path?

As for the draft-fuxh- set, I think the only requirement not met is
the delay difference between working and protect paths.  For that you
need something along the lines of a min delay and max delay.  Where
the working and protect are expected to each fall within a range, such
that this requirement is met.  Note that the mpls-tp-use-cases draft
has the same exaple.  The authors of the draft-fuxh- set can correct
me if I missed anything.
This is a fair summary. Note that the requirement is desirable (SHOULD).

Again, the draft-fuxh-set is trying to do much more complicated stuff for computation;
I can see those authors writing a suitable draft to explain how to compute LSPs
between a min delay and a max delay.

If you don't mind, I think we should take this thread to MPLS.

At the last IETF, the mpls wg and the AD's decided that the fuxh drafts should go to camp. I missed the deadline to submit a -00 draft to camp and they won't get discussed there. Would really like to get some feedback on what aspects are more important than others. I made a number of the requirements SHOULD/MAY since IMO they are of lower priority. There are still a number of Open MUST requirements. Cross posting this thread to camp for this reason.

 I
copied MPLS and bcc'd ISIS.  This way ISIS will get this message and
know the conversation moved, but the conversation continues on MPLS.
On the MPLS list I'll make more detailed comments on a new thread.

I've included ISIS back as well, because the concerns were raised there about
draft-previdi-isis-te-metric-extensions and the application and usage discussed
is important for ISIS to draw conclusions.

Alia

Curtis

In message <[email-masked]<mailto:[email-masked]>>
Alia Atlas writes:

Curtis,

I believe that we are emphatically saying that:

  draft-fuxh-mpls-delay-loss-te-problem-statement and/or
  draft-fuxh-mpls-delay-loss-te-framework

are UNCONNECTED

I would say not completely unconnected, the drafts mentioned below meet some of the requirements in   draft-fuxh-mpls-delay-loss-te-problem-statement-01 as documented in  draft-fuxh-mpls-delay-loss-te-framework draft.

to the drafts that we are discussing, which are:

   draft-atlas-mpls-te-express-path
   draft-ietf-ospf-te-metric-extensions
   draft-previdi-isis-te-metric-extensions

They are also not trying to solve the composite-link requirements.

I know that all of these talk about delay and jitter concerns, but the 3 drafts we're discussing are trying to narrow down what to send to a reasonable set that doesn't provoke stability concerns and can be used to solve the types of issues that Spencer described as in the mpls draft.

Alia

-----Original Message-----
From: Curtis Villamizar [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Tuesday, February 19, 2013 1:57 AM
To: Alia Atlas
Cc: Lou Berger; [email-masked]<mailto:[email-masked]>; John E Drake; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: ISIS TE Metric Extension Discussion (IETF-85 Followup)

In message <[email-masked]<mailto:[email-masked]>>
Alia Atlas writes:

> Should we add a reference to draft-atlas-mpls-te-express-path to help
> clarify that this is explicitly NOT delay-based IP routing?
>
> The Fuxh drafts were trying to go closer to that (and worry about
> queue-delay) last time I looked - certainly closer than these drafts
> are.
>
> Alia

Alia,

We should make it clear which express-path work is going forward and whether it has any relationship to draft-ietf-rtgwg-cl-requirements.

Right now we have:

  draft-atlas-mpls-te-express-path

  draft-giacalone-ospf-te-express-path
  (presumed abandonned)

  draft-fuxh-mpls-delay-loss-te-problem-statement and/or
  draft-fuxh-mpls-delay-loss-te-framework

The latter two were presented at the last IETF.

Would the authors of the set of documents above please combine them.
Would they also please consider explicitly addressing the requirments in draft-ietf-rtgwg-cl-requirements, such as stability, keeping the amount of churn down, supporting a grouping of CL component links.  If you do this, would you also please include me in the discussion.
 draft-fuxh-mpls-delay-loss-te-problem-statement-01 tried to explicitly state the CL requirements which overlap with it.

If I missed something, please identify and I can update -- if the community believes collecting these requirements (near and longer term, straightforward and controversial in one place) is of value, I would like to receive some affirmation before investing more time updating draft-fuxh-mpls-delay-loss-te-problem-statement and -framework.

IMO - the ospf and isis work should be halted until there is agreement on what problem is being solved, and the mechanisms to go about solving it.  Until then, we have protocol syntax and no semantics, and no assurance that the current syntax will match the semantics that is later decided is needed (by way of a framework document).

Curtis

ps - sorry to join the discussion on this list late, but I was out for a few days last week and over the long weekend (long in the US).

_______________________________________________
mpls mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mpls

---

From: [email-masked]
Date: Fri, 22 Feb 2013 00:50:46 -0500

------- Blind-Carbon-Copy

To: "Mcdysan, David E" <[email-masked]>
cc: Alia Atlas <[email-masked]>,
    "[email-masked]" <[email-masked]>,
    "[email-masked]" <[email-masked]>,
    "Bitar,
    Nabil N" <[email-masked]>,
    "[email-masked]" <[email-masked]>,
    "[email-masked]" <[email-masked]>,
    "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>,
    "[email-masked]" <[email-masked]>,
    "[email-masked]" <[email-masked]>,
    Alia Atlas <[email-masked]>,
    "[email-masked]" <[email-masked]>,
    "[email-masked]" <[email-masked]>
Reply-To: [email-masked]
From: Curtis Villamizar <[email-masked]>
Subject: Re: [mpls] ISIS TE Metric Extension Discussion (IETF-85 Followup)
In-reply-to: Your message of "Thu, 21 Feb 2013 16:48:34 EST."
             <[email-masked]>
Date: Fri, 22 Feb 2013 00:50:46 -0500

Dave,

It is extremely obnoxious to those who prefer to read IETF email as
plain text when someone indicates their comments in HTML attributes
such as color and font.  Your mailer
(Microsoft-MacOutlook/14.2.5.121010 according to the headers) did a
terrible job of providing a text alternative.

It is also somewhat obnoxious to cross post.  We now have three IETF
WGs on the Cc.  We need to consolidate to one.  Therefore my followup
will be on the MPLS mailing list with CCAMP and ISIS in the Bcc only,
so they get dropped from the thread but know why it happenned.

Please do not add more WGs to the Cc.

Also please note that I sent mail to the authors of
draft-fuxh-mpls-delay-loss-te-problem-statement and got no reply.

I tend to agree with Alia and John Drake that
draft-fuxh-mpls-delay-loss-* are not related and I do think those
drafts should not go forward at all at least in their current state.

In my prior email I suggested that the authors of the set of drafts
draft-atlas-mpls-te-express-path and draft-fuxh-mpls-delay-loss-* get
together and resolve differences.  If there is anything reasonable in
draft-fuxh-mpls-delay-loss-*, then it can be put into
draft-atlas-mpls-te-express-path.  If there is anything that needs to
be done in followup work, then any draft-fuxh-mpls-delay-loss-*
residuals can become a new followup draft covering topics such as
interlayer interworking or inter-domain extensions.

See inline.

Curtis

> Some responses in line below related to
> draft-fuxh-mpls-delay-loss-te-problem-statement and -framework in line
> below. Cross posting to camp since the mols wg and the AD's decided in
> Atlanta that these drafts should be discussed in camp since they
> include multi-layer aspects.
>  
> Dave
>  
> > From: Alia Atlas <[email-masked]>
> > To: "[email-masked]" <[email-masked]>
> > Cc: Mpls <[email-masked]>, Nabil Bitar
> >     <[email-masked]>, "[email-masked]"
> >     <[email-masked]>, "[email-masked]" <[email-masked]>,
> >     "[email-masked]" <[email-masked]>, "[email-masked]"
> >     <[email-masked]>, "[email-masked]" <[email-masked]>,
> >     "[email-masked]" <[email-masked]>,
> >     "[email-masked]" <[email-masked]>, Alia Atlas
> >     <[email-masked]>, "[email-masked]" <[email-masked]>,
> >     "[email-masked]"
> >     <[email-masked]>
> > On Tue, Feb 19, 2013 at 7:56 PM, Curtis Villamizar <[email-masked]> wrote:
> >>
> >>
> >> Alia,
> >>
> > In general, we should avoid duplication.  Where there is strong
> > disagreement about the requirements and usage, or where there isn't
> > sufficient maturity on all the aspects, then it makes sense to think
> > about the problem.
> >
> > Finally, it is good to get things done - particularly when drafts
> > are sitting for 1+ years without comments...
> >
> >>
> I think that mpls-te-express-path and the igp drafts cover case 1 of
> Section 5.3 of draft-fuxh-mpls-delay-loss-te-problem-statement-01
>  
> 1.  The case of few nodes with large geographic separation,
> (e.g.,trans-oceanic), where link delay alone would be a good
> approximation.

Node delay is a small contributor in any network where links are
greater than about 20 km and therefore link delays are greater than
100 usec.

> Would like to hear from others if the data center and mix of link/node
> delay for cases 2 and 3 is of strong interest to anyone.

It is mentioned in te-express-path that node delay (ie: fabric
contribution to delay) can be added to the link delay.  That would
cover the small metro (links < 20km) and data centers.

> draft-fuxh-mpls-delay-loss-te-framework documented our understanding
> of what requirements the express path igp and mpls drafts met. Suggest
> that the IGP and MPLS draft authors review that draft to see if
> anything in there is covered, or could be easily added/changed.

There is apparently some question among te-express-path authors
whether draft-fuxh-mpls-delay-loss-* has any relevance (and I tend to
agree).  Perhaps the authors of this set of draft can work this out
offline, but I'd like to be on the Cc.

> > include node queuing delay and use that as a criteria for paths
> > without addressing any of the stability concerns 
> In the -01 version we tried to state requirements that would address
> these stability concerns, for example in section 5.2,
> draft-fuxh-mpls-delay-loss-te-problem-statement-01
>  
> The solution SHOULD provide a means to dynamically measure and/or
> estimate the one-way link and node performance parameters for delay,
> loss and delay variation.
>  
> As defined in section 2.2. , the estimation interval for the
> performance parameters is assumed to be on the order of minutes. The
> solution MUST not impact stability nor significantly increase
> convergence time if performance parameters change over a timeframe on
> the order of minutes.

That is the slow slosh option we discussed on ISIS.  It is not a good
solution.  Excluding queuing delay is a better solution.  Providing a
range allows the delay TLV to remain unchanged as long as it moves
around within the advertised range (and if not, then the range can be
widenned).

> In my opinion, the mandatory requirement is to allow configuration of
> the parameters, which would enable an operator to measure parameters
> and configure them that could be reported on in an analysis/study as
> stated in 5.2.
>  
> The solution MUST provide a means to configure the one-way link and
> node performance parameters for delay, loss and delay variation.

You and Alia seem to be talking past each other.  Alia is defining
node delay to include queuing delay which is just plain wrong.  You
are referring to the fabric delay that is only applicable to
geographically small networks.  The problem Alia doesn't want to
tackle without research is the problem of queuing delay.

> >  Someone else can define a new sub-TLV to advertise node delays and
> > related information.   In this case, IMHO, it's useful to separate
> > out the fairly static link delay from the traffic-dependent queuing
> > delay.
> > So - no duplication there.   It needs significant research and
> > analysis to prove it's useful; I don't see a reason to wait on the
> > draft-ietf-ospf-te-metric-extensions or
> > draft-previdi-isis-te-metric-extensions for that.
>  
> Agreed, Section 5.3 of
> draft-fuxh-mpls-delay-loss-te-problem-statement-01 states that node
> level delay is a separate requirement that is desirable (SHOULD), not
> mandatory.

Its nice that you agree but if you are defining node delay differently
than Alia it is hard to understand what you are agreeing to.

> > As for the RSVP-TE path related requirements, these drafts do go
> > much much farther than draft-atlas-mpls-te-express-path does.   This
> > is the ocean-boiling (perhaps pond) of handling inter-area, inter-as,
> > and LSPs without explicit routes completely computed by the ingress.
> > Those aspects may be addressing very real use-cases - but that is
> > much more extensive and complicated than a simple informational draft
> > showing how the extended TE metrics can be useful today with minimal
> > effort.
> >
> >>
> > Certainly, if draft-ietf-ospf-te-metric-extensions and
> > draft-previdi-isis-te-metric-extensions meet 
> In my opinion, the mandatory requirement from fuxh-problem-statement
> in section 5.2 is to allow configuration of the parameters, which
> would enable an operator to change these based upon static or dynamic
> measurements.
>  
> The solution MUST provide a means to configure the one-way link and
> node performance parameters for delay, loss and delay variation.
>  

I think the "Anomalous State" mechanism is a bad idea.  We can discuss
that later on the MPLS WG mailing list only.

> This is a fair summary. Note that the requirement is desirable (SHOULD).

If you mean supporting the delay difference is a SHOULD, does that
mean that it is an option to not support it in the protocol.  If so,
then maybe te-express-path doesn't need to support it.

> At the last IETF, the mpls wg and the AD's decided that the fuxh
> drafts should go to camp. I missed the deadline to submit a -00 draft
> to camp and they won't get discussed there. Would really like to get
> some feedback on what aspects are more important than others. I made a
> number of the requirements SHOULD/MAY since IMO they are of lower
> priority. There are still a number of Open MUST requirements. Cross
> posting this thread to camp for this reason.

If it is decided that draft-fuxh-mpls-delay-loss-* are not related to
te-express-path, then draft-fuxh-mpls-delay-loss-* serves no purpose.

It might be better to combine and remove anything related to
interlayer interworking and inter-domain and put that in separate
drafts which extend draft-atlas-te-express-path.

> >> In message <[email-masked]>
> >> Alia Atlas writes:
> >>
> >> Curtis,
> >>
> I would say not completely unconnected, the drafts mentioned below
> meet some of the requirements in
> draft-fuxh-mpls-delay-loss-te-problem-statement-01 as documented in
> draft-fuxh-mpls-delay-loss-te-framework draft.

I think what Alia and John are trying to say is (sic)
draft-fuxh-mpls-delay-loss-* are trying to boil the ocean and
therefore are making themselves irrelevant in doing so.

> >> I know that all of these talk about delay and jitter concerns, but
> >> the 3 drafts we're discussing are trying to narrow down what to send
> >> to a reasonable set that doesn't provoke stability concerns and can
> >> be used to solve the types of issues that Spencer described as in
> >> the mpls draft.
> >>
> >> Alia
> >>
> >>
> >> From: Curtis Villamizar [mailto:[email-masked]]
> >> Cc: Lou Berger; [email-masked]; John E Drake;
> >>     [email-masked]; [email-masked];

------- End of Blind-Carbon-Copy