Subject: Comments on draft-zhang-bfd-new-use-cases

From: "Nobo Akiya (nobo)" <[email-masked]>
Date: Wed, 9 Jul 2014 00:42:16 +0000

[chair hat off]

Hi Authors,

Interesting use cases. Please see below for some questions/comments.

> 3.1. Use Case #1: Reliable Detection for Multipath

In Figure 3.1, do R1-R2-R3-R5 and R1-R4-R5 have same cost?

If Figure 3.1, if there was a link between R4 and R3, and all possible paths between R1 and R5 were same cost, then how many BFD sessions do you want at R1 and to which node?

> 3.2. Use Case #2: Application Consistence

I tend to think that the system should simply take the most aggressive detection time out of all the applications for the same target. Are there any applications that can get negatively impacted by BFD running more aggressive than requested detection time?

> 3.3. Use Case #3: Capability Inquiry and Feedback
> 
>    If the local system restarts, it may resume the BFD session. Suppose
>    the link has been failed or the peer has no resources to create the
>    BFD session or the peer had been taken down administratively during
>    the absence of the local system.

By "system restarts", do you mean a BFD module has gone through a graceful restart, or a system hosting the BFD module has gone through a graceful reload (w/o forwarding impact)? And by "it may resume the BFD session", doesn't that imply that BFD sessions exist locally as well as on the peer? If that is true, then I'm not clear on what you mean by " or the peer has no resources to create the BFD session".

Additionally, RFC5880 states:

>    BFD Control packets
>    MAY be transmitted indefinitely after transitioning to AdminDown
>    state in order to maintain session state in each system (see section
>    6.8.18 below).

So if the BFD session on the peer went AdminDown during the absence of local BFD module, and if peer BFD session continued to send AdminDown, then the last case of "or the peer had been taken down administratively during the absence of the local system" is no longer an issue, since local (resumed) BFD session will receive AdminDown packet?

If the two cases are handled, then that only leaves "the link has been failed", which is no longer ambiguous and failing the local session is the right thing to do. Just some thoughts. Perhaps you can further elaborate the exact requirement/use-case on this one.

> 3.4. Use Case #4: State Relay

Sounds like a similar concept to the BFD proxy described in this document.

http://datatracker.ietf.org/doc/draft-snyder-bfd-proxy-connections-monitored-links/

Do look through the document to see if the concept is similar.

Question is, how do we provision the proxy BFD that is tied to specific "remote" session.

> 3.5. Use Case #5: Detection of Asymmetric LSPs

Controlling the reverse path for BFD control packets, like:

http://tools.ietf.org/html/rfc7110
http://datatracker.ietf.org/doc/draft-mirsky-mpls-bfd-directed/

makes sense. But I'm not convinced if similar is required for BFD echo packets. If you really require this, then you probably want a mechanism to discover the local label on the egress for the reverse LSP so that BFD echo packets can be sent with label stack [forward label, egress local label for reverse LSP], so cause the packets to u-turn. That mechanism will not be anything BFD specific.

Looking forward for discussions/clarifications on the use cases, and also your presentation at IETF90/Toronto.

Thanks!

-Nobo

---

From: Mahesh Jethanandani <[email-masked]>
Date: Thu, 10 Jul 2014 06:31:50 -0700

Agree with Nobo, that these are interesting use cases to document.

One comment/question that I have is ...

Would it make more sense to run BFD on each of the possible paths and report its individual path status to a higher level protocol? So in Figure 3.2, would it make more sense to run 3 BFD sessions between R1 and R4 one on each of the links, each of them reporting their status to its listeners, be it OSPF ECMP, LAG, or MPLS link bundles and letting them decide what is a failure scenario?

> Looking forward for discussions/clarifications on the use cases, and also your presentation at IETF90/Toronto.

Mahesh Jethanandani
[email-masked]

---

From: Mingui Zhang <[email-masked]>
Date: Fri, 11 Jul 2014 09:35:04 +0000

Hi Mahesh, Nobo,

Thanks for your comments. Please see my response inline. 

Regards,
Mingui

>-----Original Message-----
>From: Rtg-bfd [mailto:[email-masked]] On Behalf Of Mahesh
>Jethanandani
>Sent: Thursday, July 10, 2014 9:32 PM
>To: Nobo Akiya
>Cc: [email-masked]; [email-masked]
>Subject: Re: Comments on draft-zhang-bfd-new-use-cases
>
>Would it make more sense to run BFD on each of the possible paths and report
>its individual path status to a higher level protocol? So in Figure 3.2, would it
>make more sense to run 3 BFD sessions between R1 and R4 one on each of the
>links, each of them reporting their status to its listeners, be it OSPF ECMP, LAG,
>or MPLS link bundles and letting them decide what is a failure scenario?

[Mingui] That's a potential method. Let me interpret it a bit further. Each possible forwarding path gets a BFD session. The application need summarize all the state of these sessions to get the overall health status of the multipath. 

The difficulty is that two end systems need probe and maintain all possible forwarding paths. For example, a failure of link in the middle of the chains may affect more than one forwarding paths. Hence, the two end systems need re-establish those affected BFD sessions. Also, considering the random hashing that may be adopted (look at the two links between R1 and R4 in Figure 3.2), the probing of those forwarding paths might be inaccurate.

>From the angle of the carriers, unless all possible forwarding paths are failed, they wish the BFD does not report a failure. That's why we call it "reliable".
With this bottom line requirement, we may also figure out a solution works in another way: two end points _dynamically_ pick one of the remained active interfaces to deliver the BFD control packets. There is always a single BFD session per multipath.

>> If Figure 3.1, if there was a link between R4 and R3, and all possible paths
>between R1 and R5 were same cost, then how many BFD sessions do you want
>at R1 and to which node?

[Mingui] As I stated above, we may want to establish only one session.

>> I tend to think that the system should simply take the most aggressive
>detection time out of all the applications for the same target. Are there any
>applications that can get negatively impacted by BFD running more aggressive
>than requested detection time?

[Mingui] The application may not want that aggressive detection due to the stability. Think about the dampening measures towards the state oscillation of network devices.

>> By "system restarts", do you mean a BFD module has gone through a graceful
>restart, or a system hosting the BFD module has gone through a graceful reload
>(w/o forwarding impact)? And by "it may resume the BFD session", doesn't that
>imply that BFD sessions exist locally as well as on the peer? If that is true, then
>I'm not clear on what you mean by " or the peer has no resources to create the
>BFD session".
>>
>> So if the BFD session on the peer went AdminDown during the absence of local
>BFD module, and if peer BFD session continued to send AdminDown, then the
>last case of "or the peer had been taken down administratively during the
>absence of the local system" is no longer an issue, since local (resumed) BFD
>session will receive AdminDown packet?

[Mingui] It sounds the inquiry is still needed if the BFD control packets are not indefinitely sent.

>>
>> If the two cases are handled, then that only leaves "the link has been failed",
>which is no longer ambiguous and failing the local session is the right thing to do.
>Just some thoughts. Perhaps you can further elaborate the exact
>requirement/use-case on this one.
>>
>> makes sense. But I'm not convinced if similar is required for BFD echo packets.
>If you really require this, then you probably want a mechanism to discover the
>local label on the egress for the reverse LSP so that BFD echo packets can be
>sent with label stack [forward label, egress local label for reverse LSP], so cause
>the packets to u-turn. That mechanism will not be anything BFD specific.

[Mingui] I agree this kind of requirements can be achieved through changing applications rather than changing the BFD itself.

---

From: "Nagendra Kumar Nainar (naikumar)" <[email-masked]>
Date: Fri, 11 Jul 2014 20:42:00 +0000

Hi Mingui,

Interesting document. Most of the queries that popped are already raised
by Nobo and Mahesh. So I am using the same thread.

For clarity, I think you should change the figure with cost mentioned so
that the path really looks like ECMP. Currently it appears that the best
path from R1 to R5 is R1-R4-R5.

Regarding Use case #3, I am not sure I understood the use case.

"If the local system restarts, it may resume the BFD session. Suppose
   the link has been failed or the peer has no resources to create the
   BFD session or the peer had been taken down administratively during
   the absence of the local system. Since no BFD control packets will be
   received from the peer system, the BFD will report a Down state.
   Rather, the real state of the forwarding path can either be Down or
   Up.²

Rom the above - ³Suppose the link has been failed", I believe you meant
the link between R1 and R2 failed?. If so, will not the forwarding path is
also affected?. Further regarding the no resource scenario - Per my
understanding, the control plane protocol packets (like hello) is
sufficient to bring the session up even when the BFD didn¹t come up
right?. IOW, I believe, BFD state change from up to down will intimate the
client to bring the control plane session down immediately, but I think it
will not hold the protocol to change the state to up based on its own
hello messages.

Thanks,
Nagendra

On 7/11/14, 5:35 AM, "Mingui Zhang" <[email-masked]> wrote:

>[Mingui] That's a potential method. Let me interpret it a bit further.
>Each possible forwarding path gets a BFD session. The application need
>summarize all the state of these sessions to get the overall health
>status of the multipath.
>
>The difficulty is that two end systems need probe and maintain all
>possible forwarding paths. For example, a failure of link in the middle
>of the chains may affect more than one forwarding paths. Hence, the two
>end systems need re-establish those affected BFD sessions. Also,
>considering the random hashing that may be adopted (look at the two links
>between R1 and R4 in Figure 3.2), the probing of those forwarding paths
>might be inaccurate.
>
>>From the angle of the carriers, unless all possible forwarding paths are
>>failed, they wish the BFD does not report a failure. That's why we call
>>it "reliable".
>With this bottom line requirement, we may also figure out a solution
>works in another way: two end points _dynamically_ pick one of the
>remained active interfaces to deliver the BFD control packets. There is
>always a single BFD session per multipath.
>
>>
>[Mingui] The application may not want that aggressive detection due to
>the stability. Think about the dampening measures towards the state
>oscillation of network devices.
>
>>>
>>http://datatracker.ietf.org/doc/draft-snyder-bfd-proxy-connections-monito
>>red-li
>>nks/
>>>
>[Mingui] I agree this kind of requirements can be achieved through
>changing applications rather than changing the BFD itself.
>
>>>
>>> Looking forward for discussions/clarifications on the use cases, and
>>>also your
>>presentation at IETF90/Toronto.
>>>
>>> Thanks!
>>>
>>> -Nobo
>>>
>>>
>>
>>Mahesh Jethanandani
>>[email-masked]
>>
>>
>

---

From: "Nobo Akiya (nobo)" <[email-masked]>
Date: Fri, 11 Jul 2014 23:24:21 +0000

Hi Mingui,

> From: Mingui Zhang [mailto:[email-masked]]
> Sent: Friday, July 11, 2014 5:35 AM
> To: Mahesh Jethanandani; Nobo Akiya (nobo)
> Subject: RE: Comments on draft-zhang-bfd-new-use-cases
> 
> Hi Mahesh, Nobo,
> 
> >Would it make more sense to run BFD on each of the possible paths and
> >report its individual path status to a higher level protocol? So in
> >Figure 3.2, would it make more sense to run 3 BFD sessions between R1
> >and R4 one on each of the links, each of them reporting their status to
> >its listeners, be it OSPF ECMP, LAG, or MPLS link bundles and letting them
> decide what is a failure scenario?
> 
> [Mingui] That's a potential method. Let me interpret it a bit further. Each
> possible forwarding path gets a BFD session. The application need
> summarize all the state of these sessions to get the overall health status of
> the multipath.
> 
> The difficulty is that two end systems need probe and maintain all possible
> forwarding paths. For example, a failure of link in the middle of the chains
> may affect more than one forwarding paths. Hence, the two end systems
> need re-establish those affected BFD sessions. Also, considering the
> random hashing that may be adopted (look at the two links between R1 and
> R4 in Figure 3.2), the probing of those forwarding paths might be inaccurate.

If the requirement for IP or MPLS or something else?
It might be good to clarify that in this use case, as available OAM and techniques differ quite a bit.

> failed, they wish the BFD does not report a failure. That's why we call it
> "reliable".

Well, that's a preference. You are black holing on the one failed path, and I would not like that if I was a customer :)

It's probably important to keep an open mind about how aggregation of state from related sessions produce the final result.

> With this bottom line requirement, we may also figure out a solution works
> in another way: two end points _dynamically_ pick one of the remained
> active interfaces to deliver the BFD control packets. There is always a single
> BFD session per multipath.
> 
> >
> >between R1 and R5 were same cost, then how many BFD sessions do you
> >want at R1 and to which node?
> 

Let's further clarify the use case first.

> >detection time out of all the applications for the same target. Are
> >there any applications that can get negatively impacted by BFD running
> >more aggressive than requested detection time?
> 
> [Mingui] The application may not want that aggressive detection due to the
> stability. Think about the dampening measures towards the state oscillation
> of network devices.

Dampening often controls DOWN->UP sequence, not UP->DOWN.
So you'll have to come up with another reason why this use case is valid :)

> >>>  If the local system restarts, it may resume the BFD session.
> >>> Suppose  the link has been failed or the peer has no resources to
> >>> create the  BFD session or the peer had been taken down
> >>> administratively during  the absence of the local system.
> >>
> >restart, or a system hosting the BFD module has gone through a graceful
> >reload (w/o forwarding impact)? And by "it may resume the BFD session",
> >doesn't that imply that BFD sessions exist locally as well as on the
> >peer? If that is true, then I'm not clear on what you mean by " or the
> >peer has no resources to create the BFD session".
> >>
> >BFD module, and if peer BFD session continued to send AdminDown, then
> >the last case of "or the peer had been taken down administratively
> >during the absence of the local system" is no longer an issue, since
> >local (resumed) BFD session will receive AdminDown packet?
> 

So remote session having no resources is not an issue, and it's only remote session being in AdminDown whilst absence of local session? Maybe you can write up the sequence of what's happening in local/remote, so that we are on the same page about the problem. -> Toronto presentation? :)

> >http://datatracker.ietf.org/doc/draft-snyder-bfd-proxy-connections-moni
> >tored-li
> >nks/
> >>
> >> Question is, how do we provision the proxy BFD that is tied to
> >> specific
> >"remote" session.
> >>
> >If you really require this, then you probably want a mechanism to
> >discover the local label on the egress for the reverse LSP so that BFD
> >echo packets can be sent with label stack [forward label, egress local
> >label for reverse LSP], so cause the packets to u-turn. That mechanism will
> not be anything BFD specific.
> 
> [Mingui] I agree this kind of requirements can be achieved through changing
> applications rather than changing the BFD itself.

Ok.

Thanks!

-Nobo

---

From: Mingui Zhang <[email-masked]>
Date: Mon, 14 Jul 2014 08:59:57 +0000

Hi Nobo,

<snip>
>Well, that's a preference. You are black holing on the one failed path, and I
>would not like that if I was a customer :)

[Mingui] I think I should say "Detection for Reliable Multipath" rather than "Reliable Detection for Multipath". :-)

>
>It's probably important to keep an open mind about how aggregation of state
>from related sessions produce the final result.

[Mingui] I agree. I think the logic is to have BFD per any possible forwarding path for the multipath. If there is any UP BFD session, the multipath is UP.

<snip>
>It might be good to clarify that in this use case, as available OAM and techniques

[Mingui] Yes, I agree. The multipath can be the multipath per [RFC7226] or a layer3 multi-hop multipath with lower layer TRILL links on the path (e.g., the link between R1 and R4 in Figure 3.2). Those are the original scenarios that cause us to bring forward use case #1.

<snip>
>So you'll have to come up with another reason why this use case is valid :)

[Mingui] Yes, dampening control the DOWN->UP sequence. Now, consider no extra dampening measure is taken. Rather, the application simply expects to dampen oscillation using a large desired detection time. 

The dampening use case is just used to explain that the application subscribes the BFD and expect it acts as subscribed. It may not need surprise:). So we require a method to negotiate the detection requirements.

We have another use case at hand for this requirement. You can determine whether it is relevant. In the following figure, suppose two different MC-LAG links between CE and PEs share the same heart-beat link between PE1 and PE2. LAG1 creates bfd1 at PE1 but not at PE2. LAG2 creates bfd2 at PE2 but not PE1. This is a mis-configuration. But we will find the BFD session can be set up with bfd1 talking with bfd2. 

    PE1
    /|
   / |
  /  |
CE   |
  \  |
   \ |
    \|
    PE2

<snip>
>So remote session having no resources is not an issue, and it's only remote
>session being in AdminDown whilst absence of local session? Maybe you can
>write up the sequence of what's happening in local/remote, so that we are on
>the same page about the problem. -> Toronto presentation? :)

[Mingui] Well. The "having no resources" is still an issue. Let me introduce the following scenario. 
Suppose link (if1--if2) is the primary while link (if3--if4) is the backup. The line card attached to if2 is pulled out. So two endpoints resort to link (if3--if4). Suppose this link is working before the switching but if4 has too much BFD sessions running on it, so the processor of the NP does not have resources to create any more BFD session. The application wishes to use link (if3--if4), but the BFD keeps holding a DOWN state for the session. 

The sequence of the "AdminDown" scenario is like this: the local system fails. The remote system takes down the session administratively during its failure and then ceases to transmit BFD control packets. Afterwards, the local system reboots and tries to recreate the BFD session. However, the BFD will keep holding a DOWN state for the session even the forwarding path is UP.

Note that the restart here is not graceful restart. It's a local system reboot. I want to rectify one word in the text: resume->re-create.

I will be in Toronto. See you there.

Thanks,
Mingui

>From: Nobo Akiya (nobo) [mailto:[email-masked]]
>Sent: Saturday, July 12, 2014 7:24 AM
>To: Mingui Zhang; Mahesh Jethanandani
>Well, that's a preference. You are black holing on the one failed path, and I
>would not like that if I was a customer :)
>
>It's probably important to keep an open mind about how aggregation of state
>from related sessions produce the final result.
>
>So remote session having no resources is not an issue, and it's only remote
>session being in AdminDown whilst absence of local session? Maybe you can
>write up the sequence of what's happening in local/remote, so that we are on
>the same page about the problem. -> Toronto presentation? :)
>
>>
>> >>

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 18 Jul 2014 12:16:55 -0400

> I tend to think that the system should simply take the most aggressive detection time out of all the applications for the same target. Are there any applications that can get negatively impacted by…

Most deployed applications that I'm aware of tend to piggy-back multiple
users of BFD on the timing requirements of the most aggressive timers.
Nothing stops multiple sessions from being used but of course that impacts
session scaling.

One point that is interesting here can be represented by the following
contrived example:

A---B

A and B have OSPF and BGP between them.  OSPF and BGP are configured to use
BFD.

OSPF's failure requirements are set to 100ms timers with a 3x detection
interval.

BGP's failure requirements are set to 500ms timers with a 3x detection
interval.

If both BGP and OSPF rely on OSPF's timers, the OSPF session may drop but
may take BGP along with it, despite the fact that BGP is more tolerant.

FWIW, I don't see any need to update specs against such a detail.
Implementations are free to require separate sessions.  The interesting use
case is should we permit some way for detections to "fall back" to less
granular timings in a pre-negotiated way?

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 18 Jul 2014 12:20:48 -0400

> Would it make more sense to run BFD on each of the possible paths and report its individual path status to a higher level protocol? So in Figure 3.2, would it make more sense to run 3 BFD sessions…

One of the big challenges in the multipath or LAG scenarios is detecting the
various components of the link and exercising the forwarding.  In the case
of LAG, we went with the simplest scenario (back to back connected links)
and expose that information only to the layer managing the LAG.  If you want
an application BFD session, that is an additional session.

Nothing stops the application sessions from being able to make use of that
BFD component member state, but most views of the desired behavior is more
of an end-to end exercise, much like LSPing.  And that's where we run into
frustrations.

>From an end to end, we can't ensure that a LAG will permit forwarding to be
controlled over the underlying layer 3 load balancer.  

Similarly, we have no way to influence non-LAG multipath link hashing.

Obviously this changes when we talk about LSPs and things like entropy
labels.

-- Jeff

---

From: Mahesh Jethanandani <[email-masked]>
Date: Fri, 18 Jul 2014 11:26:41 -0700

<snip>

Agreed. 

The general deployment vision I have is that faster BFD sessions e.g. 3.3ms are deployed on the member links of the multipath, and a slower e.g. 100ms session deployed end-to-end. The faster sessions detect and correct any errors on the multipaths before an end-to-end session can detect a fault.

Mahesh Jethanandani
[email-masked]

---

From: "Nobo Akiya (nobo)" <[email-masked]>
Date: Fri, 18 Jul 2014 23:38:48 +0000

> From: Mahesh Jethanandani [mailto:[email-masked]]
> Sent: Friday, July 18, 2014 2:27 PM
> To: Jeffrey Haas
> Cc: Nobo Akiya (nobo); [email-masked]; rtg-
> [email-masked]
> report its individual path status to a higher level protocol? So in Figure 3.2,
> would it make more sense to run 3 BFD sessions between R1 and R4 one on
> each of the links, each of them reporting their status to its listeners, be it
> OSPF ECMP, LAG, or MPLS link bundles and letting them decide what is a
> failure scenario?
> >
> > One of the big challenges in the multipath or LAG scenarios is
> > detecting the various components of the link and exercising the
> > forwarding.  In the case of LAG, we went with the simplest scenario
> > (back to back connected links) and expose that information only to the
> > layer managing the LAG.  If you want an application BFD session, that is an
> additional session.
> <snip>
> 
> Agreed.
> 
> The general deployment vision I have is that faster BFD sessions e.g. 3.3ms
> are deployed on the member links of the multipath, and a slower e.g. 100ms
> session deployed end-to-end. 

Concept of running "lower" sessions faster than "higher" sessions is fine, this works (ex: single-hop BFD and multi-hop BFD). But you are now introducing another layer (i.e. BFD on LAG, single-hop BFD, multipath BFD, multi-hop BFD) which each need to detect & converge before next "higher" sessions time out. And that's just the underlay. There will be more sessions running on the overlays. This BFD layering and ensuring they all run at appropriate rate, manually or automatically, may itself be an interesting use-case. Something to think about  :)

Additionally, if you ran BFD sessions per member link of the multipath at the ECMP branch nodes, then you do eliminate the load balancing issue (via tying the send/receive of the session to a particular interface). However, you may be introducing further problems which are likely equally complex.

- BFD sessions need to be per path (i.e. prefix/LSP) to be effective (i.e. link health monitoring is done via single-hop BFD already).
- Due to per path needs, BFD sessions will likely require TTL tweaking (i.e. BFD termination point is no longer the path target). 
- BFD sessions will need to be tied to specific interfaces, but need to survive FRR/LFA.
- Packet paths are different between locally generated/terminated packets vs. packets forwarding through the node (i.e. multipath BFD sessions being up does not necessary mean end-to-end path is functioning properly over all multipath).

Something to keep in mind :)

Because of above, end-to-end monitoring is still seen to be attractive by some, and some OAM folks are excited about the introduction of Entropy Labels (as Jeff also wrote in his reply). That allows _a_ solution for MPLS data plane, but AFAIK, there is _no_ solution for IP data plane.

Lastly ...

> The faster sessions detect and correct any
> errors on the multipaths before an end-to-end session can detect a fault.

It will be beneficial for authors of draft-zhang-bfd-new-use-cases to clearly describe the "errors" that needs to be detected to get us on the same page, as this is a use-case document. I know many of us are very technical and love talking about solutions, but we can talk about solutions once there's consensus on the use-cases/requirements.

Thanks!

-Nobo

---

From: Mingui Zhang <[email-masked]>
Date: Mon, 21 Jul 2014 02:08:02 +0000

Hi Nobo, Mahesh & Jeff,

Thanks for the deep discussion on use case #1.

>- BFD sessions will need to be tied to specific interfaces,

I think the above point proposed by Nobo is highly relevant to our use case. Since BFD session is tied to a specific pair of interfaces, when one of the interface card is pulled out, this session will fail and a failure will be reported to the application. However, carriers prepare more than one pair of interfaces for a multipath (It can be either a MPLS or IP multipath). Suppose a local interface card is pull out, the router is likely to quickly fail-over to another interface card so the multipath survives. At this time point, the BFD control packets are actually forwarded on a different forwarding path as the data packets. That is the root cause of the incorrect detection in our use case.

So we have already raised several potential solutions in our discussion:

1. BFD sessions per path
2. One dynamic BFD session which can adapt to the remained forwarding paths
3. Making use of the entropy label for a MPLS multipath
4. Two layered BFDs

Thanks,
Mingui

-----Original Message-----
From: Nobo Akiya (nobo) [mailto:[email-masked]] 
Sent: Saturday, July 19, 2014 7:39 AM
To: Mahesh Jethanandani; Jeffrey Haas
Cc: [email-masked]; [email-masked]
Subject: RE: Comments on draft-zhang-bfd-new-use-cases

> Cc: Nobo Akiya (nobo); [email-masked]; 
> rtg- [email-masked]
> and R4 one on each of the links, each of them reporting their status 
> to its listeners, be it OSPF ECMP, LAG, or MPLS link bundles and 
> letting them decide what is a failure scenario?
> >
> > (back to back connected links) and expose that information only to 
> > the layer managing the LAG.  If you want an application BFD session, 
> > that is an
> additional session.
> <snip>
> 
> Agreed.
> 
> The general deployment vision I have is that faster BFD sessions e.g. 
> 3.3ms are deployed on the member links of the multipath, and a slower 
> e.g. 100ms session deployed end-to-end.

Lastly ...

> The faster sessions detect and correct any errors on the multipaths 
> before an end-to-end session can detect a fault.

Thanks!

-Nobo