Subject: RE: document layout for singlehop, multihop and generic

From: [email-masked]
Date: Wed, 26 Oct 2005 17:57:47 +0100

Peter,

I fail to see how using BFD to detect component link failures in a link-group will be useful, perhaps you can provide more details on your proposal? 

If an individual link fails then the traffic using the failed link should be switched over to one of the other links in the link-group, i.e. as long as at least one link in the link-group is up and has enough bandwidth available, no packets should be dropped (including BFD over UDP packets). For distributed BFD implementations where BFD packets are generated/processed on a per line card basis, it might be possible to check that a BFD packet for a particular session is received via the correct link. However, depending on the system architecture, checking which link a BFD packet is received on may be complex/costly if not impossible for implementations that use a centralised CPU for BFD packet generation/processing. This restricts the applicability of your proposal to either (i) using BFD to detect link failures as a backup measure in case the 802.3ad component link failure mechanism isn't working, or (ii) providing failure detection faster than the average/maximum 802.3ad link failure detection time.

Regarding (i), if 802.3ad link failure detection isn't working then there is obviously a problem so its quite likely that BFD will also be effected (especially as the BFD session setup and state maintenance will somehow need to be linked to the 802.3ad load balancing policy/algorithm). Regarding (ii), the 802.3ad implementations that I know of use some kind of polling to detect component link failures. Failure detection time is normally between 1-100ms, depending on when the next poll is due when the failure occurs. IMO an average link failure detection time of 50ms is more than fast enough for most if not all applications.

There are also other factors that need to be considered as well such binding both directions to the same component link (particularly when using systems with different 802.3ad load balancing policies/algorithms), and issues caused by intermediate hops between the BFD systems also running 802.3ad.

In general, IMO it is not sensible to use an IP/UDP based protocol to detect component link failures for a link aggregation mechanism that has been purposely designed to be transparent to IP/UDP.

Regards,
Richard

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On
> Behalf Of Peter Arberg
> Sent: 25 October 2005 22:07
> To: 'David Ward'; 'Pekka Savola'; [email-masked]; 'Jeffrey Haas'
> Subject: RE: document layout for singlehop, multihop and generic
> 
> 
> Hi,
> 
> I have one suggestion which I would like to see in one of the
> documents, best fit would proabably be in the 1hop document,
> but could also be in the generic one.
> 
> The usage of BFD over 802.3ad ethernet link-groups, in an
> application to detect faults in the individual links inside
> the link-group.
> 
> (not BFD over ethernet, but using the BFD over UDP approach)
> 
> I could propose some text for this, and we can discuss it
> based on this, and then if you do not think it belongs in either 
> of these document, or it is to late in the game to propose
> changes,  then I could create a new internet-draft with a 
> suggestion for this functionality.
> 
> thanks,
> Peter
> 
> > -----Original Message-----
> > From: [email-masked] 
> > [mailto:[email-masked]] On Behalf Of David Ward
> > Sent: 25. oktober 2005 22:46
> > To: Pekka Savola; [email-masked]; David Ward; Jeffrey Haas
> > Subject: Re: document layout for singlehop, multihop and generic
> > 
> > We covered this at the last several WG meetings. The issue 
> > is: reorganizing
> > the docs w/o changing the content vs getting the docs to spec 
> > ... The WG
> > agreed to get the docs to full standard.
> > 
> > -DWard
> > 
> > 
> > On 10/25/05 5:22 AM, "Pekka Savola" <[email-masked]> wrote:
> > 
> > > Hi,
> > > 
> > > I'm still puzzled by the coverage of various documents:
> > > 
> > > 1) bfd-multihop-03: about 3 pages of very generic (non-protocol
> > > specific) guidance on multihop BFD sessions.
> > > 
> > > 2) bfd-v4v6-1hop-04: the first 4.5 pages are very generic 
> single-hop
> > > BGP guidance (applicable to any single hop BFD 
> > application), the next
> > > 4 pages detail a number of OSPF/IS-IS details for single-hop cases
> > > 
> > > 3) bfd-generic-01: about 5 pages of generic BFD guidance (not
> > > specific to either single or multihop).
> > > 
> > > This seems like patchwork.  IMHO, a more logical 
> separation might be
> > > having everything generic (either single or multihop) in 
> > one document
> > > and moving the protocol-specific details out of v2v6-1hop to a
> > > separate BFD+OSPF/ISIS document.
> > 
> 
> 
>

---

From: "Peter Arberg" <[email-masked]>
Date: Thu, 27 Oct 2005 23:24:31 +0200

Richard,

I have added some comments and answers inline in the email. 

> From: [email-masked] [mailto:[email-masked]] 
> Sent: 26. oktober 2005 18:58
> To: [email-masked]; [email-masked]; [email-masked]; 
> [email-masked]; [email-masked]
> I fail to see how using BFD to detect component link failures 
> in a link-group will be useful, perhaps you can provide more 
> details on your proposal? 

Sure. The idea of using BFD on individual links inside a link-group
is no difference than using BFD as link failure detection on
single links.

It is for bidirectional link failure detection as well as fast
detection of failure.

> If an individual link fails then the traffic using the failed 
> link should be switched over to one of the other links in the 
> link-group, i.e. as long as at least one link in the 
> link-group is up and has enough bandwidth available, no 
> packets should be dropped (including BFD over UDP packets). 

Yes, but the link-group implementation still have to detect
that the link is down to avoid using the individual link.

Some link-group implementations are using static group configuration,
and as such no LACP to detect if each individual link is active and
as such the link failure detection is dependant on light out.

In this case having a bidirectional link failure detection like
BFD will be beneficial, as relying on light out have the 
possibility of creating unidirectional failures and as such
blackholing of packets.

> For distributed BFD implementations where BFD packets are 
> generated/processed on a per line card basis, it might be 
> possible to check that a BFD packet for a particular session 
> is received via the correct link. 

Correct, that was the idea we have been investigating, and as 
such initiate n*BFD sessions, 1 per link in the link-group,
all with the same ip-source and ip-destination address, but
all with different my & yours discriminators.

> However, depending on the 
> system architecture, checking which link a BFD packet is 
> received on may be complex/costly if not impossible for 
> implementations that use a centralised CPU for BFD packet 
> generation/processing. 

Agree, there can be some possible architectures which will 
have more difficulty doing the proposed solution than others,
but all in all it should be doable for all architectures with
varying degree of speed. And is this not the case for all 
features, that some architectures are better suited than others 
for specific functionalities.

> This restricts the applicability of 
> your proposal to either (i) using BFD to detect link failures 
> as a backup measure in case the 802.3ad component link 
> failure mechanism isn't working, or (ii) providing failure 
> detection faster than the average/maximum 802.3ad link 
> failure detection time.

The idea was to use BFD in case eihter LACP is not present, or
a faster link failure detection is needed than normal LACP 
implementations will provide.

LACP is typical a control protocol, and as such implemented on 
system route processors, or central control units in network
equipment, and as such the speed where it can detect and react
to a link failure can vary, but are seldom in 10's of milliseconds.

> 
> Regarding (i), if 802.3ad link failure detection isn't 
> working then there is obviously a problem so its quite likely 
> that BFD will also be effected (especially as the BFD session 
> setup and state maintenance will somehow need to be linked to 
> the 802.3ad load balancing policy/algorithm). 

The only thing in common BFD will have to the 802.3ad 
load-balance as I see it is the fact that different BFD 
sessions need to be setup on individual links inside the 
link-group. As soon as a BFD session is assosiated with 
the link, it is as such independant of if the link is a 
seperate link or part of a link-group, it will run as a 
independant BFD session.

> Regarding (ii), 
> the 802.3ad implementations that I know of use some kind of 
> polling to detect component link failures. Failure detection 
> time is normally between 1-100ms, depending on when the next 
> poll is due when the failure occurs. IMO an average link 
> failure detection time of 50ms is more than fast enough for 
> most if not all applications.

Yes I agree that in the case of a link component failure is
in fact bidirectional, then each ends of a link-group can 
react fast to the "light out" condition and as such retract
the link from the link-group in this case BFD will be of
no help.

In the case where a uni-directional failure happens, and one 
peer have to signal to the other peer that a failure has
occured, a lot of packets will be lost, and it is in this
scenario BFD will be useful, and should be able to bring us 
down into the 50msec. time for all kind of link-failures
inside a link-group.

> There are also other factors that need to be considered as 
> well such binding both directions to the same component link 
> (particularly when using systems with different 802.3ad load 
> balancing policies/algorithms), and issues caused by 
> intermediate hops between the BFD systems also running 802.3ad.

I agree more study is needed in the case of intermediate systems
running 802.3ad link-groups, but if we look at this in a 
"single hop, or direct connected" solution, then I don't think
there will be any issues no matter what kind of load-balancing
algorithm each peer uses, as the BFD sessions will not be part 
of the load-balancing.  A BFD session will be tied to a link,
and as such as long as both systems can agree to tied it to 
the same link as packets initially are received on, then there
should be no issues.

> In general, IMO it is not sensible to use an IP/UDP based 
> protocol to detect component link failures for a link 
> aggregation mechanism that has been purposely designed to be 
> transparent to IP/UDP.

I'm not sure I understand why not, other links are transparent
to IP/UDP, and as such used to transport IP/UDP and yet we
use BFD to detect failures of these links.

Individual links inside a link-group will from BFD perspective
be treated as a single link.

I'm not looking at running BFD detection on the link-group as
an aggregate, as I agree this is not useful.

cheers,
Peter