Subject: Re: I-D ACTION:draft-shen-bfd-intf-p2p-nbr-00.txt

From: "Thomas D. Nadeau" <[email-masked]>
Date: Mon, 2 Apr 2007 10:01:18 -0400

On Mar 30, 2007:9:29 PM, at 9:29 PM, Naiming Shen wrote:

>
> On Mar 30, 2007, at 5:52 PM, Dave Katz wrote:
>
>>
>> On Mar 30, 2007, at 4:01 PM, Naiming Shen wrote:
>>
>>>>
>>>> But secondly, there is a self-referential problem.  What you'd  
>>>> *really* like to do on a BFD failure is to take down the failing  
>>>> protocol and keep it down until the BFD session resumes.  But  
>>>> you can't do this if BFD is running on top of that failing  
>>>> protocol (which it will be) which thus requires the "special  
>>>> flag" hack and only temporarily taking down the protocol.  The  
>>>> fate-sharing implications of this are more serious, because it  
>>>> is possible to get false positives in some cases when the data  
>>>> protocol has failed.  Imagine, for example, running ISIS with  
>>>> this method.  If IPv4 fails, the BFD session goes down, the  
>>>> interface flaps for five seconds, and then is reenabled.  At  
>>>> that point ISIS will happily come back up and report IP  
>>>> reachability even though it doesn't exist.  You can bet that  
>>>> folks who have not yet implemented BFD will be tempted to do this.
>>>
>>>
>>> If you are talking about sharing fate between ipv4 bfd and ISIS  
>>> is a problem, then I agree,
>>> this scheme is only meant for sharing fate among protocols, it  
>>> should not be used
>>> if this assumption is not true. But using ISIS to carry IP  
>>> information itself is violating this
>>> principle;-) don't know who to blame, although it's not a BFD issue.
>>
>> I guess my point is that you need to be very explicit about what  
>> works and what doesn't.  It's certainly the case that, without the  
>> presence of BFD, ISIS will do Bad Things if IP forwarding breaks,  
>> since it will continue to advertise IP connectivity.  But the  
>> whole point of BFD is to sense data protocol connectivity and  
>> provide that (presumably useful) information to other parts of a  
>> system.  If BFD is providing information directly to ISIS, it can  
>> withdraw IP connectivity (or tear down the adjacency if need be)  
>> and keep it that way until connectivity is restored.  If ISIS  
>> relied on this scheme, and IP connectivity failed (but datalink  
>> connectivity remained), the ISIS adjacency would flap, and then  
>> ISIS would proceed to black hole traffic.
>
> Even in the normal BFD interacting with ISIS(for ipv4), I would  
> think it can also do this
> black holing. Since ipv4 bfd session is flapping, and datalink  
> layer is fine, and ISIS
> packets is going through ok, hellos are happy. ISIS will bring up  
> the adjacency, and
> then register with bfd, and bfd later failed again, which will  
> bring down the ISIS session.
> I fail to see the difference between the two schemes.

	A simple example of how this can be broken is that if you
consider a distributed router, where you run ISIS on the routing
processor, and BFD on specialized hardware down on a line card.
While the line card's hardware could continue to send/receive
BFD packets, the ISIS process on the RP could crash or get wedged,
and cause it to ignore routing updates. If the routing protocol
timers are sufficiently high (usually on the order of minutes or
even hours), then no one will know until this timer goes off.

>
>> I think you need to specifically disallow this mechanism for cases  
>> like this, namely, applications that will continue to run even  
>> with the failure of a data protocol, but whose correct operation  
>> requires that data protocol.  (Note that this sentence describes  
>> both ISIS and static routes.)
>>
>> OSPFv3 is another interesting example if you're running BFD in  
>> this configuration only over IPv4.  If there is a v4 failure,  
>> OSPFv3 will flap unnecessarily.  This gets back to the IPv4 ==  
>> everything fate sharing that is at the heart of the way you've  
>> specified it, and which I think is an unnecessary restriction.  A  
>> number of systems (including the one I'm most familiar with these  
>> days) has an interface hierarchy that includes the data protocol.   
>> Such systems are likely better served by having, say, separate v4  
>> and v6 BFD sessions and flapping the appropriate data protocol up/ 
>> down status in the face of a BFD session failure.  This would  
>> allow the OSPFv3 case to run unmolested when v4 died.  I would  
>> suggest to at least offer this up as a MAY when you discuss the  
>> fate sharing implications of this mechanism, since it should be  
>> essentially no more work to implement if the system is already  
>> built this way.
>
> Sure. There can be an configuration option for bring down the whole  
> thing or bring
> down the data protocol part if the platform supports that.
>
> Even though from architecture wise, the separation of bfds is  
> clean, ipv4 controls the
> ipv4 protocols and ipv6 controls the ipv6 protocols. there are  
> still much to be desired
> from implementation point of angle. On many routers BFD packets  
> going out not really
> through the exact data packets forwarding path or the packets are  
> sent out from the
> same software process, be it v6 or v6. So the argument of data  
> separation is rather
> mood. And I'm yet to see a case BFD session down is actually caused  
> by the layer 3
> lookup engine which is only responsible for ipv4;-) I would rather  
> do the re-route
> altogether though if we know one of the data plane is already in  
> trouble.

	Right. Just look at the example I gave above. Even if you run the ISIS
process on the line card's CPU, if BFD is run in special LC hardware  
outside
of the CPU, that constitutes two disjoint forwarding IP stacks; BFD  
will only
be testing its own.

>
>>
>>>>
>>>> In light of this, my preference would be for all of the verbiage  
>>>> about static routes and dynamic protocols and special fiags to  
>>>> be removed.  In place of this, add text that is very specific  
>>>> about the fate-sharing implications of this mechanism as  
>>>> outlined above, and point out that any application of BFD that  
>>>> does not automatically share fate with the data protocol over  
>>>> which BFD is running (such as ISIS or static routes) MUST have  
>>>> some form of explicit interaction with BFD in order to avoid  
>>>> false positives, and leave it at that.  The "special bit" hack  
>>>> is orthogonal to this mechanism;  it could just as well have  
>>>> been specified in the generic spec (and would have been just as  
>>>> inappropriate there.)
>>>
>>> I think the dynamic and static difference still needs to be  
>>> mentioned, although should not
>>> be directly linked with a 'special flag' for the static routing.
>>
>> But I think the "difference" here is fundamental--as soon as you  
>> have any special case communication between BFD and a part of the  
>> system, you've basically discarded the point of the draft (if I  
>> understand it) which is to be able to leverage BFD without  
>> changing your "clients" to specifically use it.  What you're  
>> specifying here is functionally *exactly* the same as what the  
>> generic spec talks about for static routes and other non-protocol  
>> applications, and only muddies the spec, IMHO.
>
> only the UP->Down portion is different between the two schemes. the  
> rest is the same.
> but the bring down itself is different from the point of dynamic or  
> static.
>
>>
>> Why not just say that this mechanism only provides a way of more  
>> rapidly taking down applications that would otherwise go down  
>> eventually and which will stay down on their own until the path is  
>> healed (namely, control protocols), and leave statics out of it  
>> altogether?
>
> Maybe you have a point. I'll think about that.

	I agree with Katz here. The point of this draft should be to suggest  
that
this approach AUGMENT the normal routing hellos, as well as existing  
ifUp/Down
mechanisms inside your box.

	--Tom

> thanks.
> - Naiming
>> --Dave

---

From: Naiming Shen <[email-masked]>
Date: Mon, 2 Apr 2007 10:14:27 -0700

Thomas,

On Apr 2, 2007, at 7:01 AM, Thomas D. Nadeau wrote:

>>>>> *really* like to do on a BFD failure is to take down the  
>>>>> failing protocol and keep it down until the BFD session  
>>>>> resumes.  But you can't do this if BFD is running on top of  
>>>>> that failing protocol (which it will be) which thus requires  
>>>>> the "special flag" hack and only temporarily taking down the  
>>>>> protocol.  The fate-sharing implications of this are more  
>>>>> serious, because it is possible to get false positives in some  
>>>>> cases when the data protocol has failed.  Imagine, for example,  
>>>>> running ISIS with this method.  If IPv4 fails, the BFD session  
>>>>> goes down, the interface flaps for five seconds, and then is  
>>>>> reenabled.  At that point ISIS will happily come back up and  
>>>>> report IP reachability even though it doesn't exist.  You can  
>>>>> bet that folks who have not yet implemented BFD will be tempted  
>>>>> to do this.
>>>>
>>>>
>>> works and what doesn't.  It's certainly the case that, without  
>>> the presence of BFD, ISIS will do Bad Things if IP forwarding  
>>> breaks, since it will continue to advertise IP connectivity.  But  
>>> the whole point of BFD is to sense data protocol connectivity and  
>>> I think you need to specifically disallow this mechanism for  
>>> cases like this, namely, applications that will continue to run  
>>> even with the failure of a data protocol, but whose correct  
>>> operation requires that data protocol.  (Note that this sentence  
>>> describes both ISIS and static routes.)
>>>
>>> days) has an interface hierarchy that includes the data  
>>> protocol.  Such systems are likely better served by having, say,  
>>> separate v4 and v6 BFD sessions and flapping the appropriate data  
>>> protocol up/down status in the face of a BFD session failure.   
>>> This would allow the OSPFv3 case to run unmolested when v4 died.   
>>> I would suggest to at least offer this up as a MAY when you  
>>> discuss the fate sharing implications of this mechanism, since it  
>>> should be essentially no more work to implement if the system is  
>>> already built this way.
>>
>> Sure. There can be an configuration option for bring down the  
>> whole thing or bring
>> mood. And I'm yet to see a case BFD session down is actually  
>> caused by the layer 3
> process on the line card's CPU, if BFD is run in special LC  
> hardware outside
>>>>> In light of this, my preference would be for all of the  
>>>>> verbiage about static routes and dynamic protocols and special  
>>>>> fiags to be removed.  In place of this, add text that is very  
>>>>> specific about the fate-sharing implications of this mechanism  
>>>>> as outlined above, and point out that any application of BFD  
>>>>> that does not automatically share fate with the data protocol  
>>>>> over which BFD is running (such as ISIS or static routes) MUST  
>>>>> have some form of explicit interaction with BFD in order to  
>>>>> avoid false positives, and leave it at that.  The "special bit"  
>>>>> hack is orthogonal to this mechanism;  it could just as well  
>>>>> have been specified in the generic spec (and would have been  
>>>>> just as inappropriate there.)
>>>>
>> only the UP->Down portion is different between the two schemes.  
>> the rest is the same.
>> but the bring down itself is different from the point of dynamic  
>> or static.
>>
>>>
>>> eventually and which will stay down on their own until the path  
>>> is healed (namely, control protocols), and leave statics out of  
>>> it altogether?
>>
> 	I agree with Katz here. The point of this draft should be to  
> suggest that
> this approach AUGMENT the normal routing hellos, as well as  
> existing ifUp/Down

I agree the intf UP->Down signal is the key of this draft; but as I  
pointed out
in the previous email, there is also an important difference in the  
bring-up stage
in terms of non-protocol services, that the bfd session state is  
condensed into
an intf-based indication, which is very easy for implementation.  
Since those
services looking at the intf state anyway, to augment with tins intf- 
p2p bfd,
is just simply a one-line diff to those services.

thanks.
- Naiming

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Mon, 2 Apr 2007 13:21:58 -0400

On Apr 2, 2007:1:14 PM, at 1:14 PM, Naiming Shen wrote:

>>>>>> cases when the data protocol has failed.  Imagine, for  
>>>>>> example, running ISIS with this method.  If IPv4 fails, the  
>>>>>> BFD session goes down, the interface flaps for five seconds,  
>>>>>> and then is reenabled.  At that point ISIS will happily come  
>>>>>> back up and report IP reachability even though it doesn't  
>>>>>> exist.  You can bet that folks who have not yet implemented  
>>>>>> BFD will be tempted to do this.
>>>>>
>>>>>
>>>> breaks, since it will continue to advertise IP connectivity.   
>>>> But the whole point of BFD is to sense data protocol  
>>>> connectivity and provide that (presumably useful) information to  
>>>> other parts of a system.  If BFD is providing information  
>>>> directly to ISIS, it can withdraw IP connectivity (or tear down  
>>>> the adjacency if need be) and keep it that way until  
>>>> connectivity is restored.  If ISIS relied on this scheme, and IP  
>>>> connectivity failed (but datalink connectivity remained), the  
>>>> ISIS adjacency would flap, and then ISIS would proceed to black  
>>>> hole traffic.
>>>
>>>> specified it, and which I think is an unnecessary restriction.   
>>>> A number of systems (including the one I'm most familiar with  
>>>> these days) has an interface hierarchy that includes the data  
>>>> separate v4 and v6 BFD sessions and flapping the appropriate  
>>>> data protocol up/down status in the face of a BFD session  
>>>> failure.  This would allow the OSPFv3 case to run unmolested  
>>>> when v4 died.  I would suggest to at least offer this up as a  
>>>> MAY when you discuss the fate sharing implications of this  
>>>> mechanism, since it should be essentially no more work to  
>>>> implement if the system is already built this way.
>>>
>>> lookup engine which is only responsible for ipv4;-) I would  
>>> rather do the re-route
>> of the CPU, that constitutes two disjoint forwarding IP stacks;  
>> BFD will only
>> be testing its own.
>>
>>>
>>>>
>>>>>>
>>>>>> avoid false positives, and leave it at that.  The "special  
>>>>>> bit" hack is orthogonal to this mechanism;  it could just as  
>>>>>> well have been specified in the generic spec (and would have  
>>>>>> been just as inappropriate there.)
>>>>>
>>>> have any special case communication between BFD and a part of  
>>>> the system, you've basically discarded the point of the draft  
>>>> (if I understand it) which is to be able to leverage BFD without  
>>>> generic spec talks about for static routes and other non- 
>>>> protocol applications, and only muddies the spec, IMHO.
>>>
> services looking at the intf state anyway, to augment with tins  
> intf-p2p bfd,

	Precisely. If it is condensed into functions
like the following, I think we are on the same page.
Note the subtlety of the ifUp state and how you
get there.

	if (oldIfUp && bfdUP )
		ifUp();

	if (oldIfDown || bfdDown)
		ifDown();

	--Tom

---

From: Naiming Shen <[email-masked]>
Date: Mon, 9 Apr 2007 00:56:21 -0700

Just to add one more thing that, even though this draft does not change
anything in terms of BFD base spec, but this scheme does generate unique
error conditions, such as detection of multiple peers over a point-to- 
point
interface. If the implementation does bring the session down, it  
needs to
report this with a new DIAG code to reflect this error. Although the  
detection
mechanism is out side the scope of this document. Thanks to the folks
brought this to my attention. Will fix.

thanks.
- Naiming

---

From: "Nitin Bahadur" <[email-masked]>
Date: Mon, 9 Apr 2007 10:15:41 -0700

If you add a new diag-code, then you either need to do it to the base
spec or make your spec Standards track...I prefer the former.

Nitin

> -----Original Message-----
> From: Naiming Shen [mailto:[email-masked]]
> Sent: Monday, April 09, 2007 12:56 AM
> To: [email-masked]
> Subject: Re: I-D ACTION:draft-shen-bfd-intf-p2p-nbr-00.txt
> 
>

---

From: Naiming Shen <[email-masked]>
Date: Mon, 09 Apr 2007 10:25:23 -0700

Nitin,

> spec or make your spec Standards track...I prefer the former.

Same here. thanks.

- Naiming

>>Just to add one more thing that, even though this draft does not
> 
> change
> 
>>anything in terms of BFD base spec, but this scheme does generate
> 
> unique
>