Subject: Fwd: I-D ACTION:draft-shen-bfd-intf-p2p-nbr-00.txt

From: Naiming Shen <[email-masked]>
Date: Fri, 30 Mar 2007 10:21:43 -0700

hi bfd-wg,

comments are welcome.

thanks.
- Naiming

Begin forwarded message:

> From: [email-masked]
> Date: March 28, 2007 12:50:02 PM PDT
> To: [email-masked]
> Subject: I-D ACTION:draft-shen-bfd-intf-p2p-nbr-00.txt
> Reply-To: [email-masked]
>
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories.
>
>
> 	Title		: Interface Based Point-to-Point Neighbor BFD
> 	Author(s)	: N. Shen
> 	Filename	: draft-shen-bfd-intf-p2p-nbr-00.txt
> 	Pages		: 7
> 	Date		: 2007-3-28
> 	
>    This document describes an application of Bidirectional Forwarding
>    Detection (BFD) protocol named Interface Based Point-to-Point
>    Neighbor BFD. This mechanism can be used to simplify the BFD
>    operations over point-to-point physical and logical links in
>    networks.
>
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-shen-bfd-intf-p2p-nbr-00.txt
>
> To remove yourself from the I-D Announcement list, send a message to
> [email-masked] with the word unsubscribe in the body of
> the message.
> You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce
> to change your subscription settings.
>
> Internet-Drafts are also available by anonymous FTP. Login with the
> username "anonymous" and a password of your e-mail address. After
> logging in, type "cd internet-drafts" and then
> "get draft-shen-bfd-intf-p2p-nbr-00.txt".
>
> A list of Internet-Drafts directories can be found in
> http://www.ietf.org/shadow.html
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>
> Internet-Drafts can also be obtained by e-mail.
>
> Send a message to:
> 	[email-masked].
> In the body type:
> 	"FILE /internet-drafts/draft-shen-bfd-intf-p2p-nbr-00.txt".
> 	
> NOTE:	The mail server at ietf.org can return the document in
> 	MIME-encoded form by using the "mpack" utility.  To use this
> 	feature, insert the command "ENCODING mime" before the "FILE"
> 	command.  To decode the response(s), you will need "munpack" or
> 	a MIME-compliant mail reader.  Different MIME-compliant mail readers
> 	exhibit different behavior, especially when dealing with
> 	"multipart" MIME messages (i.e. documents which have been split
> 	up into multiple messages), so check your local documentation on
> 	how to manipulate these messages.
>
> Below is the data which will enable a MIME compliant mail reader
> implementation to automatically retrieve the ASCII version of the
> Internet-Draft.
> Content-Type: text/plain
> Content-ID: <[email-masked]>
>
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/i-d-announce

---

From: Dave Katz <[email-masked]>
Date: Fri, 30 Mar 2007 14:44:56 -0700

Hi Naiming, thanks for submitting this.

I've pointed out privately to Naiming that I think this is a rather  
ugly architectural hack, albeit a pragmatic one.  The Right Way™ to  
do this would be to run BFD directly over the datalink layer and take  
down all data protocols (but not the datalink) in sympathy with the  
BFD session state.  However, there are (at least) two problems with  
this.  Firstly, it means specifying how to run BFD over a plethora of  
datalink layers.  Secondly, the IETF cannot standardize how to run  
BFD over datalink layers it does not control (so we could standardize  
something directly over PPP, for example, but not IEEE or cisco HDLC.)

The architectural ugliness, aside from being a layer violation, rears  
its head in a couple of ways related to fate-sharing.  Firstly, it  
means that everything shares fate with the IP over which BFD is being  
run, so if, say, IPv4 forwarding fails, non-v4 applications will see  
the failure.  This isn't *too* terrible;  it means that there may be  
false negatives, causing the application to temporarily flap.

But secondly, there is a self-referential problem.  What you'd  
*really* like to do on a BFD failure is to take down the failing  
protocol and keep it down until the BFD session resumes.  But you  
can't do this if BFD is running on top of that failing protocol  
(which it will be) which thus requires the "special flag" hack and  
only temporarily taking down the protocol.  The fate-sharing  
implications of this are more serious, because it is possible to get  
false positives in some cases when the data protocol has failed.   
Imagine, for example, running ISIS with this method.  If IPv4 fails,  
the BFD session goes down, the interface flaps for five seconds, and  
then is reenabled.  At that point ISIS will happily come back up and  
report IP reachability even though it doesn't exist.  You can bet  
that folks who have not yet implemented BFD will be tempted to do this.

My other objection to the "special flag" is that it is arguably  
overspecified.  As far as I can tell, this is isomorphic with simply  
having static routes interact with BFD "directly" and is already  
covered by the generic spec (which carefully says only what the  
effect of the interaction should be, not the implementation of it.)   
It is an explicit signaling mechanism from BFD, albeit a primitive one.

In light of this, my preference would be for all of the verbiage  
about static routes and dynamic protocols and special fiags to be  
removed.  In place of this, add text that is very specific about the  
fate-sharing implications of this mechanism as outlined above, and  
point out that any application of BFD that does not automatically  
share fate with the data protocol over which BFD is running (such as  
ISIS or static routes) MUST have some form of explicit interaction  
with BFD in order to avoid false positives, and leave it at that.   
The "special bit" hack is orthogonal to this mechanism;  it could  
just as well have been specified in the generic spec (and would have  
been just as inappropriate there.)

It would be nice to point out that the only function of flapping the  
interface is to provide an Up->Down edge for protocols to see, and  
that the only requirement for duration is for it to be long enough so  
that it isn't absorbed by any hysteresis mechanism that might be  
sitting on top of it, and short enough so that the protocol isn't  
held down for "too long" (another ugly interaction.)

--Dave

---

From: Naiming Shen <[email-masked]>
Date: Fri, 30 Mar 2007 16:01:30 -0700

Hi Dave,

Good comments, see some replies inline.

On Mar 30, 2007, at 2:44 PM, Dave Katz wrote:

> do this would be to run BFD directly over the datalink layer and  
> take down all data protocols (but not the datalink) in sympathy  
> with the BFD session state.  However, there are (at least) two  
> problems with this.  Firstly, it means specifying how to run BFD  
> over a plethora of datalink layers.  Secondly, the IETF cannot  
> standardize how to run BFD over datalink layers it does not control  
> (so we could standardize something directly over PPP, for example,  
> but not IEEE or cisco HDLC.)

Yes, ugly architecture to avoid endless of clean work;-) only trying  
to be an engineer.

>
> The architectural ugliness, aside from being a layer violation,  
> rears its head in a couple of ways related to fate-sharing.   
> Firstly, it means that everything shares fate with the IP over  
> which BFD is being run, so if, say, IPv4 forwarding fails, non-v4  
> applications will see the failure.  This isn't *too* terrible;  it  
> means that there may be false negatives, causing the application to  
> temporarily flap.
>
> implications of this are more serious, because it is possible to  
> get false positives in some cases when the data protocol has  
> failed.  Imagine, for example, running ISIS with this method.  If  
> IPv4 fails, the BFD session goes down, the interface flaps for five  
> seconds, and then is reenabled.  At that point ISIS will happily  
> come back up and report IP reachability even though it doesn't  
> exist.  You can bet that folks who have not yet implemented BFD  
> will be tempted to do this.

I don't quick get this point. Let's take an easy example: if the bfd  
always uses the
connected nexthop of the peer address as it's bfd peer address, how  
ISIS is doing
should be independent of the intf-p2p BFD session and vise versa.

If you are talking about sharing fate between ipv4 bfd and ISIS is a  
problem, then I agree,
this scheme is only meant for sharing fate among protocols, it should  
not be used
if this assumption is not true. But using ISIS to carry IP  
information itself is violating this
principle;-) don't know who to blame, although it's not a BFD issue.

if you are talking about bfd connection depending on the ISIS routes,  
then the original
BFD spec also has this problem, this is not a new feature of this draft.

> overspecified.  As far as I can tell, this is isomorphic with  
> simply having static routes interact with BFD "directly" and is  
> already covered by the generic spec (which carefully says only what  
> the effect of the interaction should be, not the implementation of  
> it.)  It is an explicit signaling mechanism from BFD, albeit a  
> primitive one.

Ok, I agree. This 'special flag' is not a requirement of this scheme,  
it's a purely implementation
issue, be it a 'flag', a 'registration/callback' function, or  
something else. I'll change that part.

> removed.  In place of this, add text that is very specific about  
> the fate-sharing implications of this mechanism as outlined above,  
> and point out that any application of BFD that does not  
> automatically share fate with the data protocol over which BFD is  
> running (such as ISIS or static routes) MUST have some form of  
> explicit interaction with BFD in order to avoid false positives,  
> and leave it at that.  The "special bit" hack is orthogonal to this  
> mechanism;  it could just as well have been specified in the  
> generic spec (and would have been just as inappropriate there.)

I think the dynamic and static difference still needs to be  
mentioned, although should not
be directly linked with a 'special flag' for the static routing.

>
> It would be nice to point out that the only function of flapping  
> the interface is to provide an Up->Down edge for protocols to see,  
> and that the only requirement for duration is for it to be long  
> enough so that it isn't absorbed by any hysteresis mechanism that  
> might be sitting on top of it, and short enough so that the  
> protocol isn't held down for "too long" (another ugly interaction.)
>

Yes, I should emphasize this Up->Down edge transition and the timing  
thing as suggested
in this draft.

thanks again.
- Naiming

---

From: "Nitin Bahadur" <[email-masked]>
Date: Fri, 30 Mar 2007 16:41:01 -0700

Not sure of the exact problem you are trying to solve. Instead of using
BFD as a link failure detection mechanism for Ethernet, can't you use
Ethernet OAM? If this draft is a stop-gap mechanism before Ethernet OAM
is implemented/put-in-use, maybe it's ok.

To implement the draft correctly, a lot of special case handling would
need to be added in multiple places. It would be better to use a
link-layer mechanism for detecting link failures.

Also, with the draft you are tying in the concept of a link failure to a
bfd session failure...which might not necessarily be true. BFD sessions
might fail due to firewall filters, IP packet handling errors. You would
need more tools to tell the customer/operator that the link is *actually
not down* and it's BFD that has marked the link as down.

Nitin

---

From: Naiming Shen <[email-masked]>
Date: Fri, 30 Mar 2007 16:54:32 -0700

> is implemented/put-in-use, maybe it's ok.

If Ethernet OAM works for you, yes use it. This draft is not just designed
for Ethernet though.

> link-layer mechanism for detecting link failures.

You mean bring down and bring up and intf is more work than putting hooks
everywhere in every protocol and every service? not to speak for every
implementations, for certain ones, one and half day of coding and testing
should cover this draft.

> not down* and it's BFD that has marked the link as down.

to be honest, generate an meaningful error message saying 'intf-p2p bfd
brought down intf' will probably do for most of the customers.

thanks.
- Naiming

---

From: Dave Katz <[email-masked]>
Date: Fri, 30 Mar 2007 17:52:56 -0700

On Mar 30, 2007, at 4:01 PM, Naiming Shen wrote:

>> IPv4 fails, the BFD session goes down, the interface flaps for  
>> five seconds, and then is reenabled.  At that point ISIS will  
>> happily come back up and report IP reachability even though it  
>> doesn't exist.  You can bet that folks who have not yet  
>> implemented BFD will be tempted to do this.
>
>
> If you are talking about sharing fate between ipv4 bfd and ISIS is  
> a problem, then I agree,
> this scheme is only meant for sharing fate among protocols, it  
> should not be used

I guess my point is that you need to be very explicit about what  
works and what doesn't.  It's certainly the case that, without the  
presence of BFD, ISIS will do Bad Things if IP forwarding breaks,  
since it will continue to advertise IP connectivity.  But the whole  
point of BFD is to sense data protocol connectivity and provide that  
(presumably useful) information to other parts of a system.  If BFD  
is providing information directly to ISIS, it can withdraw IP  
connectivity (or tear down the adjacency if need be) and keep it that  
way until connectivity is restored.  If ISIS relied on this scheme,  
and IP connectivity failed (but datalink connectivity remained), the  
ISIS adjacency would flap, and then ISIS would proceed to black hole  
traffic.

I think you need to specifically disallow this mechanism for cases  
like this, namely, applications that will continue to run even with  
the failure of a data protocol, but whose correct operation requires  
that data protocol.  (Note that this sentence describes both ISIS and  
static routes.)

OSPFv3 is another interesting example if you're running BFD in this  
configuration only over IPv4.  If there is a v4 failure, OSPFv3 will  
flap unnecessarily.  This gets back to the IPv4 == everything fate  
sharing that is at the heart of the way you've specified it, and  
which I think is an unnecessary restriction.  A number of systems  
(including the one I'm most familiar with these days) has an  
interface hierarchy that includes the data protocol.  Such systems  
are likely better served by having, say, separate v4 and v6 BFD  
sessions and flapping the appropriate data protocol up/down status in  
the face of a BFD session failure.  This would allow the OSPFv3 case  
to run unmolested when v4 died.  I would suggest to at least offer  
this up as a MAY when you discuss the fate sharing implications of  
this mechanism, since it should be essentially no more work to  
implement if the system is already built this way.

>> and leave it at that.  The "special bit" hack is orthogonal to  
>> this mechanism;  it could just as well have been specified in the  

But I think the "difference" here is fundamental--as soon as you have  
any special case communication between BFD and a part of the system,  
you've basically discarded the point of the draft (if I understand  
it) which is to be able to leverage BFD without changing your  
"clients" to specifically use it.  What you're specifying here is  
functionally *exactly* the same as what the generic spec talks about  
for static routes and other non-protocol applications, and only  
muddies the spec, IMHO.

Why not just say that this mechanism only provides a way of more  
rapidly taking down applications that would otherwise go down  
eventually and which will stay down on their own until the path is  
healed (namely, control protocols), and leave statics out of it  
altogether?

--Dave

---

From: Naiming Shen <[email-masked]>
Date: Fri, 30 Mar 2007 18:29:56 -0700

On Mar 30, 2007, at 5:52 PM, Dave Katz wrote:

>>> (which it will be) which thus requires the "special flag" hack  
>>> and only temporarily taking down the protocol.  The fate-sharing  
> point of BFD is to sense data protocol connectivity and provide  
> that (presumably useful) information to other parts of a system.   
> If BFD is providing information directly to ISIS, it can withdraw  
> IP connectivity (or tear down the adjacency if need be) and keep it  
> that way until connectivity is restored.  If ISIS relied on this  
> scheme, and IP connectivity failed (but datalink connectivity  
> remained), the ISIS adjacency would flap, and then ISIS would  
> proceed to black hole traffic.

Even in the normal BFD interacting with ISIS(for ipv4), I would think  
it can also do this
black holing. Since ipv4 bfd session is flapping, and datalink layer  
is fine, and ISIS
packets is going through ok, hellos are happy. ISIS will bring up the  
adjacency, and
then register with bfd, and bfd later failed again, which will bring  
down the ISIS session.
I fail to see the difference between the two schemes.

> the failure of a data protocol, but whose correct operation  
> requires that data protocol.  (Note that this sentence describes  
> both ISIS and static routes.)
>
> configuration only over IPv4.  If there is a v4 failure, OSPFv3  
> will flap unnecessarily.  This gets back to the IPv4 == everything  
> fate sharing that is at the heart of the way you've specified it,  
> and which I think is an unnecessary restriction.  A number of  
> systems (including the one I'm most familiar with these days) has  
> an interface hierarchy that includes the data protocol.  Such  
> systems are likely better served by having, say, separate v4 and v6  
> BFD sessions and flapping the appropriate data protocol up/down  
> status in the face of a BFD session failure.  This would allow the  
> OSPFv3 case to run unmolested when v4 died.  I would suggest to at  
> least offer this up as a MAY when you discuss the fate sharing  
> implications of this mechanism, since it should be essentially no  
> more work to implement if the system is already built this way.

Sure. There can be an configuration option for bring down the whole  
thing or bring
down the data protocol part if the platform supports that.

Even though from architecture wise, the separation of bfds is clean,  
ipv4 controls the
ipv4 protocols and ipv6 controls the ipv6 protocols. there are still  
much to be desired
from implementation point of angle. On many routers BFD packets going  
out not really
through the exact data packets forwarding path or the packets are  
sent out from the
same software process, be it v6 or v6. So the argument of data  
separation is rather
mood. And I'm yet to see a case BFD session down is actually caused  
by the layer 3
lookup engine which is only responsible for ipv4;-) I would rather do  
the re-route
altogether though if we know one of the data plane is already in  
trouble.

>>> the fate-sharing implications of this mechanism as outlined  
>>> above, and point out that any application of BFD that does not  
> But I think the "difference" here is fundamental--as soon as you  
> have any special case communication between BFD and a part of the  
> system, you've basically discarded the point of the draft (if I  
> understand it) which is to be able to leverage BFD without changing  
> your "clients" to specifically use it.  What you're specifying here  
> is functionally *exactly* the same as what the generic spec talks  
> about for static routes and other non-protocol applications, and  
> only muddies the spec, IMHO.

only the UP->Down portion is different between the two schemes. the  
rest is the same.
but the bring down itself is different from the point of dynamic or  
static.

> healed (namely, control protocols), and leave statics out of it  

Maybe you have a point. I'll think about that.

thanks.
- Naiming

---

From: Dave Katz <[email-masked]>
Date: Fri, 30 Mar 2007 19:20:26 -0700

On Mar 30, 2007, at 6:29 PM, Naiming Shen wrote:

>> since it will continue to advertise IP connectivity.  But the  
>> whole point of BFD is to sense data protocol connectivity and  
>> provide that (presumably useful) information to other parts of a  
>> system.  If BFD is providing information directly to ISIS, it can  
>> withdraw IP connectivity (or tear down the adjacency if need be)  
>> and keep it that way until connectivity is restored.  If ISIS  
>> relied on this scheme, and IP connectivity failed (but datalink  
>> connectivity remained), the ISIS adjacency would flap, and then  
>> ISIS would proceed to black hole traffic.
>
> Even in the normal BFD interacting with ISIS(for ipv4), I would  
> think it can also do this
> black holing. Since ipv4 bfd session is flapping, and datalink  
> layer is fine, and ISIS
> packets is going through ok, hellos are happy. ISIS will bring up  
> the adjacency, and
> then register with bfd, and bfd later failed again, which will  
> bring down the ISIS session.

In the v4 failure case, the BFD session would stay down, and the ISIS  
adjacencies (or announcement of IP reachability) would be held down  
as well, thus no black hole.  The problem in this scheme is that the  
circularity problem requires that you only temporarily assert that  
the interface is down;  you can't hold the whole interface down  
because BFD would never be able to come up.  Thus in this case ISIS  
would not know that IP reachability was still broken, and would black  
hole the traffic.

>> like this, namely, applications that will continue to run even  
>> with the failure of a data protocol, but whose correct operation  
>> OSPFv3 is another interesting example if you're running BFD in  
>> this configuration only over IPv4.  If there is a v4 failure,  
>> OSPFv3 will flap unnecessarily.  This gets back to the IPv4 ==  
>> everything fate sharing that is at the heart of the way you've  
>> specified it, and which I think is an unnecessary restriction.  A  
>> number of systems (including the one I'm most familiar with these  
>> days) has an interface hierarchy that includes the data protocol.   
>> Such systems are likely better served by having, say, separate v4  
>> and v6 BFD sessions and flapping the appropriate data protocol up/ 
>> down status in the face of a BFD session failure.  This would  
>> allow the OSPFv3 case to run unmolested when v4 died.  I would  
>> suggest to at least offer this up as a MAY when you discuss the  
>> fate sharing implications of this mechanism, since it should be  
>> essentially no more work to implement if the system is already  
>> built this way.
>
> Even though from architecture wise, the separation of bfds is  
> clean, ipv4 controls the
> ipv4 protocols and ipv6 controls the ipv6 protocols. there are  
> still much to be desired
> from implementation point of angle. On many routers BFD packets  
> going out not really
> lookup engine which is only responsible for ipv4;-) I would rather  
> do the re-route

But once again, because of the temporary nature of the "downness" of  
the interface, it will not really reroute the second protocol, but  
just cause it to flap (twice.)  But so long as the spec allows  
multiple sessions (since currently it does not) we can leave the  
mootness or lack thereof to the individual implementor.

>> understand it) which is to be able to leverage BFD without  
>> changing your "clients" to specifically use it.  What you're  
>> specifying here is functionally *exactly* the same as what the  
>> generic spec talks about for static routes and other non-protocol  
>> applications, and only muddies the spec, IMHO.
>

But the "bring down" is the crux of the proposal.

> Maybe you have a point. I'll think about that.

Thanks...

--Dave

---

From: Naiming Shen <[email-masked]>
Date: Fri, 30 Mar 2007 20:00:19 -0700

On Mar 30, 2007, at 7:20 PM, Dave Katz wrote:

>>> works and what doesn't.  It's certainly the case that, without  
>>> the presence of BFD, ISIS will do Bad Things if IP forwarding  
>>> breaks, since it will continue to advertise IP connectivity.  But  
>>> the whole point of BFD is to sense data protocol connectivity and  
> In the v4 failure case, the BFD session would stay down, and the  
> ISIS adjacencies (or announcement of IP reachability) would be held  
> down as well, thus no black hole.  The problem in this scheme is  
> that the circularity problem requires that you only temporarily  
> assert that the interface is down;  you can't hold the whole  
> interface down because BFD would never be able to come up.  Thus in  
> this case ISIS would not know that IP reachability was still  
> broken, and would black hole the traffic.

Ok, I see your point. So in this case as you mentioned, isis dealing  
could be categorized or
implemented the same as static routing if this is a concern.

>
>>
>>>
>>> I think you need to specifically disallow this mechanism for  
>>> cases like this, namely, applications that will continue to run  
>>> even with the failure of a data protocol, but whose correct  
>>> operation requires that data protocol.  (Note that this sentence  
>>> describes both ISIS and static routes.)
>>>
>>> days) has an interface hierarchy that includes the data  
>>> protocol.  Such systems are likely better served by having, say,  
>>> separate v4 and v6 BFD sessions and flapping the appropriate data  
>>> protocol up/down status in the face of a BFD session failure.   
>>> This would allow the OSPFv3 case to run unmolested when v4 died.   
>>> I would suggest to at least offer this up as a MAY when you  
>>> discuss the fate sharing implications of this mechanism, since it  
>>> should be essentially no more work to implement if the system is  
>>> already built this way.
>>
>> Sure. There can be an configuration option for bring down the  
>> whole thing or bring
>> mood. And I'm yet to see a case BFD session down is actually  
>> caused by the layer 3
> But once again, because of the temporary nature of the "downness"  
> of the interface, it will not really reroute the second protocol,  
> but just cause it to flap (twice.)  But so long as the spec allows  

Ok.

>> only the UP->Down portion is different between the two schemes.  
>> the rest is the same.
>> but the bring down itself is different from the point of dynamic  
>> or static.
>

Something else reminds that, there is a difference between the normal  
bring up stage
and this scheme in terms of the 'non-dynamic' applications, that was  
why I wrote this
'special flag' in the first place, is that the flag/callback/whatever  
with this scheme can be
an interface based trick, which is very easy to implement (or some  
multiple
interface/dataplane based) ; while the normal bfd which is nexthop  
based, need to have
proper registration/etc mechanism, it's rather hard to handle. It may  
worth to mention
this in the draft.

thanks.
- Naiming

>>> eventually and which will stay down on their own until the path  
>>> is healed (namely, control protocols), and leave statics out of  
>>> it altogether?
>>