Subject: RE: New Version Notification for draft-mbind-bfd-redundancy-01.txt

From: "Nobo Akiya (nobo)" <[email-masked]>
Date: Tue, 7 May 2013 16:15:48 +0000

Hello BFD WG Members,

Marc and myself have just rolled out rev -01 of BFD redundancy draft.

Some minor changes throughout the document, but core changes are:

(1) Section 2 has been re-written to better describe use case scenarios.
(2) Section 7 has been added to describe how the mechanism works for LSP ping bootstrapped sessions.

We believe that this revision makes better reading material over coffee, and we hope to see some response.

Best Regards,
Nobo & Marc

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Tuesday, May 07, 2013 12:09 PM
To: Marc Binderberger (mbinderb); Nobo Akiya (nobo)
Subject: New Version Notification for draft-mbind-bfd-redundancy-01.txt

A new version of I-D, draft-mbind-bfd-redundancy-01.txt
has been successfully submitted by Marc Binderberger and posted to the
IETF repository.

Filename:	 draft-mbind-bfd-redundancy
Revision:	 01
Title:		 Redundant BFD sessions
Creation date:	 2013-05-07
Group:		 Individual Submission
Number of pages: 9
URL:             http://www.ietf.org/internet-drafts/draft-mbind-bfd-redundancy-01.txt
Status:          http://datatracker.ietf.org/doc/draft-mbind-bfd-redundancy
Htmlized:        http://tools.ietf.org/html/draft-mbind-bfd-redundancy-01
Diff:            http://www.ietf.org/rfcdiff?url2=draft-mbind-bfd-redundancy-01

Abstract:
   This document defines a second or "shadow" BFD session to an existing
   "primary" BFD session, providing resiliency against BFD failures that
   are not legitimate.

   Scenarios will be discussed on how presence of a shadow BFD session
   will be beneficial in the context of high availability.

The IETF Secretariat

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 10 May 2013 12:19:48 -0400

[Speaking as an individual contributor.]

Nobo, Marc,

> We believe that this revision makes better reading material over coffee, and we hope to see some response.
[...]
> URL:             http://www.ietf.org/internet-drafts/draft-mbind-bfd-redundancy-01.txt

I just had a chance to read through this draft.  I think the use cases are
nicely stated.  Some comments on the draft.

Encoding:
Version numbers are *not* a good way for us to look at extending BFD as a
whole.  We don't have much of a space for such things.  For what it's worth,
Dave and I previously had some discussions about ways to piggy-back
additional data into BFD in the existing 5880 standard, but we've avoided it
every single time thus far.  Keeping in mind the underlying principles of
BFD, let's keep the protocol simple if possible and let the applications
make rich use of it.

Making use of distinct discriminators for the sessions lets us do this, and
should we pursue this draft, that would be my vote.  While the base BFD
specification encourages sharing the same session between common endpoints,
it doesn't preclude multiple sessions.  (And to some extent, that's what
we're doing for BFD over LAG - multiple sessions covering a single
endpoint.)

What becomes the interesting problem with multiple sessions is use of this
information by the application (e.g. BGP).  It would be necessary for both
sides of the connection to negotiate more than a single session and bind the
fate of the applications to the associated set of sessions.  In the absence
of both sides being aware that multiple sessions are involved (i.e. one side
is an existing BFD v1 speaker expecting a single session), the side
expecting a single session wouldn't use multiple sessions for fate-sharing
purposes.  In other words, regardless of packet format or multiple sessions,
the important thing is that the applications themselves know to use multiple
sessions.

Multiple sessions and arcane failures:
The covered use cases presume some failure of the BFD speaker in the
associated device.  There are a few additional scenarios that make multiple
sessions themselves somewhat problematic:
- The additional session doesn't share enough fate with the forwarding for
  the application (e.g. routing) for its input to be valid.  Such an
  analysis obviously will be vendor and implementation dependent.
- An interesting bit of complexity in requesting an additional session is
  also figuring out how to ask the remote side to associate that session
  with a particular resource such as a line card, main CPU, etc.  For
  example, if session 1 on router A is bound to a line card and bound to
  router B's main CPU, are you really monitoring what you thought?

Detection intervals:
You cover this to some extent in your draft under "Scale aspect".  A major
implication of multiple sessions is that timing is likely to be less
granular on one session.  You thus potentially gain resiliency to some
faults at the cost of a longer detection interval.

------

IMO, it's worth continuing to discuss the uses cases and possible mechanisms
for this draft.  However, I'd encourage the implementation to focus on the
application bootstrapping considerations for multiple sessions rather than
trying to add this to the packet format.

-- Jeff

---

From: "Nobo Akiya (nobo)" <[email-masked]>
Date: Fri, 10 May 2013 20:47:46 +0000

Hello Jeff,

Many thought provoking responses, thank you!

> BFD, let's keep the protocol simple if possible and let the applications make
> rich use of it.

What you and Dave had discussed in past regarding "piggy-back" will be an very interesting topic itself ... perhaps you can share one of these days :)

Yes I agree with the "simple" philosophy of BFD. I believe there are two aspects to this.
(1) Keep BFD a simple protocol.
(2) Keep BFD a simple protocol to use.

We definitely should keep BFD a simple protocol, but at the same time, I believe it will be beneficial to keep BFD a simple protocol to use, to promote rich use.

> specification encourages sharing the same session between common
> endpoints, it doesn't preclude multiple sessions.  (And to some extent,
> that's what we're doing for BFD over LAG - multiple sessions covering a
> single
> endpoint.)

Yes BFD over LAG is similar in some way. Micro sessions, fortunately, had distinct in/out interfaces to [de]mux. Distinct discriminators will be nice & clean. Tricky aspect is that we need distinct [de]mux "variable" in order to have distinct discriminators. UDP ports are not preferred since we would then need to come up different mechanism for IP-less sessions from. So the initial [de]muxer was exactly what Marc and I had problem with, and we were only left with option (a) and (b) from the documents. Sometimes I wish there were more "lego pieces" :)
 
> fate of the applications to the associated set of sessions.  In the absence of
> both sides being aware that multiple sessions are involved (i.e. one side is
> an existing BFD v1 speaker expecting a single session), the side expecting a
> single session wouldn't use multiple sessions for fate-sharing purposes.  In
> other words, regardless of packet format or multiple sessions, the
> important thing is that the applications themselves know to use multiple
> sessions.

I see that it's possible to absorb this logic in BFD. Document provides rules to consolidate the state.

   Final state is UP when the state of the primary session is UP or the
   state of the shadow session is UP.

   Final state is DOWN when both the state primary session is DOWN and
   the state of the shadow sessions is DOWN.

Even with one side being only v1 speaker, v1/v2 speaker side can produce correct final state to applications.

>   analysis obviously will be vendor and implementation dependent.

Yes.

>   router B's main CPU, are you really monitoring what you thought?

Interesting. Right now, I see the placement of two sessions is strictly local matter. In other words, if router A notices that router B has primary/shadow, then router A should assume that router B has placed the two sessions in optimal locations.

> implication of multiple sessions is that timing is likely to be less granular on
> one session.  You thus potentially gain resiliency to some faults at the cost
> of a longer detection interval.

Correct. If a system has max BFD provisioned, then v2 can only be introduced at the cost of stealing v1 resources. This is the unfortunate drawback. But even with this cost, I believe resiliency can bring benefits ... based on my guess of many production routers not max'ing out BFD resources. In addition, on centralized BFD, shared resources required to run shadows in secondary route processor may be smaller. So some aspects of this is implementation specific. 

> 
> ------
> 
> IMO, it's worth continuing to discuss the uses cases and possible
> mechanisms for this draft. 

I'm happy to see this statement, thank you :)

> However, I'd encourage the implementation to
> focus on the application bootstrapping considerations for multiple sessions
> rather than trying to add this to the packet format.

Proposed solution is what Marc and I thought as best at the time being. We also understand that this problem & solution falls in the gray zone between implementation vs. standard. We also _also_ believe that, due to aggressive nature of BFD, simple and consistent solution will be beneficial to vendors & end-users, but that can only be achieved if there was a standard. For those reasons, I would like to ensure, if this draft progresses, that outcome is indeed something simple and consistent, and is beneficial to _many_ vendors and _many_ end-users.

To conclude:
- We welcome more comments from vendors and end-users.
- Need more thought on how to use distinct discriminators for primary/shadow, if that's possible. If not, is usage of v2 is the right mechanism?

Regards,
Nobo

---

From: Marc Binderberger <[email-masked]>
Date: Sat, 11 May 2013 20:52:40 +0200

Hello Jeff,

thanks for the detailed feedback.

Let me comment on some key aspects:

> information by the application (e.g. BGP).

and

> However, I'd encourage the implementation to focus on the

This is where we have a directly opposed opinion. While I don't mind to provide the primary/shadow information details to the application if the application wants to know I am fairly sure that most applications do _not_ want to know. Applications like BGP "buy an liveliness service" from BFD; there is no interest from BGP to have exposure to BFD-internal challenges. Nor  do I think using application protocols as out-of-band to bootstrap BFD is a preferable design (sometimes you cannot avoid it, true), it makes things complex.

> whole.  We don't have much of a space for such things.

Partially agree. The version number space is limited but looking at the rate we use version numbers I don't see this as a problem per se.

> Keeping in mind the underlying principles of

That's why we tried to keep the packet layout the same. The packet engine remains the same, except the filter for v=2 (or v=1,2) instead of v=1. 

> should we pursue this draft, that would be my vote.

Once the session is Init or Up you are right. Problem is to bootstrap it. We do not plan for out-of-band signalling unless this is used already for "primary" sessions, e.g. RFC5884 BFD over LSPs, to avoid complexity.

Thus we need to separate the primary Down from the shadow Down packets. All the fields and bits are used already. The version field has the interesting property that an implementation only supporting v1 would (or should) simply drop the shadow packets.

Could we use discriminators ... I admit I ruled this out as we have only zero as a reserved value, otherwise it's up to the implementation how it assigns the value and the peer just reflects the discriminator back. We could start to define additional reserved discriminator values, e.g. 0xFFFFFFFF as the "shadow zero" but the problem remains that a v1-only implementation may see this discriminator value as a valid one (i.e. it's in use).

>  router B's main CPU, are you really monitoring what you thought?

I think Nobo answered this already: every implementation optimizes locally, i.e. tries to make it as resilient/redundant as it can. In this example at least A protects against a linecard failure with the help of B. For B, if there is only one CPU there is not much we can do but B at least can support A's effort by supporting shadow sessions.

There will be many discussions following, I'm sure ;-)

Regards, Marc