Subject: Re: BFD WG adoption for draft-haas-bfd-large-packets

From: "Albert Fu (BLOOMBERG/ 120 PARK)" <[email-masked]>
Date: Mon, 29 Oct 2018 17:38:32 -0000

Hi Les,

> Jeff/Albert -
> 
> Given the MTU issue is associated with a link coming up - and the use of Echo would allow the problem to be detected and prevent the BFD session from coming up - 
> and you are acknowledging that the protocol allows padded Echo packets today ...
> 
> is there really a need to do anything more?
> 
>    Les
> 

Actually, all the issues we have observed were not associated 
with link going up. The MTU issues occurred after OSPF/BGP had 
established adjacency without any events on the routers. 
ospf/BGP hellos/keepalives continued to be transmitted fine (small
packet size), but applications sending max size packets over the 
link would time out and fail. 

Hence, I mentioned several times that this issue is rather time 
consuming to troubleshoot, as the cause is with the Telco network
and outside of our control and we do not see any alarms.

I did also look at BFD echo mode. As Jeff indicated, this is not 
widely deployed (among the vendors we use, only one supports it).

Thanks
Albert

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Mon, 29 Oct 2018 18:13:53 +0000

Albert –

Do not confuse the current lack of detection with when the problem gets introduced.

The fact that the problem is not detected on protocol adjacency formation does not mean the problem gets introduced afterwards. Unless you are saying that folks change the link MTU AFTER the link comes up and has been used for a while the problem exists as soon as the link comes up. You could therefore detect this in a number of ways:

1)Use IS-IS ☺
2)Enhance other routing protocols to do what IS-IS does
3)Send BFD large packets (Echo or async)
4)Potentially some other OAM mechanism

In regards to #3, unless you believe link MTUs will change “on the fly”, sending the large packets during initial session bringup would be sufficient. If folks are concerned (as I am) that sending large BFD packets all the time introduces some risks for scalability/stability of BFD sessions, then one strategy would be to send the large packets only on session bringup.

FYI, some implementations have knobs on IS-IS to do exactly this i.e., send padded hellos until the adjacency is formed – then revert to small hellos. In the case of BFD I think there is a more compelling reason to be conservative in how often you send large packets given where it is implemented and how often the BFD packets are sent.

   Les

From: Albert Fu (BLOOMBERG/ 120 PARK) <[email-masked]>
Sent: Monday, October 29, 2018 10:39 AM
To: Les Ginsberg (ginsberg) <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: BFD WG adoption for draft-haas-bfd-large-packets

Hi Les,

Thanks
Albert

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 29 Oct 2018 14:26:16 -0400

Les,

On Mon, Oct 29, 2018 at 06:13:53PM +0000, Les Ginsberg (ginsberg) wrote:
> The fact that the problem is not detected on protocol adjacency formation
> does not mean the problem gets introduced afterwards. Unless you are
> saying that folks change the link MTU AFTER the link comes up and has been
> used for a while the problem exists as soon as the link comes up. You
> could therefore detect this in a number of ways:

It feels like you're missing part of the issue, Les.  Here's an example
timeline.

1. Customer is delivered a link that is provisioned with 1500 MTU.

2. The link is implemented, e.g., with l2vpn.  It could be frame relay. It
could be whatever, but is likely not a TDM link as of old.

3. The link starts getting used.  Things are fine.

4a. There's a topology change in the l2vpn network.  A result is that another
label gets used and effective path MTU is reduced.  We now start dropping
full sized packets.

4b. The l2vpn traverses an internal ECMP, and one leg of it develops an MTU
issue due to a topology change and additional label usage.  We now start
dropping full sized packets sometimes.

> 1)Use IS-IS ☺

Great when you can!

> 3)Send BFD large packets (Echo or async)

3 - see 2. :-)

> 4)Potentially some other OAM mechanism

If you have a list of other mechanisms, please list them.  Minimally,
they're good fodder for the draft.

> In regards to #3, unless you believe link MTUs will change “on the fly”,

Effectively, this happens.

> sending the large packets during initial session bringup would be
> sufficient. If folks are concerned (as I am) that sending large BFD
> packets all the time introduces some risks for scalability/stability of
> BFD sessions, then one strategy would be to send the large packets only on
> session bringup.

The scalability/stability issues are understood to be a potential issue.
Hence doing this as a draft and engaging discussion.

But hopefully you can see above where simply doing MTU probing only at the
beginning is not sufficient.

It's also arguable that doing it for every single packet isn't necessary for
many people's scenarios.  How often is it needed?  By whom?  With what
detection interval?  Again, good points for discussion.

> FYI, some implementations have knobs on IS-IS to do exactly this i.e.,
> send padded hellos until the adjacency is formed – then revert to small
> hellos.

This wouldn't cover Albert's scenario.

> In the case of BFD I think there is a more compelling reason to be
> conservative in how often you send large packets given where it is
> implemented and how often the BFD packets are sent.

You don't need to convince me. :-)

-- Jeff