Subject: Re: WGLC for draft-ietf-bfd-large-packets

From: Jeffrey Haas <[email-masked]>
Date: Thu, 3 Oct 2019 16:12:53 -0400

Les,

On Fri, Sep 27, 2019 at 09:14:08PM +0000, Les Ginsberg (ginsberg) wrote:
> > The primary reason this is a "may" in the non-RFC 2119 sense is that our
> > experience also suggests that when the scaling impacts are primarily pps
> > rather than bps that this feature will likely have no major impact on
> > implementations beyond your valid concerns about exercising bugs.
> > 
> > I suspect had this not been mentioned at all, you would have been happier.
> > But you're not the target audience for this weak caveat.
> > 
> 
> [Les:] I am not opposed to a discussion of potential issues in the draft -
> rather I am encouraging it. But the current text isn't really on the mark
> as far as potential issues - and we seem to agree on that. It also
> suggests lengthening detection time to compensate - which I think is not
> at all what you want to suggest as it diminishes the value of the
> extension. It also isn't likely to address a real problem.

I think what I'm seeing from you is roughly:
- Note that larger MTUs may have impact on some implementations for BFD
  throughput.
- And simply stop there.

> For me, the potential issues are:
> 
> a)Some BFD implementations might not be able to handle MTU sized BFD
> packets - not because of performance - but because they did not expect BFD
> packets to be full size and therefore might have issues passing a large
> packet through the local processing engine.

In such cases, the BFD session wouldn't be able to come up.  Are you
picturing a problem more dire than that?

> b)Accepted MTU is impacted by encapsulations and what layer is being
> considered (L2 or L3). And oftentimes link MTUs do not match on both ends
> ("shudder"), so you might end up with unidirectional connectivity.

Did you mean for BFD or more in the general sense?

For BFD, if you have one side testing for large MTU but not the other, we
can still have a Up BFD session with possible packet drop for large packets
on the opposite side.  But there's the chance in some paths that MTU may be
unidirectionally different - e.g. satellite down vs. land up.[1]

In such cases, configuring BFD large on both sides would be the right
answer.  But it's also possible that large packets may only need to be
unidirectionally delivered.

> I
> appreciate that this is exactly the problem that the extensions are
> designed to detect. I am just asking that these issues be discussed more
> explicitly as an aid to the implementor. If that also makes Transports ADs
> happier that is a side benefit - but that's not my motivation.

We're happy to have that in the document.

> > > What might be better?
> > >
> > > 1)Some statement that MTU isn't necessarily a consistent value for all
> > > systems connected to an interface - which can impact the results when large
> > > BFD packets are used. Implementations might then want to consider
> > > supporting "bfd-mtu" configuration and/or iterating across a range of packet
> > > sizes to determine what works and what doesn't.
> > 
> > I'm not clear what you intend by this statement.
> > 
> > Are you asking that we emphasize the use case in a different way?  The
> > Introduction currently states:
> >   "However,
> >    some applications may require that the Path MTU [RFC1191] between
> >    those two systems meets a certain minimum criteria.  When the Path
> >    MTU decreases below the minimum threshold, those applications may
> >    wish to consider the path unusable."
> > 
> > I'm also unclear what "Implementations" may refer to here.  BFD?  An
> > arbitrary user application?  If the latter, the application may not have
> > strict control over the generation of a given PDU size; e.g. TCP
> > applications.
> > 
> 
> [Les:] I am talking about BFD implementations.
> I suppose one can imagine each BFD client requesting a certain MTU value -
> but that wouldn't be my choice.

BFD conversations happen between pairs of devices.  In the case that you
have multiple devices connected to a network segment, each conversation
could (and may intentionally) have different properties.  

An easy example of this is two devices running an IGP may want fast failure
and two other devices running BGP may be happy with just under second-level
failure.  So too could some device decide that it cares about bi-directional
path MTU while the others may not.

Given prior BFD documents' lack of discussion about such multi-access
network considerations, I'm not sure it's in character to have it just for
such a case, if that's what you're concerned with.

> I would think the value we want is really the maximum L3 payload that the
> link is intended to support - which should be independent of the BFD
> client. This might be larger than any client actually uses - but that
> seems like a good thing.

In this case we have actual existence proof of desired behavior.  The links
may be 9k but the user cares only about 1500 bytes end to end. If 1500 bytes
for BFD large works but 9k doesn't, we've not tested what the user actually
desired.

-- Jeff

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Fri, 4 Oct 2019 00:02:04 +0000

Jeff -

For some reason this is proving to be harder than I think it should be.

I keep thinking I am being transparent - yet you keep reading "ulterior motives" into what I say.
There are no ulterior motives.

Let me try again...inline...

> -----Original Message-----
> From: Jeffrey Haas <[email-masked]>
> Sent: Thursday, October 03, 2019 1:13 PM
> To: Les Ginsberg (ginsberg) <[email-masked]>
> Cc: Ketan Talaulikar (ketant) <[email-masked]>; Reshad Rahman
> (rrahman) <[email-masked]>; [email-masked]
> Subject: Re: WGLC for draft-ietf-bfd-large-packets
> 
> Les,
> 
[Les:] What I would like to see discussed are points "a" and "b" below.
This is a section on deployment issues - not a normative part of the spec.

> picturing a problem more dire than that?

[Les:] No. Again, as this is a discussion of deployment considerations I see this as an aid to indicate what problems may be seen.
I am not asking you to "fix" the extension to overcome this.

> Did you mean for BFD or more in the general sense?

[Les:] It is a problem in the general sense, but it is relevant here because the extension proposes to send large packets. Absent that, MTU mismatches would be very unlikely to affect BFD since the BFD packet size is small.

> unidirectionally delivered.

[Les:] I agree - and I think it is valid to use the extension unidirectionally in such cases.

> We're happy to have that in the document.

[Les:] Great!!

> failure.
>  So too could some device decide that it cares about bi-directional

[Les:] I agree. My point was BFD sessions are requested by clients (such as a routing protocol). That client may/may not care about MTU e.g., a routing protocol may not use MTU sized packets.
But if the goal is to validate that MTU sized data traffic can successfully be sent then "someone" has to enable that. And I would argue that the most logical place to enable the feature is under BFD itself since a routing protocol (BGP/OSPF) won't necessarily care about MTU.
Since you are speaking at the "device" level (not BFD client level) I think we are in agreement.

> for BFD large works but 9k doesn't, we've not tested what the user actually

[Les:] This is fine. This is consistent with my suggestion that an implementation supports a "bfd-mtu" knob. This value can be <= link_mtu.

    Les

---

From: Gyan Mishra <[email-masked]>
Date: Wed, 16 Oct 2019 00:46:51 -0400

In-line comment
Sent from my iPhone

> On Oct 3, 2019, at 8:02 PM, Les Ginsberg (ginsberg) <[email-masked]> wrote:
> 
> Jeff -
> 

[Gyan] As far as routing protocols go OSPF does MTU check which has to match or the neighbor will not establish and that is during the initial exchange.  BGP does not have an MTU check however pmtud is generally enabled by default so the MSS should change from default 560 to 1460 on 1500 MTU interfaces and higher on jumbo enabled interfaces but still an MTU mismatch will not prevent the neighbor from coming up.  ISIS also check MTU which has to match as well or the neighbor will not establish. I think this draft is very useful in testing the large jumbo packets throughout bidirectional state with the interface MTU size used in the control packets instead of the default smaller packets which should detect MTU mismatch on point to point links and asynchronous mode or even echo mode won’t bring routing protocol up with the mismatch since now the larger jumbo configured interface MTU is used in the control packets.

---

From: Gyan Mishra <[email-masked]>
Date: Wed, 16 Oct 2019 19:29:12 -0400

All,

I support this draft as I think this would be very useful for IPv6 use
cases where EH headers are utilized excessively such as for an SRv6 use
case for traffic engineering over the internet and would be a method to
test via BFD multihop the path mtu where pmtud has failed to adjust MSS on
endpoints due to firewalls or other devices dropping ICMP unreachable
packet to big  messages resulting in 1280 mtu.

Thank you

Gyan S. Mishra

IT Network Engineering & Technology

Verizon Communications Inc. (VZ)

13101 Columbia Pike FDC1 3rd Floor

Silver Spring, MD 20904

United States

Phone: 301 502-1347

Email: [email-masked]

www.linkedin.com/in/GYAN-MISHRA-RS-SP-MPLS-IPV6-EXPERT

On Wed, Oct 16, 2019 at 12:46 AM Gyan Mishra <[email-masked]> wrote:

> > [Les:] No. Again, as this is a discussion of deployment considerations I
> see this as an aid to indicate what problems may be seen.
> > [Les:] It is a problem in the general sense, but it is relevant here
> because the extension proposes to send large packets. Absent that, MTU
> mismatches would be very unlikely to affect BFD since the BFD packet size
> is small.
> >
> >>
> > [Les:] I agree. My point was BFD sessions are requested by clients (such
> as a routing protocol). That client may/may not care about MTU e.g., a
> routing protocol may not use MTU sized packets.
> > But if the goal is to validate that MTU sized data traffic can
> successfully be sent then "someone" has to enable that. And I would argue
> that the most logical place to enable the feature is under BFD itself since
> a routing protocol (BGP/OSPF) won't necessarily care about MTU.
> > [Les:] This is fine. This is consistent with my suggestion that an
> implementation supports a "bfd-mtu" knob. This value can be <= link_mtu.
> >
> >    Les
>
> [Gyan] As far as routing protocols go OSPF does MTU check which has to
> match or the neighbor will not establish and that is during the initial
> exchange.  BGP does not have an MTU check however pmtud is generally
> enabled by default so the MSS should change from default 560 to 1460 on
> 1500 MTU interfaces and higher on jumbo enabled interfaces but still an MTU
> mismatch will not prevent the neighbor from coming up.  ISIS also check MTU
> which has to match as well or the neighbor will not establish. I think this
> draft is very useful in testing the large jumbo packets throughout
> bidirectional state with the interface MTU size used in the control packets
> instead of the default smaller packets which should detect MTU mismatch on
> point to point links and asynchronous mode or even echo mode won’t bring
> routing protocol up with the mismatch since now the larger jumbo configured
> interface MTU is used in the control packets.
> >>
> >> -- Jeff
> >
>
-- 

Gyan S. Mishra

IT Network Engineering & Technology

Verizon Communications Inc. (VZ)

13101 Columbia Pike FDC1 3rd Floor

Silver Spring, MD 20904

United States

Phone: 301 502-1347

Email: [email-masked]

www.linkedin.com/in/networking-technologies-consultant

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 17 Oct 2019 09:35:33 +0200

Dear WG,

Thank you Gyan for your note.

It very clearly highlights my primary concern expressed earlier of false
assumptions on how engineers may try to (mis)use bfd-large in multihop
cases.

Below note is a brilliant example of how one may not realize that actual
paths BFD packets take can be just a fraction of paths their data plane or
even other control plane packets may traverse over a network or set of
networks.

I am always concerned when protocol extensions being standardized are known
to only work in 1 out of 10 deployment scenarios and when chances of such
opportunity of incorrect use are evident yet no safety inline fuse exist.

Many thx,
Robert.