Subject: Re: [v6ops] New Version Notification for draft-palet-v6ops-p2p-from-customer-prefix-00.txt

From: Mark Smith <[email-masked]>
Date: Thu, 2 Nov 2017 07:35:27 +1100

Hi Ole,

On 30 October 2017 at 08:49, Ole Troan <[email-masked]> wrote:
> Mark,
>
>> On 29 Oct 2017, at 22:27, Mark Smith <[email-masked]> wrote:
>>
>> I think ND should be used on PPP,/POS etc. links for that reason. It ensures the address is present on the link before trying to send a packet to it. RFC4861 seems to be saying that because it says it applies to all link types, and it only makes an exception of NBMA links.
>
> The IPv6 implementations I work on, only does address resolution on link types which have L2 addresses.
>
> There’s nothing stopping you from doing it on those links, but I can certainly find no requirement to do it in 4861 either.
>

While I agree it is a little ambiguous, I think there are direct
statements and statements that make implications that ND as described
in RFC4861 is to be used over true p2p links, with ND meaning more
than just link layer address resolution - ND meaning the description
in the Abstract and Introduction i.e.

"IPv6 nodes on the same link use Neighbor Discovery to
   discover each other's presence, to determine each other's link-layer
   addresses, to find routers, and to maintain reachability information
   about the paths to active neighbors."

"... Finally, nodes use the protocol to actively keep track
   of which neighbors are reachable and which are not, and to detect
   changed link-layer addresses."

I think the two key statements are "discover each other's presence"
and "actively keep track
   of which neighbors are reachable and which are not"

if "address resolution" isn't performed on a true p2p link, then a
node is also not discovering a neighbor's presence. If it isn't
discovering a neighbor's presence, then it can't actively track
whether the neighbor is reachable or not. The ND NS/NA transaction
provides both the link-layer address of the neighbor if it has one,
and positive confirmation of the existence or presence of the address.

I think there are more statements that imply ND NS/NAs are expected to
be used over true p2p links.

1.  Introduction

"Unless specified otherwise (in a document that covers operating IP
   over a particular link type) this document applies to all link types."

There is no separate RFC about running ND over p2p links, so RFC4861
applies to true p2p links.

"However, because ND uses link-layer multicast for some of its
   services, it is possible that on some link types (e.g., Non-Broadcast
   Multi-Access (NBMA) links), alternative protocols or mechanisms to
   implement those services will be specified (in the appropriate
   document covering the operation of IP over a particular link type)."

The exception link types seem to be links that don't support multicast
and are NBMA links.

2.2.  Link Types

"point-to-point - a link that connects exactly two interfaces.  A
                    point-to-point link is assumed to have multicast
                    capability and a link-local address."

So true p2p links are assumed to be multicast capable links, meaning
they don't fall into the NBMA link exception.

7.3.  Neighbor Unreachability Detection

   "...  Thus, a node actively tracks the reachability "state" for the
neighbors to which it is sending
   packets.

   Neighbor Unreachability Detection is used for all paths between hosts
   and neighboring nodes, including host-to-host, host-to-router, and
   router-to-host communication.  Neighbor Unreachability Detection may
   also be used between routers, but is not required if an equivalent
   mechanism is available, for example, as part of the routing
   protocols."

No mentioned exclusions for p2p or any type of link. Very explicitly
states that nodes' track neighbor reachability.

The exception in the second paragraph is covering using something like
OSPF's hello protocol as a substitute to track neighbor
reachability/unreachability. However, static routing or DHCPv6-PD may
be being used over the link to "distribute" routing information, and
neither of them have a neighbor presence detection protocol, so NUD
has to be used. Unreachability detection requires discovery of
reachability.

There are two choices. Either prevent the packet loop occurring if a
packet is sent to a destination that doesn't exist, by probing for its
existence via ND NS, or mitigate the consequences of the loop after it
has already occurred. I think prevention is better than mitigation.

Regards,
Mark.

---

From: Ole Troan <[email-masked]>
Date: Wed, 1 Nov 2017 22:05:02 +0100

Mark,

I most definitely think doing address resolution on links without link-layer addresses is the wrong thing to do.
Sure, other ND functions are being used. NUD isn't typically useful on router to router links. Often p2p links have link-layer notification on link state too, and I'd argue that the benefit of a router doing NUD towards hosts have limited usefulness, unless it is to garbage collect state.
If we ever get to update NUD, I would argue quite hard for simplification of the protocol.

> has already occurred. I think prevention is better than mitigation.

If this is the problem you are trying to solve, then we have already solved that.
See rfc4443, from changelog:

    - In the description of Destination Unreachable messages, Code 3,
     added rule prohibiting forwarding of packets back onto point-to-
     point links from which they were received, if their destination
     addresses belong to the link itself ("anti-ping-ponging" rule).

Was there any other unresolved problem I missed?

Cheers,
Ole

---

From: Mark Smith <[email-masked]>
Date: Tue, 7 Nov 2017 18:15:11 +1100

Hi Ole,

> I most definitely think doing address resolution on links without link-layer addresses is the wrong thing to do.

Neighbor Discovery is more than just address resolution. It does two things:

1. discovers and proves the presence of the neighbor (or more
accurately, the presence of the address being discovered)
2. resolves the link-layer address if the link uses them.

If the only purpose of Neighbor Discovery was address resolution, then
RFC4861 would be called "IPv6 Address Resolution", and would certainly
have text saying clearly that it didn't apply to point-to-point links
without link-layer addresses.

> Sure, other ND functions are being used. NUD isn't typically useful on router to router links. Often p2p links have link-layer notification on link state too, and I'd argue that the benefit of a ro…

That state can be very useful for troubleshooting. It provides a table
of devices/addresses present on the link. Because the presence of
devices is discovered and tracked, other functions such as security
functions can leverage it.

> Was there any other unresolved problem I missed?

So that's a solution to the problem not performing neighbor discovery
on p2p links created.

I assume not performing neighbor discovery on p2p links is thought to
be an optimisation. However it also created an exception. Instead of
all of the ND functions in RFC4861 being performed on all links,
leaving out the Source/Target Link-layer Address NS/NA option when it
isn't necessary, we now have two distinct classes of links:

- links for which neighbors and their IPv6 addresses are discovered
- links for which neighbors and their IPv6 addresses aren't discovered

IPv6 seems to have had a goal of generalisation (e.g., link-layer ARP
protocol functions moved into ICMPv6, address and parameter setting
moved out of PPP IPCP). Treating all links as similarly as possible,
hiding the minor differences (which in the case of p2p links is just
leaving out the "Source/Target Link-layer Address NS/NA option" and
related processing), is simpler and more general. Much of the text I
provided from RFC4861 reads to me as though generalisation of
different link type handling is the goal.

This optimisation then created issues that then caused RFC4443 to have
to be modified to suit. It also resulted in RFC6164 which implies it
fixes the problem, however as it doesn't discover if the remote
address in the /127 exists, a ping-pong loop can still form.

 If the optimisation causes issues that have to be worked around
somewhere else, and the optimisation doesn't provide an obvious
benefit, is it really a useful and worth while optimsation?

Regards,
Mark.

---

From: Gert Doering <[email-masked]>
Date: Tue, 7 Nov 2017 21:00:21 +0100

Hi,

> benefit, is it really a useful and worth while optimsation?

people point routing towards p2p *links* and not "next-hops" because
they do not care, and do not *want* to care what the neighbouring IP
address is.

"ipv6 route 2001:db8::/48 serial3"

is legit and useful.  Starting to do ND for every single address in that
/48 would be totally the wrong way to do.

The fact that there is no link layer resolution on p2p links is a plus,
trying to make it "look like ethernet" would be a major drawback (and
I strongly defute any claim that "the generalization of ARP into IPv6 
ND is a *good thing*" - L2 resolution using a routable protocol and relying 
on implementations to ensure that it's then "not routed" or "properly
filtered of TTL is not 255" was just plain stupid)

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Mark Smith <[email-masked]>
Date: Wed, 8 Nov 2017 18:12:42 +1100

On 8 Nov. 2017 7:00 am, "Gert Doering" <[email-masked]> wrote:

Hi,

ND is a *good thing*"

It already happened.

- L2 resolution using a routable protocol and relying
on implementations to ensure that it's then "not routed" or "properly
filtered of TTL is not 255" was just plain stupid)

Just plain stupid is thinking you always know best.

Gert Doering
        -- NetMaster

---

From: Erik Kline <[email-masked]>
Date: Wed, 8 Nov 2017 16:52:45 +0900

> I strongly defute any claim that "the generalization of ARP into IPv6

I'm not sure what is meant by the claims here, but in case it helps:
Linux's ARP neighbor reachability statemachine entirely reuses/shares
the IPv6 ND neighbor reachability statemachine (i.e. all of the IPv6
NUD state transitions are also made by the (same) code monitoring IPv4
neighbors).

> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops
>

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 08 Nov 2017 10:31:20 +0000

> Just plain stupid is thinking you always know best.

Perhaps v6ops is not the best place for dismissive personal comments?

Gert has sound technical reasons for making the statement that he did,
namely that there are no particular advantages conferred by using icmpv6
+ multicast as the on-wire protocol for v6 layer 2 resolution, but there
are significant complications associated with doing so.  In addition to
what he mentioned above, these complications also include the
requirements for: functional multicast support on layer 2 devices;
having to inspect either multiple locations or across a very wide
bitmask in a frame in order to determine whether the frame is a
legitimate nd packet; and for lumping multiple types of functionality
into the icmpv6, making line-wire frame filtering cumbersome and
troublesome to get right.

You are, of course, welcome to disagree with these opinions, but it
would be helpful if the conversation could remain civil in the process.

Nick

---

From: Mark Smith <[email-masked]>
Date: Thu, 9 Nov 2017 00:20:26 +1100

Gert provided no reasons for his assertions other than popularity, which is
really just an appeal to authority. You have provided some of the arguments
he should have.

Furthermore, he chose to use perjoritive and dismissive terms such as
"totally wrong" and "plain stupid".

I rarely and try to rarely dismiss arguments I disagree using absolutes and
perjoritives such as those Gert used. I've been disagreeing with Ole, yet
you'll find that I have not dismissed Ole's position the way Gert dismissed
mine. Ole has also been respectful in his responses to me, even though he
has been disagreeing.

Regards,
Mark.

On 8 Nov 2017 9:31 pm, "Nick Hilliard" <[email-masked]> wrote:

Nick

---

From: Gert Doering <[email-masked]>
Date: Wed, 8 Nov 2017 14:25:32 +0100

Hi,

> really just an appeal to authority. You have provided some of the arguments

Since we're in v6*ops*, it should not be necessary to repeat the 
shortcomings of ND ad nauseam.  I was referring to your claim that
"make every link look the same" was a good decision, and that ND is
better than ARP, which the last 20 years have shown to not be.

> "totally wrong" and "plain stupid".

Obviously it's easier in hindsight, but from 2017's point of view, yes
indeed, some early design decisions of IPv6 have not been smart, and that
is plainly visible.  So, yes, plain stupid.

Gert Doering
        -- NetMaster

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 08 Nov 2017 13:32:01 +0000

Mark Smith wrote:
> Gert provided no reasons for his assertions other than popularity, which
> is really just an appeal to authority. You have provided some of the
> arguments he should have.

This isn't the first time this topic has come up.  In fact, it's been
discussed to death over the years, and Gert has talked about it on many
occasions before.

> "totally wrong" and "plain stupid".

Those comments were made about the decision and the protocol rather than
"thinking you always know best", which is a comment made about a person.
 Making pejorative comments about a person is not generally considered
acceptable in the context of working groups, IETF or otherwise.

Nick

---

From: Philip Homburg <[email-masked]>
Date: Wed, 08 Nov 2017 14:49:02 +0100

>better than ARP, which the last 20 years have shown to not be.

This (ND and p2p links) is a confusing subject. 

When I first read the ND specs, I found it a ridiculous approach compared to
ARP. However, if you jump through all the hoops, it seems to work as well as 
ARP. No real issue there. Other then just too much complexity.

On point-to-point links the situation is different. Obviously, there is no
need for link layer address resolution. However, ND also provides neighbor
reachability information.

On a 'normal' p2p link that should be redundant. The routing protocol should
take care of that.

However, for example, the CPE spec defines the upstream interface of a CPE
to be a host. 

And as a final bit of complexity, I found that on quite a few links I had to 
disable ND because to other side doesn't want to play along.

So one way forword is to specify that for p2p links, certainly between two
routers, ND should be disabled.

---

From: Gert Doering <[email-masked]>
Date: Wed, 8 Nov 2017 14:59:16 +0100

Hi,

> ARP. No real issue there. Other then just too much complexity.

Then, after 10 years of "ok, it seems to work", you get a router vendor
advisory that control-plane rate-limiting for ND does not work right
(garbage drowning out expected packets in the rate-limited queue from
"fast forwarding hardware" towards the CPU).  Then you notice that these
are packets that have an on-link-only role, but can be nicely sent 
off-link, and receivers are expected to do TTL validation on receipt,
instead of "just using a protocol that *cannot* be sent off-link".

And *then* you figure out that for a single protocol on a single link
you need to maintain multiple lines of ACL because source and destionation
addresses can be lots of interesting combinations of ::, link-local and
global addresses.

(And after that, you notice that one of the more widespread router
vendors out there happily forwards packets with a link-local source
and an off-link global destination, so not even the "we only permit 
::, LLA or on-link source" ACL rule will be able to catch all the garbage)

Up to that point, I also thought "yeah, a bit complicated, but seems to 
work okayish, as long as you remember to permit ND through your interface 
ACLs"...  (well, except for the state exhaustion attacks, the multicast,
and all that).

Good work.

Gert Doering
        -- NetMaster

---

From: Philip Homburg <[email-masked]>
Date: Wed, 08 Nov 2017 15:25:09 +0100

>ACLs"...  (well, except for the state exhaustion attacks, the multicast,

Comes with built-in job security.

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 08 Nov 2017 14:52:14 +0000

> Comes with built-in job security. 

I'd happily trade job security for protocol simplicity.  There are
plenty of other interesting problems in life.

Nick

---

From: Philip Homburg <[email-masked]>
Date: Wed, 08 Nov 2017 15:58:21 +0100

>plenty of other interesting problems in life.

Even on day one, it was already clear that ND was a lot more complex than ARP.

Then we discovered how complex it really was. 

And due to the dislike of DHCPv6, it will get even more complex.

But I guess it is bit too late to propose that we dump the whole mess and 
go back to ARP and DHCP.

---

From: Gert Doering <[email-masked]>
Date: Wed, 8 Nov 2017 20:30:42 +0100

Hi,

> But I guess it is bit too late to propose that we dump the whole mess and 

Propose, we can...  succeed, unlikely.

Gert Doering
        -- NetMaster