Subject: [OPSEC] Last Call: <draft-ietf-opsec-lla-only-07.txt> (Using Only Link-Local Addressing Inside an IPv6 Network) to Informational RFC

From: The IESG <[email-masked]>
Date: Mon, 24 Mar 2014 08:51:35 -0700

The IESG has received a request from the Operational Security
Capabilities for IP Network Infrastructure WG (opsec) to consider the
following document:
- 'Using Only Link-Local Addressing Inside an IPv6 Network'
  <draft-ietf-opsec-lla-only-07.txt> as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
[email-masked] mailing lists by 2014-04-07. Exceptionally, comments may be
sent to [email-masked] instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract

   In an IPv6 network it is possible to use only link-local addresses on
   infrastructure links between routers.  This document discusses the
   advantages and disadvantages of this approach to help the decision
   process for a given network.

The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-opsec-lla-only/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-opsec-lla-only/ballot/

No IPR declarations have been submitted directly on this I-D.

---

From: Randy Bush <[email-masked]>
Date: Wed, 26 Mar 2014 07:40:58 +0900

this documents a bad idea.  you may notice that it is a vendor, not
operator document.  it makes the network harder to operate, harder to
debug, ...

please do not let this out without a toxic warning sign.

randy

---

From: "Adrian Farrel" <[email-masked]>
Date: Tue, 25 Mar 2014 22:48:28 -0000

Hi Randy,

For my enlightenment, are you saying:

a. The benefits are over-hyped in this document and the issues with drawbacks
are not sufficiently highlighted.

b. The very existence of this document is a problem

I think the answer to that might guide how the community responds to your
review.

Thanks,
Adrian

> -----Original Message-----
> From: iesg [mailto:[email-masked]] On Behalf Of Randy Bush
> Sent: 25 March 2014 22:41
> To: The IESG
> Cc: [email-masked]
> Subject: Re: Last Call: <draft-ietf-opsec-lla-only-07.txt> (Using Only
Link-Local
> Addressing Inside an IPv6 Network) to Informational RFC
>

---

From: Ted Lemon <[email-masked]>
Date: Tue, 25 Mar 2014 17:49:15 -0500

> operator document.  it makes the network harder to operate, harder to

It does this because...?

Last call review comments are greatly appreciated, _if_ they raise technical issues; it sounds like you think there's a technical issue here, but since you haven't said what it is, you aren't giving us any reason to take you seriously.

---

From: Randy Bush <[email-masked]>
Date: Wed, 26 Mar 2014 08:07:15 +0900

>    are not sufficiently highlighted.

yes

> b. The very existence of this document is a problem

in all honesty, i think this should be junked.  but that is unrealistic.
so i will settle for a toxic label.

randy

---

From: Randy Bush <[email-masked]>
Date: Wed, 26 Mar 2014 09:06:03 +0900

for those who seem not to have followed the ops discussions on 1918
links, this how operators think

From: [email-masked]
Subject: Re: Link addressing (was: Re: A Plea for Architectural & Specification Stability with IPv6)
To: [email-masked]
Cc: [email-masked]
Date: Tue, 25 Mar 2014 09:46:26 +0100 (CET)

> The question I've wondered a bit about is what is the common
> operational or implementation practice when configuring manual
> addresses on routers.
> 
> For people who configure static addresses on router interfaces, do
> they actively switch SLAAC off for either individual prefixes, or
> for the whole interface?

Answering only for myself, obviously:

- We use static /124 addresses on our router-router links, which means
we're pretty darn sure that SLAAC won't be working for anything other
than the link local address. We may switch to /127.
- Additionally, Juniper routers need explicit configuration to turn on
RA. We definitely don't configure this on our router-router links.
- Additionally, we're planning to explicitly turn off SLAAC on Cisco
routers for our router-router links.

> Alternatively, if SLAAC is left enabled for the LL prefix, then
> there is likely to now be two LLs on the interface. Static
> configuration of an LL address would usually indicate preference for
> its use for all LL traffic over the SLAAC LL address, which may be a
> problem, because IIRC, RFC6724 doesn't place any preference for
> static addresses over SLAAC addresses.

The static IPv6 addresses that we configure on router-router links are
*not* link local. Thus we don't have two link local addresses on the
interfaces.

Steinar Haug, AS 2116

---

From: Ted Lemon <[email-masked]>
Date: Tue, 25 Mar 2014 19:57:04 -0500

> links, this how operators think

That's better than the previous response, but still doesn't communicate a compelling issue.  You're statically configuring IP addresses on the /124; that's no easier or harder than statically configuring IP addresses link-locally.

I don't mean to say that you are wrong—just that you're not giving us much to work with.

---

From: Gert Doering <[email-masked]>
Date: Wed, 26 Mar 2014 11:21:30 +0100

Hi,

On Tue, Mar 25, 2014 at 07:57:04PM -0500, Ted Lemon wrote:
> I don't mean to say that you are wrong?just that you're not giving us much to work with.

If you go back, I have commented on that draft that we'll never ever go
link-local-only, because the drawbacks of not being able to specifically
ping one particular interface from the management system, and not having
traceroute tell me on which exact path my packets are flowing are 
not acceptable.   YMMV on this, of course.

I don't particulary object to having a RFC out there that says "it can
be done, but this is just one option, and drawbacks exist" - which the
text does.  I just won't ever *do* that.

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Randy Bush <[email-masked]>
Date: Wed, 26 Mar 2014 19:23:20 +0900

> text does.  I just won't ever *do* that.

the problem is that the naive will fall into the hole.  hence my
assertion that it needs the toxic warning.

---

From: Brian Haberman <[email-masked]>
Date: Wed, 26 Mar 2014 08:41:57 -0400

Hi Randy,

> assertion that it needs the toxic warning.

Let me be explicit for those who need it... Operating with only
link-local addresses isolates the devices from management systems
located in a NOC (i.e., not directly adjacent).  A typical first-step
diagnostic when something goes wrong is to ping the address of the
suspect interface *from the NOC*.  That can't be done if the pesky
device only has a link-local address.

I agree with Randy's pov that this document has some serious issues.

Regards,
Brian

---

From: Ted Lemon <[email-masked]>
Date: Wed, 26 Mar 2014 08:43:18 -0500

> not acceptable.   YMMV on this, of course.

Thanks, that's exactly the detail I was missing.   It's obvious in hindsight, but didn't jump out at me based on Randy's comment.   :}

---

From: Mikael Abrahamsson <[email-masked]>
Date: Wed, 26 Mar 2014 16:10:01 +0100 (CET)

On Wed, 26 Mar 2014, Ted Lemon wrote:

> Thanks, that's exactly the detail I was missing.  It's obvious in 
> hindsight, but didn't jump out at me based on Randy's comment.  :}

2.3 in the draft:

"Interface ping: if an interface doesn't have a routable address, it
    can only be pinged from a node on the same link.  Therefore, it is
    not possible to ping a specific link interface remotely."

So I don't really understand what Randy has against this document and what 
other warnings he wants to have in it. What is this "toxic" warning Randy 
wants? Randy, please enlighten me?

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Erik Muller <[email-masked]>
Date: Wed, 26 Mar 2014 12:54:11 -0400

> wants? Randy, please enlighten me?

Well, not attempting to speak for Randy, but I'd like to see a more 
balanced introduction (it summarizes the benefits; it should not just defer 
the caveats to the body), and maybe some discussion on where it might be 
useful to deploy a network using lla-only (homenets or small meshy things? 
  I can't think of any enterprise or carrier scenario where I would want to 
do this), and clearer statement that this may not be for everybody and is 
not a general BCP.

And while the caveats hint at it, there's also an operational complexity 
burden that isn't called out - the ping and NMS/discovery limitations also 
apply to human operators troubleshooting faults and attempting to 
understand a deployed topology.  LLDP and NDP add a layer of indirection in 
identifying what devices should be adjacent to a given interface, and only 
work when there is operational state available and links are up (whereas 
GUAs on interconnected devices can be compared by configuration alone, 
telling you what's supposed to be there).

-e

---

From: Randy Bush <[email-masked]>
Date: Thu, 27 Mar 2014 06:31:22 +0900

> other warnings he wants to have in it.

what is missing is "therefore all monitoring, diagnostic, management,
... tools will not work.  you will be unable to maintain your network.

---

From: Christopher Morrow <[email-masked]>
Date: Wed, 26 Mar 2014 17:38:58 -0400

> ... tools will not work.  you will be unable to maintain your network.

unless you can always be assured of testing from the device which is
having problems.

You could imagine an NMS/tool-set which knows: "Login to device before testing!"
This also doesn't work of routing protocols for the device fail...

-chris
(for the record I also think it's folly to do the process in the doc,
but figured grownups understood that, and less-grown-up-folks would
listen or learn the hard way)

---

From: Randy Bush <[email-masked]>
Date: Thu, 27 Mar 2014 06:54:41 +0900

> This also doesn't work of routing protocols for the device fail...

i am in the noc 20 router hops away.  so the code has to go hop by hop,
logging in to every router en route.  the sec folk will love that.  you
should talk to your sec folk.  oh, you are your sec folk.

randy

---

From: Randy Bush <[email-masked]>
Date: Thu, 27 Mar 2014 08:10:19 +0900

> I agree with Randy's pov that this document has some serious issues.

from my warped pov, the draft is a sell job which does mention many
basic issues, but completely omits their really bad consequences.

randy

---

From: Randy Bush <[email-masked]>
Date: Thu, 27 Mar 2014 10:34:34 +0900

> I'm, again, not arguing FOR this configuration, just saying you could
> make it work, at a price of longer outages (most likely) and more
> (much MUCH more) complexity in your operations. I don't see what LLA
> gets you that:
>   1) put all your ptp/loops into 1 aggregate
>   2) do not announce the aggregate (internally (see schilller paper)
>      nor externally)
>   3) acls on the edge that drop traffic destined to your ptp/loops addresses.
> 
> complexity is going to cause you pain, it is going to cause you
> problems and it is going to lengthen outages :( avoid complexity.

agree.  hence i am of the opinion that the class of configuration in the
draft should be clearly labeled as dangerous and ill-advised.

though you might be arguing for its use by masochists and isps who want
to lose customers. :)

randy

---

From: Christopher Morrow <[email-masked]>
Date: Wed, 26 Mar 2014 21:31:19 -0400

> should talk to your sec folk.  oh, you are your sec folk.

So, I was presuming that the LLA was ONLY for the ptp links, and that
loopbacks would still be standard old addresses like today.

If the above is the case (my presumption) then you have access to the
local-lo0 equivalent as along as your IGP is healthy, and you're ok.
Of course, as soon as you pooch your IGP you're sunk :(

-chris

---

From: Christopher Morrow <[email-masked]>
Date: Wed, 26 Mar 2014 21:39:52 -0400

> draft should be clearly labeled as dangerous and ill-advised.

sure, a clear warning that: "Doing this is loading the double-barrel
and aiming it clearly at your thigh!" seems ok to me.

> though you might be arguing for its use by masochists and isps who want

"I encourage my competitors to do this..."
  --<internet operator curmudgeon>

-chris

---

From: Paul Ferguson <[email-masked]>
Date: Wed, 26 Mar 2014 20:26:45 -0700

On 3/26/2014 6:39 PM, Christopher Morrow wrote:

> "I encourage my competitors to do this..." --<internet operator
> curmudgeon>

It's funny how we all start to agree on a lot of things after a lot of
years.  :-)

- - ferg

- -- 
Paul Ferguson
VP Threat Intelligence, IID
PGP Public Key ID: 0x54DC85B2

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 27 Mar 2014 08:10:36 +0100 (CET)

> ... tools will not work.  you will be unable to maintain your network.

Since the loopback address still works to poll SNMP from, ssh to, etc, I 
don't agree with your statement. You have to use extra special care if you 
want to number using LL, but stating that it *won't work at all* isn't 
correct in my book.

There are people who number P-T-P links using "unnumbered" between CE and 
PE today. This works for them. You just have to know the caveats, which is 
what the documents tries to describe. If there are caveats not listed, do 
tell and let's include them.

---

From: Lee <[email-masked]>
Date: Thu, 27 Mar 2014 10:42:23 -0400

On 3/27/14, Mikael Abrahamsson  wrote:

> PE today. This works for them. You just have to know the caveats ...

Is this a fair summary of the caveats mentioned earlier?[1]

- not being able to specifically ping one particular interface from
the management system
- not having traceroute tell me on which exact path my packets are flowing
- NMS/discovery limitations
- attempting to understand a deployed topology
- as soon as you pooch your IGP you're sunk
- longer outages and more complexity in your operations.

Personally, I think 'attempting to understand a deployed topology' is
a bit understated.  The ability to match up interfaces on different
routers & do off-line analysis can be huge.  Anyone that hasn't seen
  Managing IP Networks with Free Software
  https://www.nanog.org/meetings/abstract?id=785
should go look now.

And to ratchet the level of hyperbole down a notch, enabling uRPF can lead to
- not being able to specifically ping one particular interface from
the management system
- not having traceroute tell me on which exact path my packets are flowing
- NMS/discovery limitations
and yet enabling uRPF is generally considered A Good Idea.

It's a trade-off; at work we have a requirement for either uRPF or an
input access list on "user subnet" interfaces.  I got lazy, enabled
uRPF and had someone at my desk two days ago saying they couldn't ping
the HSRP interface address.  ^shrug^  the input access list is a pain
to create and manual verification is so error-prone that you really
have to have a program that looks at the config to verify correctness.
 The point being that I went with uRPF _knowing_ it was going to cause
some small level of operational complexity/breakage (oh Noes!! I can't
ping ...) because I thought the alternative was worse.

Which is a very long intro to the section 2.3 caveats list should also include

off-line analysis:  using LLA interface addresses mean that you can
not do an off-line analysis of the deployed topology or do consistency
checks between interfaces on each side of a link.

- as soon as you pooch your IGP you're sunk
should be listed but darn if I can come up with the wording

- longer outages and more complexity in your operations.
I'm leaning towards "value judgement" and so shouldn't be listed as a caveat

Lee

[1] snippets I saved for generating the caveats summary list:

   Brian Haberman

Well, not attempting to speak for Randy, but I'd like to see a more
balanced introduction (it summarizes the benefits; it should not just
defer the caveats to the body), and maybe some discussion on where it
might be useful to deploy a network using lla-only (homenets or small
meshy things?  I can't think of any enterprise or carrier scenario
where I would want to do this), and clearer statement that this may
not be for everybody and is not a general BCP.

And while the caveats hint at it, there's also an operational
complexity burden that isn't called out - the ping and NMS/discovery
limitations also apply to human operators troubleshooting faults and
attempting to understand a deployed topology.  LLDP and NDP add a
layer of indirection in identifying what devices should be adjacent to
a given interface, and only work when there is operational state
available and links are up (whereas GUAs on interconnected devices can
be compared by configuration alone, telling you what's supposed to be
there).
 Erik Muller

>     what is missing is "therefore all monitoring, diagnostic, management, ... tools
> will not work.  you will be unable to maintain your network.

Since the loopback address still works to poll SNMP from, ssh to, etc,
I don't agree with your statement. You have to use extra special care
if you want to number using LL, but stating that it *won't work at
all* isn't correct in my book.

There are people who number P-T-P links using "unnumbered" between CE
and PE today. This works for them. You just have to know the caveats,
which is what the documents tries to describe. If there are caveats
not listed, do tell and let's include them.
  Mikael Abrahamsson

---

From: joel jaeggli <[email-masked]>
Date: Thu, 27 Mar 2014 08:22:43 -0700

>   3) acls on the edge that drop traffic destined to your ptp/loops addresses.

Having had the opportunity to do this, for better or worse several times
that's basically what I've done, drop traffic bound for the interior
infrastructure aggregate via ingress acls.

I think when were were discussing this in the w.g. that it has the
potential to make things such as recursive next-hop calculation much
harder since using link-local-scope addresses that aren't on-link for
you would seem problematic. maybe using the loopback is an acceptable
alterntive for some applications but you loose access to some tools this way

---

From: "Ivan Pepelnjak" <[email-masked]>
Date: Thu, 27 Mar 2014 16:29:01 +0100

> On 3/26/14, 6:31 PM, Christopher Morrow wrote:
[...]
> I think when were were discussing this in the w.g. that it has the potential to
> make things such as recursive next-hop calculation much harder since using
> link-local-scope addresses that aren't on-link for you would seem problematic.

I wonder what happens when you run BGP over LLA and use "neighbor next-hop-self" (which many people do on AS edge routers) or have no next hop in local BGP table (due to aggregation or static route to null 0) ...

Regardless of all the "benefits" I still think this is a bad idea.

Ivan

---

From: Christopher Morrow <[email-masked]>
Date: Thu, 27 Mar 2014 14:24:29 -0400

> potential to make things such as recursive next-hop calculation much

yup, certainly... you probably have to think long and hard about any
standard advice from vendors/third-parties about configuration
optimizations ;(

> alterntive for some applications but you loose access to some tools this way

see my comment about complexity... oh, included below, how nice of you :)

-chris

---

From: Gert Doering <[email-masked]>
Date: Thu, 27 Mar 2014 22:01:24 +0100

Hi,

> and yet enabling uRPF is generally considered A Good Idea.

Actually, enabling uRPF on core-to-core interfaces is considered a
significantly stupid idea.  You enable uRPF towards your customers, 
and loose(!) uRPF towards peers/upstreams for BGP-RTBH.

None of these will impair traceroute or NMS/discovery.

Gert Doering
        -- NetMaster

---

From: Lee <[email-masked]>
Date: Fri, 28 Mar 2014 10:22:45 -0400

Hi Gert,

> and loose(!) uRPF towards peers/upstreams for BGP-RTBH.

Is there an RFC that spells it out that clearly?   I couldn't find
anything about 3 years ago when an auditor from our security office,
using the CIS Secure IOS benchmark, decided that every router
interface needed uRPF enabled.  You want fun, try explaining
asymmetrical routing to an auditor.

> None of these will impair traceroute or NMS/discovery.

If you only do uRPF at the edge - right, it doesn't impair traceroute
but it did give us grief with NMS/discovery:

    NMS  10.10.10.10
           |
          core
        /    \
       /      \
  distA        distB
    |10.1.1.3    | 10.1.1.4
    |            |
    --- access ---

When the access layer switch says it's neighbors are 10.1.1.3 and
10.1.1.4 we had problems.  Maybe it was just the discovery software
being stupid, but we ended up putting every device into the discovery
seed file.

Lee

---

From: Gert Doering <[email-masked]>
Date: Fri, 28 Mar 2014 18:17:54 +0100

Hi,

On Fri, Mar 28, 2014 at 10:22:45AM -0400, Lee wrote:
> Is there an RFC that spells it out that clearly?   

Well, I never searched.  I find that so blindingly obvious that I never
assumed someone would want to do that, and not understand why it hurts.

> I couldn't find

Yeah, I can feel your pain.  OTOH in certain situations there's not
much you can do except "call his supervisor, declare the auditor to
be defective, RMA".

> but it did give us grief with NMS/discovery:

Mmmh.  Unless something weird comes into play here, I can't see how
uRPF could interfere if none of the links you have shown (core<->dist<->
access) has uRPF enabled...

Gert Doering
        -- NetMaster

---

From: Jared Mauch <[email-masked]>
Date: Fri, 28 Mar 2014 13:35:11 -0400

>>> problems and it is going to lengthen outages :( avoid complexity.

I think this is the critical element, many things here will cause pain.

Using link-local for these purposes has so many unintended side-effects
it's not something anyone rational will deploy.  Just because I can stick
a pencil in my eye doesn't mean it's a good idea nor something I should
write a how-to document for.  I have a redundant eye, but at some point
the consequences are likely to be quite catastrophic.

- Jared

---

From: Lee <[email-masked]>
Date: Fri, 28 Mar 2014 15:49:23 -0400

> assumed someone would want to do that, and not understand why it hurts.

Which is why I tried to summarize the caveats for this draft earlier.
Sometimes it seems like the experts think something is so blindingly
obvious they don't need to document it -- which leaves the non-experts
to blindly follow "best practices" or learn the hard way.

> much you can do except "call his supervisor, declare the auditor to

I tried that :)

> access) has uRPF enabled...

Sorry - I grabbed an old msg describing the problem and cut a bit too
much.  The access layer is an L2 switch so uRPF is enabled on the
10.1.1.3 and 10.1.1.4 interfaces at the distribution layer.

The nms machine tries to ping 10.1.1.3  (or tries to discover 10.1.1.3
- same problem)
The core router sees two equal cost paths to the 10.1.1 network.
If core routes the packet to distB the unicast RPF check passes on
distB (source addr 10.10.10.10 is routed via distB-core link) and
distB forwards the packet over the 10.1.1 access network to distA.
The RPF check fails on distA because the route to 10.10.10.10 is via
the distA-core link, not via the 10.1.1 access network interface that
the packet came in on.

Lee