Subject: [Sidrops] Comments on draft-ietf-sidrops-validating-bgp-speaker

From: Jeff Wheeler <[email-masked]>
Date: Wed, 7 Feb 2018 16:11:58 -0500

A colleague brought the recent draft titled "Signaling Prefix Origin
Validation Results from an RPKI Origin Validating BGP Speaker to BGP
Peers" to my attention.

The authors clearly recognize a hazard of implementing this draft --
trusting the claimed validation state received from an eBGP neighbor.
The draft makes this explicit in both sections 5.2 (..receiving the ..
extended community) and section 7 (security considerations.)

Rather than depend on neighboring networks to configure their devices
appropriately, the draft introduces what I believe is a novel / unique
handling for an extcomm, namely, it is transitive yet only for one
hop, and has to be deleted instead of propagated to any neighbors.

This requirement seems like a significant obstacle to implementation.

It also sets up an incorrect expectation that you should be able to
trust if you receive this extcomm that you got it from the
directly-adjacent BGP neighbor, and not from a distant device.  This
won't be true until the neighbor updates his software, and normally in
BGP, we would expect a Capability to be signaled so implementations
have explicit knowledge of neighbors' relevant BGP features.  The
draft does not call for a Capability.

In light of all this, I think a Well-Known BGP Community makes more
sense than an extcomm.  Operators are already familiar with signaling
and filtering them and this would avoid the need for vendors to write
code for the special propagation scope of the proposed extcomm.

I further believe this draft might be misguided as a whole, but I
imagine the authors, being IXPs, may have their members to answer to;
and are perhaps trying to create transition tools to satisfy everyone.
My colleague reminded me that you must actively mess yourself up to
cause your IP address space to fail RPKI validation.

Perhaps it is unwise for IXP route-servers to propagate invalid routes.

-- 
Jeff S Wheeler <[email-masked]>

---

From: Aris Lambrianidis <[email-masked]>
Date: Wed, 14 Feb 2018 23:22:48 +0100

Thank you for your comments, Jeff.

I see two prongs in your email, a technical one (signalling method) and
a "political" one (for
lack of a better term), i.e. whether this draft is misguided in principle.

Let me put aside the technical component in this email because I think
it's a matter of aligning the draft
with the wisdom of the community, which can happen relatively easily if
we first agree on what
we're trying to solve (i.e. the political issue).

What the draft is trying to accomplish is to standardize the way certain
operators, primarily IXPs,
signal ROA status to their customers by way of a BGP community. Putting
it another way,
if there is no common understanding on how to do this, we run the risk
of operators implementing
this in an ad hoc way, which increases operational overhead on both the
operator signaling
such RPKI data, as well as their customers. This is already a
problematic reality for numerous IXP offerings,
such as statistics and telemetry.

If I am correct in detecting an "outsourcing security is bad" element
when you mention the
draft potentially being misguided, I would argue that yes, I see your
point, but I believe it's generally
better to allow for such a  decision to be made by the customers themselves.

Signalling ROA status in route servers is already happening (France IX,
AMS-IX), for better or worse. 
The question then now becomes, does the community want to consolidate
into one way of doing things or not.

I also have to point out that people are outsourcing security in many
levels already, implicitly, when they use
commodity software, such as operating systems, and hardware, such as
routers and switches. This is just another
level. Any time any one is using an Intel or AMD processor, or any
processor not of their own making,
they are outsourcing security, I hope you agree.

Let me also point out that in theory, forwarding "invalid" prefixes is
bad. However, our own
operational experience so far has shown that most ROA invalid prefixes
are due to misconfiguration
rather than malicious intent.

Kind regards,
Aris

---

From: Job Snijders <[email-masked]>
Date: Thu, 15 Feb 2018 14:39:31 +0100

On Wed, Feb 14, 2018 at 11:22:48PM +0100, Aris Lambrianidis wrote:
> bad. However, our own operational experience so far has shown that
> most ROA invalid prefixes are due to misconfiguration rather than
> malicious intent.

A major consideration is that all IXP route servers only offer a
_partial_ view on the global routing table, nobody ever single-homes
behind an IXP route server. So, if the IXP route server does _not_
propagate invalid prefixes to its participants, operationally speaking
the 'worst' thing that can happen is that alternative paths are used via
either bilateral peering or other arrangements. If anything, if people
depend on keeping their traffic on the IX with the help of the RS, there
is an actual monetary incentive to repair the misconfigured ROAs. If
anything, IXPs should take a leading role in deploying RPKI Origin
Validation - this draft proposes the opposite: a watering down that
justifies propagation of bad announcements.

By merely tagging the invalid prefixes with a community, the sender of
those BGP announcements has significantly visiblity into problem, since
the sender of such announcements won't see the 'invalid community'
themselves.

I think in context of IXP Route Servers it is very detrimental to the
Internet community's overall effort to improve routing security if
invalid malicious announcements are distributed alongside these
'misconfigurations', and the propagation of misconfigurations is deemed
more important than blocking of incorrect announcements.

Given the prevalence of routing incidents and the critical effects they
have on business continuity, I'm concerned that time is wasted on
actively lowering the bar ('rpki-light' aka 'draft-ietf-sidrops-validating-bgp-speaker')
rather than spending time on setting higher standards.

ESPECIALLY given the position of route servers in the routing ecosystem.
If we consider a misconfigured RPKI ROA an 'error', favoring strict
error handling (dropping) over attempting error recovery is an effective
technique for ensuring that faults receive attention. 

Kind regards,

Job

---

From: Job Snijders <[email-masked]>
Date: Thu, 15 Feb 2018 14:44:18 +0100

> the sender of such announcements won't see the 'invalid community'

Oops, some words were missing from the above paragraph:

    By merely tagging the invalid prefixes with a community, the sender
    of those BGP announcements has significantly less visiblity into the
    problem, since the sender of such announcements won't see the
    'invalid community' themselves.

---

From: Jeff Wheeler <[email-masked]>
Date: Thu, 15 Feb 2018 12:37:42 -0500

> lack of a better term), i.e. whether this draft is misguided in principle.

I think your draft has a fair chance of progressing because it's clear
some IXPs (and their members) want route-servers to propagate invalid
paths, and if they are going to keep doing that, it is easy to
understand why the IXPs feel obligated to signal it somehow.  Once you
decided to signal it, a standard is sensible.

Basically, I see you are cutting your own hand.  I think that's stupid
but I support your plan to standardize the type of knife you use.
You're going to spill blood anyway; your members must be demanding it.

For the above reason, I would rather focus on the implementation details.

I'd like to reiterate that the new propagation scope defined by your
draft is a change to the existing, standardized behavior of BGP
Extended Communities.

The nature of this change almost dictates that you also specify a BGP
Capability Code and define a behavior for what you should do when
receiving one of these extcomm from a neighbor that has not signaled
the capability (probably drop the extcomm since its origin is unknown
at that point?)  So now you have a new parameter in the connection
process (no big deal) and new code when parsing and propagating
extcomms.

In fact, you might as well define some kind of router-hops-limit and
as-hops-limit for extcomms because the implementation cost (in
software) of that is essentially the same as what you've already
proposed and the new kind of scope would have other uses to operators.

So the current draft is more complex than it really needs to be.  A
lot of implementation obstacles can be removed by simply using a BGP
Well-Known Community.  If operators want to use your idea they will
need to configure their router, but they would not need to wait on
their vendor for new software.  On your side, you are already using a
community to signal so this clearly works fine.

I understand the long-term advantage hoped for by limiting the
propagation scope, but I think it is a more ambitious and costly plan
than the authors realize.  I ask them to back off from the extcomm, or
in the alternative, approach it as a new propagation behavior to
extcomms which can then be re-used.

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 15 Feb 2018 22:48:39 +0000

>From the bigger picture point of view, this draft is about trying to
make rpki accessible to operators, and builds on ideas first presented
in draft-ietf-sidr-origin-validation-signaling / RFC8097, which applies
to iBGP-only routing environments.

Stepping back a bit and looking at this problem, on almost every bgp
routing platform, rpki policy hooks have been implemented in a way which
makes deployment of rpki quite straightforward.  These hooks rarely
constitute more than a couple of lines for cache configuration, and
simple grammatical constructions for rpki policy matching.  In other
words, the router side of things is pretty easy.  The single item which
makes rpki deployment troublesome is operating an rpki cache or set of
caches.

If you want to deploy RPKI validation caches, there are several options
available: some of the RIRs already provide a hosted rpki validator
service, and RIPE NCC provides open source software with a GUI.
rpki-validator v2 is available now, and v3 is in the works with a
planned release date some time this month.  I.e. this is software which
is used already in operational environments, and which is under full
development support.

This means that the bar for pushing out a full rpki deployment using
actual caches is not very high in practice.  In the one case, all you
need is to create an online account and the whole lot is done for you,
albeit using an infrastructure which you probably wouldn't want to use
as a long term arrangement due to the fragility of depending on off-net
infrastructure (conversely noting that the rpki cache validation
mechanism is designed to be tolerant of situations where the cache is
unavailable).  In the other case, you need a VM or a deployment of VMs
and an amount of time and patience which is going to end up being
directly proportional to the scale of the deployment that you want to
build out.  Neither option involves rocket science, but both will
provide real RPKI, bells and whistles attached.

draft-ietf-sidrops-validating-bgp-speaker proposes to replace actual
RPKI with a synthetic flagging mechanism.  What's lost here is:
cryptographic authenticity, all the rpki validation policy hooks on
routers, and because this draft addresses ebgp rather than ibgp, we have
the extra complication that any upstream processing which involves
multiple bgp paths will also involve bgp best path selection, i.e. the
upstream is necessarily getting into the business of making implicit
routing decisions for the downstream without providing any of the hooks
that the downstream bgp speaker might want to use.

In one sense, this draft is not even about making rpki more accessible:
it's about how to translate a subset of rpki features into existing
policy methodologies, so that the operator doesn't need to use an rpki
cache.

The proposals in draft-ietf-sidrops-validating-bgp-speaker need to be
broken out into two separate sub-cases of multilateral ebgp (i.e. route
servers), and bilateral ebgp (i.e. normal bgp interconnection).

For all ebgp implementations, the moment you examine a community that
comes in over ebgp, you're implicitly trusting your peer to sanitise
that community for you.  In other words, because there is no
cryptographic authentication of the rpki state of the prefix, you're
depending on something which is no more or less reliable than hearsay
and which can be easily faked by tacking on an illegitimate community
and depending on misconfigured routers to forward the attribute.

Route servers present a particular complication because they are a 90%
solution to the problem of how to interconnect with lots of other
organisations at an ixp, with all the massive simplifications that a 90%
solution implies.  The conceptual difficulty with route servers is that
unless they provide add-path tx support (i.e. forward all prefixes to
rsclients so that the clients can make their own decisions about what to
do), they will perform bgp best path selection on behalf of their
clients, i.e. forward a single prefix, losing information in the process.

Job suggests that RPKI invalids should be dropped.  There might be sound
reasons to hold this opinion, but someone else might demand that RPKI
invalids be depreffed instead.  This outlines a fundamental difficulty:
without add-path tx support, the route server is touching the RPKI
problem and making explicit policy decisions on the basis of RPKI
validation results.  The problem with this is that rpki is subtle enough
that if you're going to want to deploy rpki at all, you probably want
enough policy manipulation hooks to get it to do what you want, not what
someone else decides for you.  These hooks would ideally need to specify
what to do with invalids, and unknowns and how to apply various
different attributes to them (e.g. tag, change localpref, change metric,
drop, keep, etc).

This sort of policy management needs to be handled from the route server
provisioning system, ideally dynamically so that client operators don't
need to prod the route server operator to make updates whenever they
want their policy changed.

If this is not present, then the route server is cooking up a bunch of
results based on a stated / implicit policy, but where the rsclient has
no means of inspecting all possible paths to make their own decision
about what to do with them.

As a further complication, the community mechanism proposed in this
draft requires code changes on the rsclient to make it work at all.
This is because generic extcomm support on routers doesn't usually exist.

This can't really be fixed by moving to bilateral peering because if you
move to bilateral, you're going to lose the community support proposed
by this draft, i.e. your peer at the IXP or on PNI probably isn't going
to support this draft.  This means that if you want RPKI, you're back to
square one: you will need to set up your own cache anyway.

The comments in this email
https://www.ietf.org/mail-archive/web/sidrops/current/msg00035.html
(text search term "tl;dr: route servers and rpki are an uncomfortable
fit", date Sat, 14 Jan 2017) are still relevant.

In summary, this draft provides a partial workaround for operators who
don't want to go to the effort of installing and configuring
off-the-shelf RPKI validator software, using a mechanism which will
provide arguably incorrect policy decisions in real world situations
using systems which they have no control over, which requires code
changes on client implementations, which doesn't provide a hook
mechanism to allow the operator to use their router's RPKI grammar, and
which in the best deployment scenario is only ever going to be possible
on a limited set of bgp peering sessions.  I can't reconcile this with
either good quality protocol definition or sound networking operational
procedures.

Regarding the argument that this already exists and that standardisation

> into one way of doing things or not.

Standardising different implementations of a bad idea doesn't make it a
better idea.  If some IXPs are tagging prefixes on their route server
with rpki assessment information, then that's a decision for them to make.

I'm sorry if this assessment comes across as being harsh, but given the
limitations of what's being proposed, it would probably be more
constructive to use the time and effort that would otherwise be spent
working on this draft to educate operators about how relatively easy it
is to implement native rpki, and that if operators are serious about
rpki on route servers, then they will need either add path tx on the
route server or else use bilateral peering.

Nick