Subject: [Sidrops] ASPA: Is this really a leak?

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Tue, 15 Dec 2020 06:45:43 +0000

https://tools.ietf.org/html/draft-ietf-sidrops-aspa-verification-06
finds suspected leaky AS paths.
However, not all routes with suspected leaky AS-PATHs should be automatically dropped,
because some of them may not be unequivocal leaks.
A BGP monitoring service should provide alerts to all suspected leaks.
However, a router should automatically drop only unequivocal leaks.

https://bgpstream.com/event/258771 is listed as a leak.
The as-path is the red line.
The black line is a customer-provider relationship that exists between
37545 and 33765 as can be seen at https://asrank.caida.org/asns?asn=33765.
Therefore, 33756 is an authorized carrier of traffic for 37545 and
ought to not be called out as a leaker.
The route may not have taken the black line because of a temporary
failure of that link. An apparently leaky detour should be allowed
for a temporary failure as long as all the ASes involved are in the
provider cone of either the source AS or the destination AS.
All ASes in the provider cone of an AS are either directly or indirectly
authorized to carry traffic for that AS.

This is just one example. It's easy to find more.

ASPA should add a section that defines an unequivocal leak, such that
a BGP router can optionally drop only unequivocal leaks.

The definition of an unequivocal leak is based on the provider cone.
The provider cone of an AS consists of all the providers of that AS and
all the providers of those providers and so on.

All providers in the provider cone of either the originator or the receiver
of the route are permitted (contracted, actually) to carry the traffic for the
route between these two ASes. Therefore, the AS-path is only invalid if it
contains any ASes not within these provider cones. If valleys exist in the
AS-path, but these valleys are entirely within these provider cones, then
all the ASes in the AS-path are still permitted to carry the traffic and the
route should be declared valid.

Regards,
Jakob.

---

From: Jared Mauch <[email-masked]>
Date: Tue, 15 Dec 2020 02:55:15 -0500

I can tell you by looking that it’s a leak (or poison) due to the AS_PATH as 1299 <-> 3491 should be adjacent 

If you go to route-views, you can see plenty of routes with _1299_3491_

- Jared

> On Dec 15, 2020, at 1:45 AM, Jakob Heitz (jheitz) <jheitz=[email-masked]> wrote:
> 
> _______________________________________________
> Sidrops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sidrops

---

From: Ben Maddison <[email-masked]>
Date: Tue, 15 Dec 2020 14:50:03 +0200

Hi Jakob,

> ought to not be called out as a leaker.

No. This is about as clear cut and obvious a leak as you could hope to
find!

33765 is a customer of both 3491 and 37100. It is learning paths from
the former, and advertising it to the later. If ASPA won't drop that on
the floor then it's basically useless.

> authorized to carry traffic for that AS.
I think this is a misunderstanding of the nature of the "authorisation" that
an ASPA object describes.
The question of whether an AS is authorized to carry traffic to a
particular destination is orthogonal to the question of whether a route
is being leaked.
The question is: when ASX advertises path P to ASY, does ASX authorise
ASY to propagate it to its non-customer peers.
In the above example, I'd expect that the answer is no (although, one
would need to get their hands on an ASPA signed by 3491 to be sure).

> route should be declared valid.
That would exclude some of the most common and harmful species of leaks.
We most often see these when an AS provides transit, but filter towards
their peers on based on prefix-lists (no community, as-path filters,
etc).
When (due to an outage, badly conceived traffic engineering, etc) they
select a route as best which is originated by a customer but learned
from a peer, the route matches their egress filters, resulting in a
leak.
These cause congestion in unexpected places, send traffic through
middleboxes that aren't configured for it, and generally cause hard to
troubleshoot pain and suffering. Usually with collateral damage
included.

Mitigating this kind of thing is right at the heart of what makes ASPA a
good idea. Let's not gut it please.

Cheers,

Ben

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Tue, 15 Dec 2020 21:48:15 +0000

In oix-full-snapshot-2020-10-11-1800 from routeviews, I see the as-path:
(1299 40065 3491 138995).
From https://asrank.caida.org/asns?asn=138995,
I see that 40065 is a provider for 138995, so
(1299 40065 138995) is possible.
Indeed, there is another route in the same file with just this AS-path.
The fact that 40065 used 3491 to reach 138995 rather than
its direct link is nobody's business but its own.

Now, if the relationship between 40065 and 138995 did not exist, then
I agree, (1299 40065 3491 138995) would be an unequivocal leak.

We need to differentiate between such "traffic engineering" leaks
and unequivocal leaks, else ASPA will cause a lot of collateral damage.

We must start slowly. Drop only what we are sure is bad.
Then carefully make it stronger.

Regards,
Jakob.

-----Original Message-----
From: Sidrops <[email-masked]> On Behalf Of Jared Mauch
Sent: Monday, December 14, 2020 11:55 PM
To: Jakob Heitz (jheitz) <jheitz=[email-masked]>
Cc: [email-masked]
Subject: Re: [Sidrops] ASPA: Is this really a leak?

- Jared

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Tue, 15 Dec 2020 22:10:45 +0000

Ben,

Your scenario is possible.
However, it is only that: possible.
That makes it a suspected leak, not an unequivocal leak.
There are plenty of unequivocal leaks to stop.
Let's start with those and not bring down a lot
of unsubstantiated "suspected leaks".

Regards,
Jakob.

-----Original Message-----
From: Ben Maddison <benm=[email-masked]> 
Sent: Tuesday, December 15, 2020 4:50 AM
To: Jakob Heitz (jheitz) <[email-masked]>
Cc: [email-masked]
Subject: Re: [Sidrops] ASPA: Is this really a leak?

Hi Jakob,

Cheers,

Ben

---

From: Jay Borkenhagen <[email-masked]>
Date: Wed, 16 Dec 2020 11:40:26 -0500

> finds suspected leaky AS paths.

No, not really.

draft-ietf-sidrops-aspa-verification rejects routes whose AS_PATHs are
contra-indicated by the expressed wishes of the AS resource-holders,
as communicated by the set of validated ASPA records.

It's thus up to each party publishing ASPA records to ensure that all
necessary upstream and mutual transit relationships are explicitly
authorized.

						Jay B.

---

From: Ben Maddison <[email-masked]>
Date: Wed, 16 Dec 2020 20:41:12 +0200

Hi Jakob,

> That makes it a suspected leak, not an unequivocal leak.

Only because ASPA is not a thing yet. When it is, and we get to see an
ASPA issued by 3491, I would bet my bottom dollar 33765 won't be in it.

At that time, we can say unequivocally "that path is garbage", and drop
it on the floor.

> of unsubstantiated "suspected leaks".

I don't think you'll find many more obvious than the one you picked up.
But that isn't really the point.

Cheers,

Ben

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Wed, 16 Dec 2020 19:57:52 +0000

Jay,

I disagree that the algorithm in ASPA rejects routes whose AS_PATHs are
contra-indicated by the expressed wishes of the AS resource-holders,
as communicated by the set of validated ASPA records.

I posit that it rejects more than that.

Suppose AS1 has providers AS2 and AS20.
AS20 is also a provider for AS2.

What I am proposing is that AS2 should be allowed to divert traffic
that it received from the Internet through AS20 on its way to AS1.
Internet --> AS2 --> AS20 --> AS1.
The algorithm stated in ASPA prevents that.

Nobody is breaking any laws or contracts by doing that. Nobody is
seeing any traffic that they are not permitted to by doing that.
This kind of diversion happens frequently on the internet and
ASPA should not prevent it.

The big difference is that AS2 is a provider for AS1.
If it were not, then ASPA absolutely should reject the path.

Regards,
Jakob.

-----Original Message-----
From: Sidrops <[email-masked]> On Behalf Of Jay Borkenhagen
Sent: Wednesday, December 16, 2020 8:40 AM
To: Jakob Heitz (jheitz) <jheitz=[email-masked]>
Cc: [email-masked]
Subject: Re: [Sidrops] ASPA: Is this really a leak?

No, not really.

						Jay B.

---

From: Lukas Tribus <[email-masked]>
Date: Wed, 16 Dec 2020 22:11:08 +0100

Hello Jakob,

On Wed, 16 Dec 2020 at 20:57, Jakob Heitz (jheitz)
<jheitz=[email-masked]> wrote:
> Nobody is breaking any laws or contracts by doing that.

I disagree. AS2 is breaking the contract which is that it won't
announce my routes to non-customers peers. The fact that we are
talking about AS1 originated routes which is a customer of both AS20
and AS2 doesn't change that.

AS20 AS1 is an as-path that AS2 must not announce to non-customers,
unless specifically authorized by AS20. AS2 doesn't get to share AS20
routes, whether the origin-as AS1 is a common customer or not.

AS20 must decide whether this is acceptable or not. Not AS2 and not AS1.

> Nobody is seeing any traffic that they are not permitted to by doing that.

I disagree. If I'm AS20 and my customer AS2 shares prefixes with
non-customers without authorization, I will order AS2 (my customer) to
stop. I don't care if those routes are AS1 originated.

What if AS2 has insufficient capacity to deal with the traffic? AS20
has SLAs with AS1, but if AS2 suddenly thinks it has to be a transit
provider for AS20->AS1, then AS20 can no longer guarantee those SLA's,
because suddenly AS2 will pick up the traffic.

What if AS1 specifically disconnects AS2 to route around temporary
technical issues in the AS2 network? Now AS2 is still involved,
despite best efforts of both AS1 and AS20 to not traverse it.

This is an unauthorized announcement (unless AS20 says otherwise), and
a nightmare with real technical and contractual consequences. If my
customer AS2 insists on doing this, actively impeding my ability to
guarantee SLA's to AS1, then I will disconnect AS2. You can bet that I
will have the legal/contractual power to do so.

> ASPA should not prevent it.

It happens frequently by mistake because of the use of static prefixes
list instead of BGP communities. It also causes damage. If this basic
and common leak is not something that ASPA prevents, then I'm afraid a
lot of network operators won't even bother with ASPA.

> The big difference is that AS2 is a provider for AS1.

Your opinion is that this changes something. I'm saying it doesn't
change anything at all.

The example you provided causes a number of issues for all 3 parties
involved. Nobody wants this:

AS1 doesn't want this, because it cannot reroute around technical
problems in AS2 by disconnecting it. It is multihomed for a reason.
AS2 doesn't want this, because it would just give AS20 free transit
and overload it's own network with traffic that is not generating
revenue (but may happen erroneously due to static prefix list).
AS20 doesn't want this, because it will be unable to guarantee SLA's
to AS1 when an unauthorized transit provider is involved.

If this is something that both AS2 and AS20 agree on, then the proper
authorizations will be in place and all is good anyway.

- regards,
lukas

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 17 Dec 2020 07:49:53 +0000

Thanks Lukas. Good explanation.

So, if AS20 were to allow this, would it list AS2 as a provider in its ASPA record?

Regards,
Jakob.

-----Original Message-----
From: Lukas Tribus <[email-masked]> 
Sent: Wednesday, December 16, 2020 1:11 PM
To: Jakob Heitz (jheitz) <[email-masked]>
Cc: Jay Borkenhagen <[email-masked]>; [email-masked]
Subject: Re: [Sidrops] ASPA: Is this really a leak?

Hello Jakob,

- regards,
lukas

---

From: Ben Maddison <[email-masked]>
Date: Thu, 17 Dec 2020 10:33:23 +0200

> So, if AS20 were to allow this, would it list AS2 as a provider in its ASPA record?
Yup, spot on.
Or, of course, not bother creating an ASPA at all, in which
case *any* of its neighbors would be allowed to announce transit for it.

Cheers,

Ben

---

From: Lukas Tribus <[email-masked]>
Date: Thu, 17 Dec 2020 13:22:46 +0100

Hello Jakob,

> So, if AS20 were to allow this, would it list AS2 as a provider in its ASPA record?

Yes, I'm saying that AS20 has to authorize AS2 as a provider in the
ASPA record. This would also mean that AS2 would be authorized to be
transit for the entire AS20, not only AS20 -> AS1. This is a drawback
that AS20 has to consider here. You can't have your cake and eat it
too ... However I don't think this is a problem in practice, because
like I wrote in the other email, usually nobody wants this. And more
complex situations can always be authorized with multiple ASPA records
(just like mutual transit relations in section 7).

I would also suggest that ASPA should remain as simple as possible.
Exceptions to the basic rules as per the draft will make it more
difficult for network operators and implementators - in all steps
(implementing the code, training folks, actually deploying it in a
network, troubleshooting live issues). We want as much traction as
possible from both global and regional network operators. ROV and it's
implementations are complex and hard enough.

lukas