Subject: Re: [IPsec] multiple windows need multiple SPIs

From: William Allen Simpson <[email-masked]>
Date: Sat, 1 Aug 2020 07:54:34 -0400

On 7/31/20 4:37 AM, Michael Rossberg wrote:
> 
> Somehow associated SAs would perhaps allow us to derive/install a key locally on demand.

Correct.  In the original IPsec design, as we had specified that there could be
multiple SPIs per SA, the definition of SA was broader than later editors.

> However, due to the combinatorial explosion, these blocks of SAs may easily become pretty
> large, ie. with a reservation for multicast senders and QoS groups SPIs may be a little short.
> 

Wow, what architecture are you implementing?  After all, 2^32 SPIs by 2^32
packets per SPI are more than the estimated number of silicon atoms!

When I originally designed PIPE and reserved v6, that initial draft
eliminated the IP ToS field.  I've never seen the need, and believe that
history has validated my view.  Since then, "integrated services" and
"differentiated services" and other efforts have long been marketing points
to extract money from naive customers, but have had little practical effect.

As some folks know, once upon a time I founded one of the first ISPs in 1994.
In more than a decade, I'd never had a customer who needed QoS.

CoDel did far more for active queue management than QoS.  (I was involved in
the CeroWRT bufferbloat project, too.)

IMnsHO, those who want to shoot themselves in the foot with QoS deserve the
complexity and overhead.

Still, I've no idea how you'd run out of SPIs.

---

From: Michael Rossberg <[email-masked]>
Date: Mon, 3 Aug 2020 10:17:38 +0200

> Still, I've no idea how you'd run out of SPIs.

Unfortunately I develop systems for a customer who uses DS for some (maybe non-technical)
reason.

The issue I am struggling with: If there are multiple “reasons” to allocate blocks of SAs we’ll
have a lot of unused SPI space. Coming back to my example data center scenario: Customer
with 10,000 VMs doing multicast + QoS + having multiple cores (say AMD with 64).
We would require 14 bits to encode all possible senders, 6 bit to encode possible DSCP
values and 6 bit to encode the cores. That is 26 bits of our 32 bits in total. We only can build
up very few of these SAs (<32 if we want rekeying with make-before-break).

The key issue is that we had to preallocate all these SPIs even if few of them are ever used.

Maybe I am seeing this all a little pessimistic, but I would not want solutions that exclude each
other, i.e. multicast SAs that do not scale over CPUs, etc.

One way out is Tero’s suggestion to add the sender IP address to the SA lookup. I have thought
about this option over the last days, but I am coming more and more to the conclusion that
this is left out of the lookup for a couple of reasons: Most prominently there maybe issues with
IPv6 renumbering and multiple/non-unique sender IP addresses. Furthermore, the IP addresses
of the senders would required to be known in advance. All of this will cause a lot of control
plane overhead and possible race conditions in the interworking between control and data
plane.

---

From: William Allen Simpson <[email-masked]>
Date: Mon, 3 Aug 2020 19:34:07 -0400

> Unfortunately I develop systems for a customer who uses DS for some (maybe non-technical)

Helpful to not use abbreviations.  DS = storage Data Servers?  AWS Directory Service?
Microsoft Domain Services?

> with 10,000 VMs doing multicast + QoS + having multiple cores (say AMD with 64).

You know of a VM doing multiple bulk multicast tasks at the same time?

10,000 VMs sending multicast?  Or receiving?

As senders, 10,000 VMs would each have their own IP address; multicast senders don't
use the multicast address as Source.

Even in my wildest dreams, I'm finding it hard to envision 10,000 replicated data
storage VMs.  But in that case, they'd be receiving (object updates).

So no matter how many cores, there shouldn't be more than that many multicast SPIs.
(Or unicast SPIs for that matter.)

> One way out is Tero’s suggestion to add the sender IP address to the SA lookup.

Yep, several of us have mentioned this.  My understanding was to check the Sequence
Number for replay attacks, and for implicit IV generation.

> Most prominently there maybe issues with
> IPv6 renumbering and multiple/non-unique sender IP addresses. 

The stated reason for needing extra fields was to differentiate senders within the
same implicit IV space.  Renumbering won't hurt that.  Indeed, would help!

AFAIK, multicast sender non-unique Source IP addresses isn't allowed either.

Are you planning on 10,000 VMs somehow sharing the same Source IP address?

> Furthermore, the IP addresses
> of the senders would required to be known in advance. 

Why?  Again, it's just to keep the Sequence Numbers separate to avoid duplicate
implicit IVs under the same secret.  Define it algorithmically.  No state.

---

From: Yoav Nir <[email-masked]>
Date: Tue, 4 Aug 2020 07:16:28 +0300

> On 4 Aug 2020, at 2:34, William Allen Simpson <[email-masked]> wrote:
> 

Differentiated Service — one of iterations of introducing quality of service to IP packets.

---

From: "Valery Smyslov" <[email-masked]>
Date: Tue, 4 Aug 2020 09:00:20 +0300

Hi Michael,

> One way out is Tero’s suggestion to add the sender IP address to the SA lookup. 

If you properly implement ESP, than you must have already included
source IP for SA lookup. It was explicitly added to cover Source-specific Multicast use case.
And in case of SSM source addresses are not unique (however, all the drafts covering 
SSM are long expired, so I'm not sure it's widely or ever used).

>From RFC4303, Section 2.1:

   Each entry in the Security Association Database (SAD) [Ken-Arch] must
   indicate whether the SA lookup makes use of the destination, or
   destination and source, IP addresses, in addition to the SPI.  For
   multicast SAs, the protocol field is not employed for SA lookups.
   For each inbound, IPsec-protected packet, an implementation must
   conduct its search of the SAD such that it finds the entry that
   matches the "longest" SA identifier.  In this context, if two or more
   SAD entries match based on the SPI value, then the entry that also
   matches based on destination, or destination and source, address
   comparison (as indicated in the SAD entry) is the "longest" match.
   This implies a logical ordering of the SAD search as follows:

         1. Search the SAD for a match on {SPI, destination address,
            source address}.  If an SAD entry matches, then process the
            inbound ESP packet with that matching SAD entry.  Otherwise,
            proceed to step 2.

         2. Search the SAD for a match on {SPI, destination address}.
            If the SAD entry matches, then process the inbound ESP
            packet with that matching SAD entry.  Otherwise, proceed to
            step 3.

         3. Search the SAD for a match on only {SPI} if the receiver has
            chosen to maintain a single SPI space for AH and ESP, or on
            {SPI, protocol} otherwise.  If an SAD entry matches, then
            process the inbound ESP packet with that matching SAD entry.
            Otherwise, discard the packet and log an auditable event.

Regards,
Valery.

> I have thought