Subject: Re: [IPsec] Additional charter items 1/4: Responder MOBIKE

From: Tero Kivinen <[email-masked]>
Date: Wed, 7 Mar 2018 23:20:23 +0200

Valery Smyslov writes:
> > There is no timers in the RFC specified for any of these operations,
> > all of them are implementation details. This is something that will
> > NOT affect interoperability, but will affect how well your
> > implementation works. If this is important matter for your
> > implementation you should research and study the problem and tune the
> > parameters suitable for your normal use case.
> 
> We are talking about different things. You are trying to convince 
> me that the cluster functionality _can_ be achieved with the current
> MOBIKE. I don't disagree, but I'm pointing out, that in this case
> it is relied on completely _optional_ features specified in RFC
> and there is no indication whether and how peer supports
> these features. These features don't affect MOBIKE interoperability, 
> but they do affect whether unmodified MOBIKE can be used
> for cluster scenario.

You are using lots of optional features too, things like NAT traversal
and Configuration payloads. Implementations implement features they
think are useful, and if you think features that allow MOBIKE movement
is useful then you will implement them. 

> Again, I'm not insisting that my proposal is the only and the best
> solution. Probably we can use the approach you suggested,
> but in this case some extension to the MOBIKE should be added
> that will make these features non optional and will add 
> a negotiation (or announcement) mechanism, so that 
> implementations can rely on peer's behavior.

Actually your cluster will see keepalive packets for both
IP-addresses, and can use this to indicate whether the client keeps
both addresses active through NAT. Also your cluster can actually test
it by sending empty informational exchange to the 2nd address first
few times, and then replacing it with 1st address if no reply (which
would indicate that 2nd ip address does not work).

So those features can detected if needed to.

> > It should detect it in few tens of seconds, and probes should find the
> > working path in tens of seconds more, or so. I.e., this should happen
> > in well under a minute.
> 
> That was my conclusion too. And I don't think that about a minute is
> a "short delay".

I have much longer delays in my network connections several times a
day. Especially when using mobile networks, or roaming between mobile
networks and wifi.

Yes, if you have streaming video or audio ongoing that will be
annoing, but even those usually buffer data for minute or so.

For conference calls or similar, minute is really noticeable break.

> > If you care about real-time traffic, then you should keep your NAT
> > mappings up for all peers, so you will get the address update message,
> > and can move to new address immediately when you receive it.
> 
> It won't help much. The MOBIKE client doesn't know that it must 
> switch to a just received additional address once he receives it. The 
> event that usually triggers this movement is an availability of current
> path. So you still will have a the delay while the client detects that
> the current path is not working and the new path works.
> You'll still have about 10-20 seconds delay at best.

No. The switch will be triggered immediately when the cluster / server
sends MOBIKE update using the 2nd ip address and does NOT include the
currently used IP address in the address list. If that message goes
through the client will start switch immediately, and probing the that
it works is just one round trip, so the delay should be less than
second.

I.e.

    Initiator                              Responder
    ---------                              ---------
                 <-- HDR(IPr2, IPi), SK { N(NO_ADDITIONAL_ADDRESSES) }
    HDR(IPi, IPr2), SK { } -->

This will immediately trigger the initiator to switch to IPr2, as IPr1
is no longer available. Note, that IPsec traffic can be switched to
new address at this point already.

    HDR(IPi, IPr2), SK { N(UPDATE_SA_ADDRESSES),
                      [N(NAT_DETECTION_SOURCE_IP),
		       N(NAT_DETECTION_DESTINATION_IP)],
		      [N(COOOKIE2)] } -->

              <--  HDR(IPr2, IPi), SK { [N(NAT_DETECTION_SOURCE_IP),
	      	   	     	         N(NAT_DETECTION_DESTINATION_IP)],
					[N(COOKIE2)] }

and after that move is finished. So if client is keeping NAT mappings
alive address change can be done without any lost packets. And the
fact that it keeps mappings alive can either be tested, or you can use
the fact that you get keepalive packets to the 2nd address as
indication of that.

> > > Then the IKE SA state must be transferred to a new node and the
> > > current node must stop responding to this client.
> > 
> > You need to move the IKE SA anyways, and you need to move it before
> > you send any update which says there is another IP address to be used.
> > And both ends MUST be able to process IKE and IPsec SA packets at the
> > same time if we want to make sure no packets are lost during the
> > transition (if you really care about real-time traffic).
> 
> No, in your case the first node must stop responding to the client,
> so that client understands that it should switch to the other address.
> With my proposal the client is explicitly asked to do that,
> so the whole procedure should take less time and be more reliable.
> so that it is easier to make the event atomic from cluster point of view.

Not true.

   Changing addresses can also be triggered by events within IKEv2. At
   least the following events can cause the initiator to re-evaluate
   its local address selection policy, possibly leading to changing
   the addresses.
...
   o  An INFORMATIONAL request containing an ADDITIONAL_IP4_ADDRESS,
      ADDITIONAL_IP6_ADDRESS, or NO_ADDITIONAL_ADDRESSES notification
      is received. This means the peer's addresses may have changed.
      This is particularly important if the announced set of addresses
      no longer contains the currently used address.

> > > Then the cluster would wait until the client detects the failure and
> > > switches itself to a new node. And there is also a chance that there
> > > are some IKE exchanges in progress, so if the node stops responding
> > > the exchanges could time out the and the IKE SA would be deleted
> > > before the movement takes place (in my proposal the MOBIKE is
> > > combined with RFC6311 exchange to make this working)...
> > 
> > If you are using MOBIKE, then the IKE exchange should not time out
> > before it has tried all possible addresses, thus there is no issue in
> > there.
> 
> Can you please point me where RFC4555 requires that 
> _any_ exchange must try all possible addresses before 
> timing out? Path testing is described in Sections 3.10 and 3.12
> and these sections only tell about using INFORMATIONAL 
> DPD exchanges for this purpose.

I do not think it says that directly, but that is only way to get
MOBIKE working. It does that explictly for the initial IKE exchange:

3.1. Initial IKE Exchange
...
   If either or both of the peers have multiple addresses, some
   combinations may not work. Thus, the initiator SHOULD try various
   source and destination address combinations when retransmitting the
   IKE_SA_INIT request.

For the other exchanges this can be seen in the section 3.5.

I.e. one of the triggers for the address change is:

   o  An IKEv2 request has been re-transmitted several times, but no
      valid reply has been received. This suggests the current path is
      no longer working.

Note, that this is done BEFORE the exchange times out.

So in the next step you pick address you want to try next, and go
forward, and update IKE SA with new addresses, and then:

   o  If there are outstanding IKEv2 requests (requests for which the
      initiator has not yet received a reply), continues
      retransmitting them using the addresses in the IKE_SA (the new
      addresses).

I.e., you start sending them out with new IP address. If you have
space in your window you might also send address update, but quite
often implementations only support window size of 1, so you send your
original packet for 2nd IP address pair for some time, and if it still
does not work, you go back to beginning, and notice this address pair
does not work, lets pick next one. And you continue doing that until
you time out the whole exchange after several minutes. Note, that you
might run out of ip-address pairs during the process so you might end
up going back to beginning again.

After you do get reply to your IKEv2 request, then you now do have
space in your window and you do send UPDATE_SA_ADDRESSES packet to the
other end... 

> Broken implementations is not an issue here (although it is always big issue :-().
> The issue is that RFC4555 in its current form is too vague to be used
> for cluster use case. This use case requires some additions to RFC4555
> or some clarifications in any case (if the cluster use case is solved
> using MOBIKE, that is not the only possible way).

I am not sure about that. Note, that we did work quite long to get the
rules in section 3.5 correct, and there are things there that will
make things work correctly if you follow the rules, even if not
everything is explained there (i.e., it does not explain why you need
to do things exactly like it says, it just assumes you do).

RFC4621 explains the design rational behind the MOBIKE, and it
explains why we did some things in RFC4555. For example the section
6.2 of the RFC4621 explains that we need to use any existing IKE
exchange as path testing message and explains why we did it. 

> OK. How about the following?
> 
> MOBIKE protocol [RFC4555] is used to move existing IKE/IPsec SA from
> one IP address to another. However, in MOBIKE it is the initiator of
> the IKE SA (i.e. remote access client) that controls this process. 
> While the responder can try to instruct the initiator to switch to a different
> IP address, the whole process is not reliable enough, especially 
> in presence of NAT or firewalls. If there are several responders each having 
> own IP address and acting together as a load sharing cluster, then it is desirable 
> for them to have ability to request initiator to switch to a particular member.
> The working group will analyze the possibility to extend MOBIKE
> protocol or to develop new IKE extension that will allow to build load
> sharing clusters in an interoperable way.
> 
> Is it better?

Much better, but I still think that this can be done without
modification to the MOBIKE itself, and you already implement it in the
cluster end without any need to modify client end. It will work better
if the client will send keepalives for all IP-addresses instead of
just using the one, or if the client probes other paths than what is
used every now and then. 
-- 
[email-masked]

---

From: "Valery Smyslov" <[email-masked]>
Date: Mon, 12 Mar 2018 11:23:04 +0300

Hi,

[I trimmed the message]

> it works is just one round trip, so the delay should be less than

I believe this interpretation is wrong. The IP address from the IP
is always implicitly included into the list of host's IP addresses (Section 3.6):

   If the exchange initiator has only a single IP address, it is placed
   in the IP header, and the message contains the
   NO_ADDITIONAL_ADDRESSES notification.  If the exchange initiator has
   several addresses, one of them is placed in the IP header, and the
   rest in ADDITIONAL_IP4_ADDRESS/ADDITIONAL_IP6_ADDRESS notifications.

So, according to the RFC4555 the currently used IP address must not be included
into the ADDITIONAL_IP*_ADDRESS notification. So, the client won't switch
to a new address immediately, it's first test an old path and if it works
it'll most probably do nothing.

>     Initiator                              Responder

With NO_ADDITIONAL_ADDRESSES it will work, but only in case the mapping 
for IPr2 exists. That's the issue.

> the fact that you get keepalive packets to the 2nd address as

You are oversimplifying the problem. To make periodic tests by responder you 
must have a copy of IKE SA on all the cluster nodes and continuously sync them.
And the client won't send any NAT keepalives until it gets reply from the cluster, 
so the IKE SA again needs to be present (and synced!) on all cluster nodes all the time.
While this is possible, it is a headache and to some extent it makes the 
whole idea of the load-sharing cluster meaningless.

Another problem is that with MOBIKE the initiator is free to switch to any path
it wants in any moment. So, to encourage the client to send NAT KA to all cluster's
addresses you must include all of them into ADDITIONAL_IP*_ADDRESS
notification, so that the client knows all of them.  You must also reply
to requests sent to any of these addresses, otherwise the client won't
start sending NAT KA. But it means that now the client can switch 
to any of these addresses on its own discretion - that's completely 
kill the idea of load sharing cluster: it is the cluster that must control 
when and where to move client.

>    source and destination address combinations when retransmitting the

These must be different requests (with different SPIs), at least if the initiator 
changes a destination IP, since the NAT_DETECTION_DESTINATION_IP would
be different. So it is not a retransmission, it's a new request.
Section 2.1 of RFC7296:

   A retransmission from the initiator MUST be
   bitwise identical to the original request.  That is, everything
   starting from the IKE header (the IKE SA initiator's SPI onwards)
   must be bitwise identical; items before it (such as the IP and UDP
   headers) do not have to be identical.

> up going back to beginning again.

There are additional complications not mentioned in the RFC.
If the exchange is a MOBIKE probe and NAT is supported, then 
it will include NAT_DETECTION_DESTINATION_IP. When the exchange
initiator tries  different destination IP addresses it must re-calculate
this notification, so that NAT is detected properly. This leads to a direct 
violation of Section 2.1 of RFC7296 (see above).

> exchange as path testing message and explains why we did it.

Sorry, but there are still some unclear places.
And I don't think in its current form it suites well for building
load-sharing cluster. You are trying to convince me otherwise, 
and I agree that it would probably _somehow_ work in _some_ 
situations, but the quality of such a solution leaves much to be desired, IMHO.

> just using the one, or if the client probes other paths than what is

Too many "if" for which the cluster end cannot be sure - 
client does it or not on its own discretion. That's what I'm complaining about.

Regards,
Valery.

---

From: Tero Kivinen <[email-masked]>
Date: Mon, 12 Mar 2018 17:10:45 +0200

>    rest in ADDITIONAL_IP4_ADDRESS/ADDITIONAL_IP6_ADDRESS notifications.

Yes, thats why the notify needs to be sent from address pair NOT in
use. In my example the header said IPr2, not IPr1. The IPr1 is the
currently used address, IPr2 was the additional address before.
Sending this notify will indicate that IPr1 is no longer in use and
because it is no longer in use it needs to be send from IPr2 as source
address.

> So, according to the RFC4555 the currently used IP address must not
> be included into the ADDITIONAL_IP*_ADDRESS notification. So, the
> client won't switch to a new address immediately, it's first test an
> old path and if it works it'll most probably do nothing.

Not true. 

> for IPr2 exists. That's the issue.

Yes the mapping for IPr2 needs to work. Note, that if the mapping for
IPr2 does not work, then this message will get thrown away by NAT, and
as there is no response to the notify the responder will at some point
assume that this path it switched to is broken and switch to use
another IP, and is does not have any other IP, it needs to trigger
some special code there to add IPr1 back to the list of allowed
addresses, and then it will retransmit this address again using (IPr1,
IPi) as addresses. This will then reach the initiator and for his
point of view this was just removing the IPr2 from the address list.

On the other hand after that the responder will know that this method
will not work as initiator does not keep mappings up, and can give up
with the load balancing for this client, and rather move the clients
which support faster redirect.

Or if it thinks load balancing is more important than the a minute
breakage, it can simply keep retransmitting it on the IPr2, and stop
responding to any packets to IPr1. After a while the initiator will
notice this and it will try to probe on IPr2 too, and that will create
the NAT mapping, and then responder can retransmit its notify to that
port, and it will reach the initiator. 

> whole idea of the load-sharing cluster meaningless.

Load balancing issues are really hard, and I assumed that those have
been taken care of by the cluster, as otherwise there is no point of
any of this. As client can at any point decide to use whatever address
the cluster gives it, the cluster MUST be able to process packets
arriving to any of its addreses always.

So load balancing issues inside the cluster is outside the scope of
this dicussion.

> when and where to move client.

Yes. If you want to have load balancing, you need to do it properly. 

>    headers) do not have to be identical.

Only if the destination address changes. Usually the client using
MOBIKE has only one destination address, and multiple source
addresses, so it will include all of its source addresses in the
NAT_DETECTION_SOURCE_IP notifies, and the one destination address for
the NAT_DETECTION_DESTINATION_IP. In that case there is no need to
change packet after it is sent, and it will be same request
transmitted over different source IP addresses.

If client do have multiple destination addresses, then it must assume
each of those is different, and create separate IKE_SA_INIT messages
for each of them.

> violation of Section 2.1 of RFC7296 (see above).

It will never re-calculate the packet. It will note this fact, and
mark this request as failed, thus it will run it through sending it to
the other end without modification, but after the process finishes, it
will throw away the result, and immediately start over with correct
destination address and correctly calculated
NAT_DETECTION_DESTINATION_IP address field.

I.e., following point in RFC4555:

   o  If a new address change occurs while waiting for the response,
      starts again from the first step (and ignores responses to this
      UPDATE_SA_ADDRESSES request).

will tell that if any change to addresses happened while it was
sending UPDATE_SA_ADDRESSES notify, then it will start over
immediately when reply is received. If while we were sending
UPDATE_SA_ADDRESSES we needed to switch to next destination address,
that means the "new address change" has occurred, thus we go back to
beginning.

> situations, but the quality of such a solution leaves much to be desired, IMHO.

Load balancing was explictly mentioned as out of scope for the MOBIKE,
we did say that the method we are using in MOBIKE should try to
support load balancing, but we never meant it to be generic solution
for load balancing. There are lots of additional issues in the load
balancing which needs to be solved than just this.

I.e. RFC4621 says:

   Note that MOBIKE does not aim to support load balancing between
   multiple IP addresses. That is, each peer uses only one of the
   available address pairs at a given point in time.
...
   Load balancing is currently outside the scope of MOBIKE; however,
   future work might include support for it. The selected format needs
   to be flexible enough to include additional information in future
   versions of the protocol (e.g., to enable load balancing). This may
   be realized with an reserved field, which can later be used to
   store additional information. As other information may arise that
   may have to be tied to an address in the future, a reserved field
   seems like a prudent design in any case.

and RFC4555 says:

   MOBIKE allows both parties to be multihomed; however, only one pair
   of addresses is used for an SA at a time. In particular, load
   balancing is beyond the scope of this specification.

For example MOBIKE always assumes that both peers are same entity,
i.e., even when there is multiple IP-addresses, they all reach the
same entity in the other end, thus either end can process any packet
sent to any of its IP-addresses. 
-- 
[email-masked]

---

From: "Valery Smyslov" <[email-masked]>
Date: Tue, 13 Mar 2018 10:42:34 +0300

> point of view this was just removing the IPr2 from the address list.

Yes, quite a lot of movements for a simple task.

> which support faster redirect.

Again, better to know this beforehand.

> port, and it will reach the initiator.

You repeat the scenario I described a couple of messages ago.

Let me cite myself once more - your explanations only support
my opinion, that it _is_ possible to use MOBIKE AS IS for load sharing, 
but it  comes out to be cumbersome and unreliable.

> arriving to any of its addreses always.

I don't think so. If we define an extension to MOBIKE for cluster, 
then the client will know that the selection of destination IP
is under cluster control and won't randomly switch from one IP to another.
Of course the selection of client's source IP is a client's prerogative.

> each of those is different, and create separate IKE_SA_INIT messages

That's what I said.

> NAT_DETECTION_DESTINATION_IP address field.

So, the first exchange is in fact timed out, but the client just ignores this
fact and instead of deleting IKE SA it starts a new exchange with a next MID, right?
Or do you mean that the new exchange will have the same MID?
RFC is silent about this detail.

Anyway, both cases lead to possible problems unless the responder
is treated these exchanges specially and ignores some RFC7296 rules.

For example, consider that the client first creates exchange with MID=5
and sends it to IP address IP1. Let's assume that the request reaches
the responder and responders sends back a reply, but the reply 
packet cannot get through due to routing asymmetry and the 
client never got it. From responder's point of view the exchange 
has completed successfully and it marks MID=5 as used.
If the client reuses this MID for the new exchange to IP2, then
the responder according to RFC7296 throws away new request
messages, as the MID=5 is already seen.

On the other hand, if the first request fails because the request
messages cannot reach the responder and the client uses next MID (6)
for its second request and this request reaches the responder, 
then the responder would never got the request with MID=5, so it 
will have a gat in its window and according to the RFC7296 
at some point it'll stop responding waiting for that MID.

Well, these issues aren't concerned with our topic of discussion,
they are just MOBIKE's complications not mentioned in the RFC.

>    seems like a prudent design in any case.

Excellent quote. That's exactly what I suggest - a "future work that includes
support for load balancing in MOBIKE". And I don't understand why 
you oppose it so strongly.

Regards,
Valery.