Subject: [core] Anycast and CoRE

From: Jim Schaad <[email-masked]>
Date: Sat, 8 Dec 2018 14:58:54 -0800

I have been looking at implementing Anycast along side of Multicast.  I am
trying to figure out if Anycast is supposed to have the response on the
Anycast address like Unicast does, or if it should be returned on a Unicast
address.  There is no clear guidance on this in RFC 7252.  Among the things
that RFC 7252 says is that coaps can be implemented using Anycast.  However,
given that the routing could be changed at any time, for example in the
middle of the DTLS handshake or processing blocks, it makes any sense to
return on the Anycast address.

Is there anybody who has looked at this or wants to provide guidance?

Jim

---

From: "Timothy L. Hirou" <[email-masked]>
Date: Sat, 8 Dec 2018 23:11:03 +0000

Timothy L. Hirou
Founder
Convergence Wireless, Inc.
23 Corporate Plaza Drive, Suite 100
Newport Beach, CA 92660
(800) 519-9820 Office
(949) 258-5592 Fax
[email-masked]
www.convergencewireless.com‎

Privileged And Confidential Communication This electronic transmission, and any documents attached hereto, (a) are protected by the Electronic Communications privacy Act (18 USC §§ 2510- 2521),(b) may contain confidential and or legally privileged information , and (c) are for the sole use of the intended recipient(s) named above.  If you have received this electronic message in error, please notify the sender and delete the electronic message and destroy any physical copies of the information that may have been generated. Any disclosure, copying, distribution, or use of the contents of the information received in error is strictly prohibited.
  Original Message
From: Jim Schaad
Sent: Saturday, December 8, 2018 2:57 PM
To: [email-masked]
Subject: [core] Anycast and CoRE

Jim

_______________________________________________
core mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/core

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 12 Dec 2018 21:58:51 +0100

On Sat, Dec 08, 2018 at 02:58:54PM -0800, Jim Schaad wrote:
> I am trying to figure out if Anycast is supposed to have the response
> on the Anycast address like Unicast does, or if it should be returned
> on a Unicast address.

I can't offer guidence, just my .02€:

The client may not know that it's talking to a multicast server.
Therefore, the client would not correlate any messages from a unicast
address.

As for the block-wise transport, the effects would not be too bad:
If it's a stateless transfer, all is fine. If the anycast servers have a
magic backplane that distributes incomplete transfer states, likewise.
Otherwise, the transmission needs to be restarted.

Can't say much about DTLS, but in OSCORE that'd be an ideal use case for
the appendix B2 protocol. When the route flips, the client will get a
4.01 once and continue exchanging data with the protocol's Req2 message.

Best regards
Christian

-- 
There's always a bigger fish.
  -- Qui-Gon Jinn

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 12 Dec 2018 22:13:48 +0100

Hi Jim,
hi Christian,

I think the assumption here was that the anycast address is usable as a unicast address.
So the server can reply from the same address that it got the request on (as it always should if it has multiple addresses, which is almost always true in IPv6).  The special exception for multicast is not invoked (and not needed).

Of course, there is an assumption here that the routing won’t change quickly enough that the reply is swallowed by a BCP38-like mechanism.  If that happens, I’d say this is like any other routing-instability induced packet loss, and for a piggybacked response the client will simply retransmit.

If the anycast goes through an ECMP-like mechanism, multiple requests in a block-wise transfer would still go to the same server.  If there is completely random distribution, well, you get what you sowed…
(That is the general load balancer problem.)

Grüße, Carsten

> On Dec 12, 2018, at 21:58, Christian Amsüss <[email-masked]> wrote:
> 
> Signed PGP part

---

From: Jim Schaad <[email-masked]>
Date: Wed, 12 Dec 2018 14:58:24 -0800

> -----Original Message-----
> From: Christian Amsüss <[email-masked]>
> Sent: Wednesday, December 12, 2018 12:59 PM
> To: Jim Schaad <[email-masked]>
> Cc: [email-masked]
> Subject: Re: [core] Anycast and CoRE
> 
> Therefore, the client would not correlate any messages from a unicast address.
> 
> If it's a stateless transfer, all is fine. If the anycast servers have a magic
> backplane that distributes incomplete transfer states, likewise.

Would this potentially have a problem if one is using ETag for correlation?  I would not assume that the ETag values would be the same on both servers.  I think you might be in a situation where you need to restart the block transfer from scratch.

> 
> Can't say much about DTLS, but in OSCORE that'd be an ideal use case for the
> appendix B2 protocol. When the route flips, the client will get a

I guess this is a case where one would expect that the same credentials would be on all of the anycast machines since otherwise one would not be able to do this roll over.  You might get a 4.01 and need to get a new token from the AS instead.

Jim

---

From: Jim Schaad <[email-masked]>
Date: Wed, 12 Dec 2018 15:01:31 -0800

> From: Carsten Bormann <[email-masked]>
> Sent: Wednesday, December 12, 2018 1:14 PM
> To: Christian Amsüss <[email-masked]>
> Cc: Jim Schaad <[email-masked]>; [email-masked]
> So the server can reply from the same address that it got the request on (as it
> always should if it has multiple addresses, which is almost always true in IPv6).
> The special exception for multicast is not invoked (and not needed).
> 
> Of course, there is an assumption here that the routing won’t change quickly
> enough that the reply is swallowed by a BCP38-like mechanism.  If that
> happens, I’d say this is like any other routing-instability induced packet loss, and
> for a piggybacked response the client will simply retransmit.

It was not clear to me that this was the case that routing does not change unless there is "a problem" or something because "radically different" in the response times.  One cannot really test this on IPv4 since it is not supported and I don't think I can do it in IPv6 because I don't have smart enough routers to be able to do this.

Jim

> 
> If the anycast goes through an ECMP-like mechanism, multiple requests in a
> block-wise transfer would still go to the same server.  If there is completely
> random distribution, well, you get what you sowed… (That is the general load
> balancer problem.)
> 
> Grüße, Carsten
> 
> 
> > If it's a stateless transfer, all is fine. If the anycast servers have
> > a magic backplane that distributes incomplete transfer states, likewise.
> > Can't say much about DTLS, but in OSCORE that'd be an ideal use case
> > for the appendix B2 protocol. When the route flips, the client will
> > get a

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 13 Dec 2018 01:30:42 +0100

> Would this potentially have a problem if one is using ETag for correlation?  I would not assume that the ETag values would be the same on both servers.  I think you might be in a situation where yo…

Multiple servers that share an anycast address need to behave like a single server, as far as that is practical.
(The client cannot help here as it does not even know the IP address is an anycast one.)
So, when using block-wise, they need to generate consistent ETags.

Grüße, Carsten