Subject: Re: I-D ACTION:draft-nordmark-multi6dt-shim-00.txt

From: Brian E Carpenter <[email-masked]>
Date: Thu, 28 Oct 2004 14:38:04 +0200

> http://www.ietf.org/internet-drafts/draft-nordmark-multi6dt-shim-00.txt

[Co-chair hat off]

I personally like this approach (I was always a fan of NOID).

Substantive remarks:
--------------------

 > 5.  Placement of the multi6 shim
...
>    ...Layering the
>    fragmentation header above the multi6 shim makes reassembly robust in
>    the case that there is broken multi-path routing which results in
>    using different paths, hence potentially different source locators,
>    for different fragments.  Thus, effectively the multi6 shim layer is
>    placed between the IP endpoint sublayer, which handles fragmentation,
>    reassembly, and IPsec, and the IP routing sublayer, which on a host
>    selects which default router to use etc.

This raises an interesting point about the flow label. According to
RFC 3697, a flow is identified by the 3tuple
    {Flow Label, Source Address, Destination Address}
and flow labels are only required to be unique when considered as
part of this 3tuple. In other words it is quite legal (even
if silly) for a host to use Flow Label = 1 for simultaneous
flows to Address A and to Address B. Now, what should we say
in the context of the multi6dt shim? Is a flow identified
by the complex tuple
    {Flow Label, {Source Locators}, {Dest Locators}}
or by
    {Flow Label, Source ULID, Dest ULID}
or do we simply consider it as a set of flows that all
happen to have the same Flow Label but various different
locator pairs?

I think it depends where you sit. Above the shim, a flow
has to be identified using its ULID pair. Below the shim,
but in the hosts, we have the necessary state to use the
complex tuple. In routers along the way, we can only
identify flows in the traditional way.

This does need to be discussed in the draft, even before
we get to section 9.2.1.

 > 8.1.  Initial Context Establishment
...
>         At this point in time it is possible for the hosts to change to
>         a different locator in the set.  But until they have exhanged
>         the locator sets, and probably until they rehome the context to
>         use different locators, they continue sending and receiving IPv6
>         packets as before.

I think you need to make it clear whether the two hosts have to
*agree* to change locators, or whether it is a single-ended decision.

 > 8.2.  Locator Change
...
>    Given that each host knows the locator set for its peer, the host can
>    just switch to using a different locator pair. 

Don't the exit router selection and ingress filtering issues need to be
handled here?

> 8.3.  Concurrent Context Establishment
> 
>    Should both A and B attempt to contact each other at about the same
>    time using the same ULIDs for each other, the context establishment
>    should create a single host-pair context.
> 
>    However, if different ULIDs are used this would result in two
>    completely independent contexts between the two hosts following the
>    basic content establishment above.

Is there really no race condition where this would fail?

> 9.2.1.  Flow-label
> 
> 
>    A possible approach is to carry the context tag in the Flow Label
>    field of the IPv6 header.  This means that when a multi6 context is
>    established, a Flow Label value is associated with this context.
>    When a packet is received, the Flow Label value is used as a key to
>    determine the context to be used for the demultiplexing operation,

That isn't enough. As noted above, a flow label is only unique within
a {Flow Label, Source Address, Destination Address} 3tuple. And in
multi6, that surely has to be extended to any 3tuple in the complex tuple
{Flow Label, {Source Locators}, {Dest Locators}}. A slightly tricky
per-packet lookup, that won't be good for performance. But there is
good news just below...

>    determine the context to be used for the demultiplexing operation,
>    hence determining the identifiers that have to be presented to the
>    upper layers.  Because this approach overloads the Flow Label field,
>    it is necessary to have an additional information to determine
>    whether the Flow Label field is actually being used as a context tag
>    or not.  In other words, additional information is needed to identify
>    multi6 packets from regular IPv6 packets.  This is because, the same
>    Flow Label value that is being used as context tag in multi6 enabled
>    communication can be used for other purposes by a non-multi6 enabled
>    host,

Good news. This is wrong (and I didn't realise it when analyzing NOID).

Flow Labels are not unique on their own and cannot be used for
anything on their own. You must always lookup a 3tuple.But if the
received  {Flow Label, Source Locator, Dest Locator} is in the set
{Flow Label, {Source Locators}, {Dest Locators}} corresponding to
a particular {Flow Label, Source ULID, Dest ULID} 3tuple, the shim
*knows* that it is a multi6 packet. (The only extra rule at the source
to make this true is: never use the same flow label for a multi6 flow
and a non-multi6 flow to the same destination.)

(One catch: the source must not send shimmed packets until it has
positive acknowledgement that the destination has built the multi6
state, but I think that catch will apply to all solutions.)

Now, I see that you list exactly this below as "Another approach."
But I think it is in fact the *only* approach that is fully
consistent with flow label semantics, since the flow label is *only*
unique per source/destination pair.

Editorial:
----------

> 4.  Locators as Upper-layer Identifiers
...
>    This implies that the ULID section is performed as today's default

s/section/selection/

> 8.4.  Handling Initial Locator Failures
> 
> 
>    Should not all locators be working when the communication is
>    initiated some extra complexity arises, because the ULP has already
>    been told which ULIDs to use.  If the locators that where selected to
>    be ULIDs are not working and the multi6 shim does not know of
>    alternate locators, it has to choice than to have the application try
>    a different ULID.

s/has to choice/has no other choice/

     Brian

---

From: Erik Nordmark <[email-masked]>
Date: Fri, 29 Oct 2004 14:13:17 -0700

Brian,

Thanks for the comments.

> This does need to be discussed in the draft, even before

In fact it is almost orthogonal to 9.2.1.
Even if the flow label is not used for demultiplexing, the draft should 
say how the shim handles the flow label.
I think the simplest approach is that the flow label is carried through 
as is. This means that the {Flow Label, Source ULID, Dest ULID} will 
appear on the wire as the set of {flow label, source locator, 
destination locator) for the different locator pairs.

This does have implications for protocols which do explicit signaling to 
create flow state; such protocols would somehow need to be multi6 aware 
so that they can perform the signaling for all the locator pairs that 
are used on the wire.

> *agree* to change locators, or whether it is a single-ended decision.

I don't know if the approach mandates this, or if it is my particular 
take on a solution, but in any case ...
They would need to agree that they have the capability to change them, 
but once that is done, each end can decide which locators (src and dst) 
it uses in the packets it sends.

> Don't the exit router selection and ingress filtering issues need to be

Yes, we need to state that assumption up front.

> Is there really no race condition where this would fail?

I believe that a signaling protocol which sets up and manages the multi6 
state can be designed so that
  - for the same ULID pair there is only a single host-pair context
  - for different ULID pairs between the same hosts one can create 
different host-pair contexts
The NOID draft is an existence proof of at least the first one, but I 
haven't verified the second one.

Note that in the second case it might be desirable to try to either 
merge those contexts together, or at least share reachability 
information between them.

> per-packet lookup, that won't be good for performance. But there is

You're right in the general case. However, one could further constrain 
flow label allocation for the hosts that implement multi6 so that for 
multi6 packets the receiver wouldn't have to compare the locators but 
only use the flow label. Note that due to deferred multi6 capability 
discovery this would have to apply to all flow label assignments on a 
host which implements multi6. And it being feasible doesn't mean it is a 
good idea :-)

> *knows* that it is a multi6 packet.

If the {flow label, src locator, dst locator} is used to identify the 
state then you are right. But that prevents a possible optimization with 
changing locator sets (think mobility) by requiring that multi6 
signaling be complete to tell the peer of the new locator before that 
locator can be used as the source.

> (The only extra rule at the source

I think things are harder than they look at first sight.

With deferred state creation, all flows are potentially multi6 flows, 
and furthermore the locator set for the peer isn't know until the 
deferred signaling exchange. This means the flow label needs to be 
unique across all communication at the sender I think.

Take host A communicating with foo.example and bar.example.
foo.example has ULID1 and ULID2 in the DNS. A picks ULID1 for the 
communication.
bar.example has ULID3 and ULID4 in the DNS. A picks ULID3 for this.
A starts communicating with both of them using some flow labels.
Later A decided independently that it is worth-while to setup multi6 
state for ULID1 and ULID3. When doing so it discovers that it is in fact 
the same host, with 4 locators (ULID1 through 4).
If it had allocated the same flow label for some flow to ULID1 and some 
flow to ULID3, it is stuck; it must then constrain to use a subset of 
the locators for each ULID otherwise the receiver would but the wrong 
ULID in the packet.

So in this approach the only sane thing is for the sender to never reuse 
a flow label (where "never" is while there is existing communication, or 
there might be some multi6 state at a peer using that flow label), 
independently of the ULIDs being used.

FWIW allocating a context tag, which goes in its own extension header,
when the multi6 state is created, and only using that extension header 
when the receiver needs to rewrite the address fields (i.e., after a 
rehoming event) is a lot simpler. Costs 8 bytes per packet but only 
after there has been a failure resulting in rehoming to different locators.

    Erik

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 30 Oct 2004 13:36:28 +0200

Erik,

Just responding on the flow label points:

Erik Nordmark wrote:
....
> In fact it is almost orthogonal to 9.2.1.

Indeed. That's what I meant to say.

> destination locator) for the different locator pairs.

Yes, I see no alternative rule that can be implemented efficiently.

> so that they can perform the signaling for all the locator pairs that 

Yes. And I hear that NSIS has realised already that they interact
with multihoming - in fact one of the reasons we moved the session
back to Monday was to avoid a clash with NSIS.

...
> only use the flow label. 

No, that doesn't work, because two quite independent source hosts might
choose the same flow label. You have no choice but to look at the source
locator as well as the label.

> Note that due to deferred multi6 capability 

Well, I think you could reasonably require that a sender never uses the same
flow label twice at the same time (that is more constraining than the global
rule, but probably not a problem in a 20 bit space). But you don't need to.
You simply apply the normal rule - make it unique per source and dest ULID
pair. That should make it unique across the {{source locator},{dest locator}}
set anyway, and thats's all you need.

> locator can be used as the source.

But you MUST know the source locator in order to discriminate flow labels,
as mentioned above.

> I think things are harder than they look at first sight.

Always :-) But the point here is that you have to deal with independent
sources that happen to choose the same flow label.

> unique across all communication at the sender I think.

Yes, but you have to know who the sender is...
> independently of the ULIDs being used.

Agreed. The problem is if host B does the same thing and picks the same
flow label values by chance. That only works if foo and bar look at
the source locators too.

> after there has been a failure resulting in rehoming to different locators.

I agree. There is just the minor MTU annoyance.

    Brian

---

From: marcelo bagnulo braun <[email-masked]>
Date: Sat, 30 Oct 2004 20:55:37 +0200

> state then you are right. But that prevents a possible optimization 
> with changing locator sets (think mobility) by requiring that multi6 

well, but in this case, there is also some security information nneded 
to authenticate the new locator, so the receiver will have additional 
information for the demux of this packet i guess.

regards, marcelo
------------------------------------------
Please note that my former email address
[email-masked] is no longer in use
Please send mail to:
marcelo at it dot uc3m dot es
------------------------------------------