Subject: Re: [v6ops] So nobody cares about load balancers? [ draft-carpenter-v6ops-label-balance-01.txt]

From: Joel jaeggli <[email-masked]>
Date: Thu, 01 Mar 2012 00:36:33 -0800

On 2/29/12 19:07 , Brian E Carpenter wrote:

>>
>> => Yes but they don't use flow labels for that. They use a unique
>> application-specific identifier.
>> flow label != cookie. The cookie is not created by the client.
> 
> Correct, and it's a fairly horrible solution, isn't it? It forces
> you into DPI for every packet in order to recognise the session.
> That's *exactly* what needs to change.

This is a fundamental misrepresentation of how load balancers work...

If you're doing l3+l4 there's nothing to "DPI" because you have to
dispatch a new TCP connection request on the basis of a SYN only,
there's literally nothing to inspect. If you are doing L7 termination,
for  example with http(s), imap or even sip there's no packet inspection
involved in plucking out a cookie, or login credentials or the like.

---

From: Warren Kumari <[email-masked]>
Date: Thu, 1 Mar 2012 10:02:05 -0500

Apologies for the top post -- this response isn't specific to this mail (and is more of a meta-comment), so I am not responding inline...

My initial response to this draft was fairly negative -- I latched onto a shortcoming (mainly the fact that the flow label is client generated) and didn't really give the whole idea the thought that it deserved -- I apologize for this.

What occurs to me is that load-balancers support all sorts of algorithms and methods for mapping (session|flow|stream-o-bits) to a backend. If this proposal / draft flies and gets adopted by the vendors, I (and you) don't *have* to use it....
This will likely be implemented as yet another knob that can be enabled or disabled as inout to the decision process -- if operators want to turn the knob on, who am I to say no? 
We all have different requirements, views, applications and traffic distributions -- I don't see a downside to this idea, and it may have utility, so I officially withdraw my earlier grumpy face  :-)

W

On Feb 29, 2012, at 10:07 PM, Brian E Carpenter wrote:

> Hesham,
> 
> On 2012-03-01 13:53, Hesham Soliman wrote:
>>> 
>>>>> Basically a browser would pick a random 20-bit number when opening a
>>>>> page and would assign this 20-bit random number to all outgoing
>>>>> connections
>>>>> to the same site, whether they're HTTP or HTTPS btw.
>>>> 
>>>> => Is it safe enough to rely on the browser's 20 bit RNG and ignore IP
>>>> addresses? 
>>> I'd say it differently. It's safe to rely on anything but the IP address.
>> 
>> => No one suggested that you rely on the address alone.
>> 
>>> One user != one IP address. It's only an approximation which is valid
>>> enough where it's only used for optimization, but not where user session
>>> stickiness is required.
>> 
>> => And the flow label can ensure that stickiness by itself? That's a
>> contradiction of what you're trying to do.
>> If you want proper stickiness then you need unique application
>> identifiers. If you think the user's IP address is not stable enough and
>> you want stronger stickiness then I'd say the flow label is much worse.
>> More below
>> 
>>> For instance, hashing a source IP address is fine
>>> to load-balance an SSL farm, because most of the users will remain on the
>>> same SSL endpoint, and the small part of those who will constantly change
>>> will just experience a minor degradation which is not a show stopper. But
>>> it's totally different when you want to ensure that once you log into a
>>> server you need to remain on this one until you click the "logout" button,
>>> because this time you cannot accept to drop connections for even 1% of
>>> your
>>> visitors. That's why LBs mostly rely on cookies and have to ignore IP
>>> addresses for this task.
>> 
>> => Ok, but my point is that you can't compare flow labels and cookies
>> because you'll end up with either incorrect load distribution (because the
>> flow label doesn't tell you enough about the connection, or with users
>> being distributed onto different servers, depending on which side your
>> load distribution algorithm favours. What am I missing?
> 
> As the draft explains, you don't use the flow label to take the
> initial decision about which server gets which new session. That
> must be done by looking at much more of the first packet, exactly
> as today. You use the flow label to accelerate treatment of the
> subsequent packets. Please tell us where in the draft we need
> to make this more clear.
> 
>> If you LB flow label 27 on server 1 blindly 
> 
> You don't do that.
> 
>> then you can very well end up
>> with server 1 being busier than others. If you split flow label 27 between
>> servers 1 and 2 without any other considerations then you'll end up where
>> you don't want to be now.
> 
> Indeed, so you don't do that either.
> 
> You do accept the 1 in 2^20 chance that two unrelated sessions have the
> same flow label; in that case they will need to be redirected to the same
> server. In any case the LB is going to need a hash table to look up current
> flow labels, but it certainly contains a monstrous hash table today for
> other reasons. Server LBs are stateful beasts, unfortunately.
> 
> (In contrast to ECMP/LAG, which should be stateless.)
> 
>> Relying on applications to do the right thing (use a good RNG) for your
>> infrastructure to work seems like a proposition that could lead to
>> unreliable results.
> 
> It can only be better than today, where there is zero entropy
> in the flow label. The draft also discusses the case where the
> source doesn't set the label.
> 
> As for choice of RNG, yes, it needs to be reasonable. I have some
> results on this coming very soon.
> 
>> 
>>> With a flow label, we have a lower layer alternative to stickiness
>>> cookies.
>>> 
>>>> The discussion was related to the change in IP address and whether the
>>>> flow label helps in that case.
>>> Yes precisely it does help. If you make it possible for the client to set
>>> the flow label on purpose on outgoing connections, and you make the LBs
>>> only rely on the flow label, then you're not sensitive to IP address
>>> changes
>>> anymore. These ones happen at enterprise proxies and in the mobile world
>>> where it's very common to lose access after some idle time.
>> 
>>>> Aside from the fact that this case is rare and probably not worth the
>>>> optimisation, my contention is that the flow label doesn't help because
>>>> it's not unique.
>>> We don't need unicity (and we don't even want it). We need a large enough
>>> value so that not everyone goes to the same server.
>> 
>> => Large enough numbers of clients producing a uniform distribution of
>> flow label values. What happens when the clients don't behave that way?
> 
> The worst case is that the LB has to drop back to today's solutions.
> 
>> 
>>> Some L3 load balancers
>>> are still hashing on the lower 8 bits of IPv4 addresses right now. It's
>>> far from being unique and still it works for most purposes, provided that
>>> the number of servers is much less than 256. With 20 bits of randomness,
>>> you can have a fair distribution up to 1 million servers, or a bit less
>>> if you want to apply smooth weighting.
>>> 
>>>>> The LB would then
>>>>> either hash this flow label or load-balance + stick on it, at the
>>>> admin's
>>>>> option. The general idea is to insert a session identifier in the
>>>> packets
>>>>> so that all packets related to a same user session can be processed
>>>>> equally
>>>>> and directed to the same place, which is the hardest thing to do right
>>>> now
>>>>> in load-balancing.
>>>> => I don't know if this is a sound idea to be honest. I can understand
>>>> the
>>>> use of the flow label instead of port numbers when combined with IP
>>>> addresses for load balancing. I think that's a fine thing to do. I just
>>>> don't think using the flow label alone as a session id is such a good
>>>> idea
>>>> given the reliance on its uniqueness. It seems to optimise for a rare
>>>> case
>>>> and for a small benefit.
>>> It's not the rare case, it's the main case. Look at how load balancing is
>>> performed everywhere. IP address is never used when you need to guarantee
>>> user stickiness (eg: HTTP, RDP, ...).
>> 
>> => Changing IP address is a rare case, 
> 
> I'll let Willy comment on that, but he tells me that it's a very common
> case, viewed from a server site.
> 
> I wasn't saying that identifying
>> users is a rare case, lets not mix things. And yes of course user
>> identification happens but this is done with a *unique* identifier. You
>> keep using *user* stickiness as a reason while ignoring that to achieve
>> *user* stickiness you need a unique identifier. The combination of
>> unreliable number generators and load balancing based on those numbers
>> alone can end up with an undesirable effect. Do you disagree with this
>> statement? 
> 
> Again: that is not the model we describe in the draft.
> 
>> 
>> 
>>> That's even why LB vendors claim to
>>> support a wide variety of protocols. In fact what they have to do is to
>>> pick the relevant part in the protocol to find the server associated with
>>> the session. For instance, in HTTP you have to parse L7 to find cookies,
>>> in HTTPS you first have to decipher SSL then to extract HTTP cookies. In
>>> RDP you have to either find the user ID or the server's address which are
>>> hashed in RDP cookies, etc...
>> 
>> => Right, all about finding user-specific credentials that are unique on
>> the server side. You're replacing that with arbitrary numbers generated by
>> the client and claiming that it will have the same effect. In an ideal
>> world where clients behave, I think you have a good point, but if clients
>> don't behave, what will you do?
> 
> The worst case is that the LB has to drop back to today's solutions.
> Also, there's an incentive for client stacks to do the right thing:
> improved average application response time.
> 
> Regards
>   Brian
> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops
>

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 1 Mar 2012 16:35:48 +0100

Hi Warren,

On Thu, Mar 01, 2012 at 10:02:05AM -0500, Warren Kumari wrote:
> Apologies for the top post -- this response isn't specific to this mail (and
> is more of a meta-comment), so I am not responding inline...
> 
> My initial response to this draft was fairly negative -- I latched onto a
> shortcoming (mainly the fact that the flow label is client generated) and
> didn't really give the whole idea the thought that it deserved -- I apologize
> for this.
> 
> What occurs to me is that load-balancers support all sorts of algorithms and
> methods for mapping (session|flow|stream-o-bits) to a backend. If this
> proposal / draft flies and gets adopted by the vendors, I (and you) don't
> *have* to use it....
> This will likely be implemented as yet another knob that can be enabled or
> disabled as inout to the decision process -- if operators want to turn the
> knob on, who am I to say no? 
> We all have different requirements, views, applications and traffic
> distributions -- I don't see a downside to this idea, and it may have
> utility, so I officially withdraw my earlier grumpy face  :-)

You're welcome, having people like you who ask for clarifying what
we wrote is extremely important because it forces us to structure our
ideas a bit more. The simple fact that people ask questions means we
need to clarify our explanations :-)

Regards,
Willy

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 02 Mar 2012 09:01:38 +1300

> involved in plucking out a cookie, or login credentials or the like.

If you are mixing the two approaches, which I understand is common,
it isn't that simple. When you see a new L4 flow starting, you have to
check whether it is in fact part of an existing L7 flow but using
a different address/port.

I guess you are correct that once you have done this for the first full
L7 packet (not the SYN packet), you can revert to only looking at the
L3/L4 bits. But with IPv6 extension headers present, looking for L4
is extra work; the flow label is in a fixed place in the L3 header.

   Brian

---

From: Hesham Soliman <[email-masked]>
Date: Fri, 02 Mar 2012 11:26:56 +1100

Thanks for the clarification Brian. I feel more comfortable with those
answers below and I'm glad you're not relying on the flow label to make
the initial decision, which makes it better.

One other minor comment to make, I noticed, at least on email, references
to "browsers" setting the flow label. I'm going to be a bit picky and say
that we are really talking about (in the case of HTTP anyway) the HTTP lib
picking the flow label. Browsers are just one of the many apps that use
http nowadays and the less we rely on the app developers the more likely
we are to get something behaving uniformly.

Hesham

-----Original Message-----
From: Brian E Carpenter <[email-masked]>
Organization: University of Auckland
Date: Thu, 01 Mar 2012 16:07:57 +1300
To: Hesham Soliman <[email-masked]>
Cc: Willy Tarreau <[email-masked]>, IPv6 Operations <[email-masked]>
Subject: Re: [v6ops] So nobody cares about load balancers? [
draft-carpenter-v6ops-label-balance-01.txt]

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 2 Mar 2012 02:08:05 +0100

> we are to get something behaving uniformly.

You're perfectly right Hesham. I did personally use the term "browser" to
illustrate what I meant but the process should happen at a lower layer,
ideally in the lib handling the sockets. I've been thinking about using a
couple of getsockopt() + setsockopt() to retrieve and set the flow label
on a connection, so that's definitely where it ought to be.

Regards,
Willy

---

From: Hesham Soliman <[email-masked]>
Date: Fri, 02 Mar 2012 12:14:55 +1100

>on a connection, so that's definitely where it ought to be.

=> Cool, then we're on the same page.

thanks,
Hesham

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 02 Mar 2012 15:54:57 +1300

> on a connection, so that's definitely where it ought to be.

Yes. The advanced socket API (RFC 3542) left this as future work.
It does define it in the header structure

      uint32_t ip6_un1_flow; /* 4 bits version, 8 bits TC, 20 bits
                                flow-ID */

but doesn't use it. Maybe it's time to fix that.

   Brian

---

From: Washam Fan <[email-masked]>
Date: Fri, 2 Mar 2012 13:12:12 +0800

Hi Brian,

> is extra work; the flow label is in a fixed place in the L3 header.

What you said above is for the case where flow label is used to
identify the 'transport layer' session (5 tuples) the packet is
belonging to (although the load distribution algorithm is based on L7
identifer). I think everyone should OK with this.

But in the section 3, it mentions flow label is used to identify the
'application layer' session the packet is belonging to. there are also
two sub-case here:

one is, for all 'transport layer' sessions between a given src IP and
a given dest IP, flow label could be used to bind multiple such
'transport layer' sessions to a same 'application layer' session by
setting the same value for the flow label field. Persionally i think
it is OK as far as the application software is trusted to do this.

the other is, for all 'transport layer' sessions between a given host
with changing IPs and a given dest IP. flow label could be used to
bind multiple such 'transport layer' sessions to a same 'application
layer' session by setting the same value for the flow label field.
Persionally I think it is not OK. And some other people have raised
security concerns about this. in the flow label spec, it suggests flow
label should be used with other fields like srcIP and destIP to
identify a session, since the srcIP is changing, i don't think it
would be compliant with the flow label spec if we allow this.

Thanks,
washam

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 2 Mar 2012 08:05:51 +0100

Hi Washam,

> would be compliant with the flow label spec if we allow this.

The reasons for this have been explained in this thread. Basically, if
the srcIP is used at all to consider the flow label, then there is no
point using flow labels for this task since the main goal is to be able
to do what is currently done at layer 7 because srcIP is not a reliable
criterion for load balancing and stickiness. srcIP breaks stickiness by
changing for a small but real percentage of users. So in short, if srcIP
needs to be considered with flow label, then neither will be used (as it
is today) and L7 only will continue to be used (as it is today too).

The security concerns that were raised can be moderated in that they are
the ones that currently affect all load balanced infrastructures working
at layer 7 (eg: ~all).

Regards,
Willy

---

From: Washam Fan <[email-masked]>
Date: Fri, 2 Mar 2012 18:18:35 +0800

Hi Willy,

> the ones that currently affect all load balanced infrastructures working

I am not sure if i understand completely. My interpretation of the
draft is, flow label is not used for choosing a backend server. L7
identifier is still used to choosing a backend server, but the {srcIP,
flow label} can be used to identify all subsequent packets belonging
to the same L4 or L7 session.

If the srcIP changes, flow label alone can not be used to identify any
session. I suggest to remove the scenario in section 3 where flow
label is used across different srcIPs.

Thanks,
washam

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 2 Mar 2012 12:21:30 +0100

Hi Washam,

> to the same L4 or L7 session.

No it's not what is suggested, so we'll probably need to rework it.

The main algorithm is :

  - if flow label is not present, fall back to current methods (eg: L7 or srcIP)
  - if flow label is present and known, use it to select the proper server
  - if flow label is present and not known, apply the usual load balancing
    algorithm to select the server and remember the flow label <-> server
    mapping.

It is *likely* that some admins will also prefer to apply a simple hash on
flow labels the following way, though this is not what is suggested right now :

  - if flow label is not present, fall back to current methods (eg: L7 or srcIP)
  - if flow label is present, hash it and use the result as the server number.

I think we'll have to discuss both methods in the draft separately to wipe
any risk of misunderstanding.

> label is used across different srcIPs.

On the opposite, flow label here makes it possible to perform load balancing
across multiple srcIP which precisely is what saves the load balancer from
having to perform L7 DPI. Otherwise the srcIP alone would be enough.

Regards,
Willy

---

From: Washam Fan <[email-masked]>
Date: Fri, 2 Mar 2012 20:59:53 +0800

Hi Willy,

Thanks for your prompt response, please see inline.

>    algorithm to select the server and remember the flow label <-> server

Yes. this algorithm is how the draft suggests, IMHO.

>  - if flow label is present, hash it and use the result as the server number.

This algorithm is not what the draft is trying to express, but many
people on the mailing list think that way, IMHO.

> having to perform L7 DPI. Otherwise the srcIP alone would be enough.

flow label should always be used with other fields like srcIP, destIP
to identify a session, per RFC6437. {srcIP, flow label} saves the load
balancer from having to perform L7 DPI on subsequent packets belonging
to the same session while load balancer has to perform L7 DPI anyway
on the first packet on the session.

If flow label alone can identify a session without srcIP, let's say.
an attacker can easily launch the attach by insert same flow label
value into packets originated form different hosts which are
controlled by the attacker, just like some guys have pointed out.

THanks,
washam

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 2 Mar 2012 15:20:38 +0100

> on the first packet on the session.

As I explained, it does not save it from anything because if the LB sticks
to the flow+srcIP, once the IP changes, there will be nothing at L7 it
could use to match the session against the previous one. Either we can
use the flow label as a stable user-level *session* identifier, or it
cannot be used at all as it won't offer anything above srcIP alone.

> controlled by the attacker, just like some guys have pointed out.

Yes and this is already the case everywhere. Connect to a server,
get a stickiness cookie, share it with 1000 buddies and you'll get
the same result. Nothing new here, the whole web infrastructure lives
very well with this right now.

Regards,
Willy

---

From: Washam Fan <[email-masked]>
Date: Fri, 2 Mar 2012 22:57:55 +0800

Hi Willy,

> could use to match the session against the previous one.

I agree once the IP changes, LB would do L7 DPI on the first packet
using new srcIP. As far as the IP keeps as it is, LB would not do L7
DPI on packets belonging to the same session identified by flow+srcIP
except the first packet.

When IP doesn't change, it does save LB from L7 DPI on packets to
identify a session.

> Either we can

LB can benefit when srcIP doesn't change. Please see above.

> very well with this right now.

OK then. I'd like other people to chime in to share their opinions on this.

Thanks,
washam

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 2 Mar 2012 16:05:37 +0100

> DPI on packets belonging to the same session identified by flow+srcIP

I think we don't understand each other well, I'm sorry. What I mean is
that if the load balancer has to rely on L7 in the middle of a session,
it will not know what to look for there. From the beginning of the
session, it only relied on the flow label. Suddenly the srcIP changes
and the label is unknown. Let's consider it has to L7 DPI. What will
it look for there ? Until then it has not had to open the L7 stream so
it doesn't know what it contains nor what to look for there. And this
is even worse when one wants to match HTTPS against HTTP and vice-versa.

Regards,
Willy

---

From: Joel jaeggli <[email-masked]>
Date: Fri, 02 Mar 2012 08:33:40 -0800

On 3/1/12 12:01 , Brian E Carpenter wrote:
> If you are mixing the two approaches,

I'm not... I'm distinguishing between two approaches neither of which
involve deep packet inspection.

> which I understand is common,

Or, I don't. Common web application tiers based on rest api are
stateless beyond a single session. so an application implemented in that
fashion has no l7 sticky implications beyond the open tcp connection.

For applications that do require adhesion today, for example realtime
game servers, if you want to load balance them, how do you do that)? you:

use an upper-layer identifier of some variety (Which involves becoming
an ALG)

use ip persistence and recognize that devices that change IP  will be
disrupted in some fashion.

does a unique flow label per host enhance the ability to do that? maybe.
if you can get clients to actually behave in the fashion you describe,
if you're marking the flow label yourself you don't have a prayer of
having it be useful for this.

> L3/L4 bits.

An application layer load balancer is terminating the incoming
connection, it's not doing l3/l4 forwarding. If you're snarfing cookies,
doing http-proxy or https/tls termination that's what you're doing. No
DPI is involved.

> But with IPv6 extension headers present, looking for L4

If you're terminating the connection on the load balancer as for example
ha-proxy does when operating as an http-proxy you've got to find the l4
header anyway.

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 2 Mar 2012 17:47:43 +0100

Hi Joel,

> doing http-proxy or https/tls termination that's what you're doing. No

DPI was used in this thread to distinguish the need to analyze up to
layer 7 vs the need to analyze up to L3. I totally agree with you that
L7 load balancers are proxy nowadays (I'm not counting the few old
Alteon and KTCPVS which used to process L7 on a per-packet basis).

> ha-proxy does when operating as an http-proxy you've got to find the l4

The difference is that you can have your L4 LB totally bypass the L7 LB
when the flow label is correctly set. This is a huge gain, especially
when it comes to https where the LB does not have to decipher anymore.

Regards,
Willy

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 03 Mar 2012 08:19:51 +1300

On 2012-03-03 03:57, Washam Fan wrote:

<snip>

> OK then. I'd like other people to chime in to share their opinions on this.

As far as I can see, you could check the L7 information on the *first*
packet with a known flow label and a different srcIP, to see if it's
the same session. That way, the attacker would have to fake the whole
packet and not just the flow label - which is exactly what they have
to do today, as Willy says.

DDOS is certainly hard to defeat, but I don't think there is really
a new attack vector here.

    Brian

---

From: Washam Fan <[email-masked]>
Date: Sat, 3 Mar 2012 14:18:29 +0800

Hi Brian,

> to do today, as Willy says.

That way, you don't need share the same flow label across different
srcIPs, since the *first" packet would be L7 checked anyway. My take
was, flow label can be shared between different srcPorts as far as the
srcIP doesn't change.

> a new attack vector here.

Right.

Thanks,
washam

---

From: Washam Fan <[email-masked]>
Date: Sat, 3 Mar 2012 14:27:15 +0800

Hi Willy,

> is even worse when one wants to match HTTPS against HTTP and vice-versa.

flow label only used to identify a session between a given srcIP and a
given dstIP. If srcIP changes, it is beyond flow label's scope, and L7
DPI would be unavoidable. But L7 DPI only used on the *first* packet
after srcIP changes. If srcIP changes frequently, i admit flow label
can not benefit much in this case. Sorry if i have not made it
clearer.

Thanks,
washam

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 3 Mar 2012 09:21:47 +0100

Hi Washam,

> can not benefit much in this case. Sorry if i have not made it

No, L7 and flow label usage *cannot* be mixed that way. Either the LB
relies on the L7 or it relies on the flow label. The reason is that for
an LB to rely on L7, it must see *all* the L7 traffic of the session.
Take this very common example :

   User foo uses his smartphone to check his bank account while he's
   been driven in a friend's car.

   1) he has IP address 1.1.1.1 dynamically assigned by the nearest cell.
   2) he opens the browser
   3) he clicks on the bookmark
   4) the browser picks a random flow label 0x12345 and connects to the
      bank doing a setsockopt() on all outgoing connections so that all
      packets have the proper flow label.
   5) incoming connections pass through the L7 LB which only see
      the srcIP (1.1.1.1) and the flow label (0x12345). Connections
      are forwarded to the servers.
   6) the home page is totally stateless, the server does not emit
      any cookie.
   7) he logs in, so that's a new request.
   8) LB knows the flow label (0x12345) and directs the stream to the
      same server.
   9) bank servers emit a session cookie which can only be seen at
      L7 but nobody is looking.
  10) the car enters a tunnel. Lack of connectivity. When leaving
      the tunnel, the phone finally recovers connectivity with a
      new 1.1.1.2 address.
  11) foo clicks on an operation on the page.
  12) the LB receives a request from IP 1.1.1.2 with the flow label
      0x12345. If it considers the IP in association with the flow
      label, then it does not know this association. If it decides
      to decipher the stream and extract the cookie, it will see a
      cookie it does not know and will pick a new server.
  13) the request goes to the wrong server, it doesn't have the user
      context and the poor guy has lost his session (and cannot even
      log it off).

Now I know what you'll say, let's have the server emit the session
ID upon first request to the home page (6). Eventhough it's in general
not acceptable because it requires some deep application level changes,
it only pushes the problem one step further, because once the session
is renewed (eg: frequent sessions renewal due to the bank's security
measures, or user foo being idle for too long talking to his friend),
the LB doesn't catch the new cookie and will still have the issue at
step 12.

Another workaround would be to have the L7 LB insert the stickiness
cookie itself at stem 6. That still doesn't survive a change of server
which would leave the initial server's cookie in the request with a
flow label associated to another server.

Also there is an architectural issue in mixing the two methods the
way you're suggesting it, it forces the flow label LB and the L7 LB
to be the same, which doesn't scale at all. The goal of using the
flow label was to be able to use distinct components so that the
flow works like this :

                         +-------+
                         | ASIC- |  flow label present
  client packets ------->| based |------------------------> servers
                         | L3/L4 |
                         |   LB  |
                         +-------+
                             | no flow label
                             |       +----------+
                             |       | software |
                             +------>| based L7 |---------> servers
                                     |    LBs   |
                                     +----------+

With this model, you can have 1 entry point which processes all
flow label enabled sessions, and keep the good old L7 LB farm
for SSL decipher/HTTP extraction/SSL cipher to go to the servers.
It offers a smooth upgrade path for existing infrastructures,
because if this mechanism is implemented in high grade switches,
then the big investments on the LBs are maintained without any
change to continue to handle all clients who do not emit flow
labels. The L3/L4 LB implemented in the switch is just an L7 LB
bypass.

Regards,
Willy

---

From: Washam Fan <[email-masked]>
Date: Sat, 3 Mar 2012 21:48:02 +0800

Hi Willy,

> No, L7 and flow label usage *cannot* be mixed that way.

I was not suggesting this. I was think that what the draft says. The
authors might clarify that.

> Either the LB
> relies on the L7 or it relies on the flow label.

I am not an expert on LB. LB could have relied on combination of L7
and flow label, per the draft. The author might clarify here, too.

> The reason is that for

that is why the draft suggests how flow label can be used to save LB
from having to see *all* the L7 traffic of the session. LB has to only
see the *first* packet fo the session, because flow label can be used
to identify subsequent packets of the same session. If {srcIP, flow}
is unknown, it is the *first* packet of a new session and L7 DPI
should be taken on it.

>      cookie it does not know and will pick a new server.

Why the cookie is unknown? cookies can be used across different
'transport layer' sessions.

> flow label associated to another server.

it is cookie associated to a backend server, and different {srcIP,
flow} pairs can be associated to the same cookie.

>  client packets ------->| based |------------------------> servers

flow label can be used for L3/L4 LB obviously. But we are talking
about how flow label could ease L7 LB's job.

Thanks,
washam

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 3 Mar 2012 15:01:00 +0100

Hi Washam,

> authors might clarify that.

Thanks, that's precisely to get this feedback on what needs to be
clarified that the draft was posted to the list :-) I'm pretty sure
we need to work a lot on it still (and possibly fix some points).

(...)
> 'transport layer' sessions.

Because at none of the steps above the L7 LB had a chance to see it,
look carefully. The only steps it saw the request/response was 5-6,
and such steps did not carry any cookie yet.

> flow} pairs can be associated to the same cookie.

It depends. If it's the cookie associated to the server (which is
commonly called stickiness cookie or persistence cookie), it has to
be inserted by the L7 LB. Then if you have two distinct LBs (L4 and
L7), none of them knows when the other decides to rebalance the traffic
(eg: a faulty server has resumed, flow label might be updated but the
L7 cookie won't be touched by anyone).

> about how flow label could ease L7 LB's job.

Not exactly, more like to bypass them, hence reducing their work.
In fact, we can also make L7 LBs use and even forward a flow label
(and I'm planning on adding that to haproxy+provide some linux
patches when we make progress on this draft). But I really think
that if we *only* focus on getting all the job done by L7 LBs,
then we'd better use something else than those bits which are
present in every packet. The reason is that an L7 LB handles
connections. Having a TCP option to carry such an ID would be
more than enough. But it would not work on totally stateless
devices (eg: L3 LBs, switches). Using the bits at L3 makes this
possible. Right now swiches are able to process terabits of
aggregated traffic, while L7 LBs are only able of tens of
gigabits at best. This is why it's much better to be able to
bypass L7 LBs than just making their work easier.

Regards,
Willy