Subject: Re: [tcpm] Automatic IW draft posted

From: Joe Touch <[email-masked]>
Date: Mon, 03 Jan 2011 16:45:50 -0800

Hi Jerry (et al.),

I'd like to hear from others before addressing these issues in detail, 
but most of them are just a matter of picking a reasonable default or 
alternative. The parameters proposed in the doc were intended as a 
starting point for discussion.

Joe

On 12/26/2010 12:57 PM, Jerry Chu wrote:
> Hi Joe,
>
> Thanks for the writeup. Your proposal seems like a good step toward
> finding a permanent solution to the IW problem. The following are my
> comments:
>
> 1. The long check period (e.g., one month) is necessary to allow
> enough samples to avoid false negative/postives and ensure
> stability/convergence. But many hosts today won't survive that long
> period without a reboot, hence losing all the IW/counter statistics
> and have to start over again. This includes not only most of the
> client devices but many servers as well.
>
> What this implies is basically the algorithm will never (or rarely)
> have enough time to move IW, either up or down. Even it did, upon the
> next reboot the latest IW is lost. Does it start from scatch then?
>
> 2. To fix the above problem requires either some local persitent
> storage or some external (e.g., through the network) storage to save
> counters more frequently. This is certainly doable, but will
> introduce a lot of complexity, basically ruining the simplicity of
> the algorithm you proposed.
>
> For some hosts the above may not even be feasible.
>
> 3. The proposed check
>
> "if (IWnotchecked&&  ((ISN - seqno)/MSS<  IW))) {"
>
> under "3. During a retransmission,...", although simple (hence
> meeting the KISS requirement), counts against IW even though the
> initial burst may be smaller than IW. OTOH, tracking the actual
> burst sizes more accurately and counting them against 3..IW buckets
> as Mark Allman described earlier adde more complexity. Is it really
> necessary?
>
> For a web server the size distribution of the HTTP responses drops
> off quickly after certain threshold. Assuming we decide our loss
> tolerant level is 5%. When the IW is bumped to certain leve, e.g.,
> 20, there may be less than 5% of HTTP responses that are greater
> than 20*1460 = 30KB. Assuming IW<= 20 is all good but IW>  20 is
> all badness, lumping all the counters together under a single IW
> may never fail the 95% check, hence allowing IW to grow unbounded
> even though those connections with response sizes greater than 16
> always suffer. One may argue the small percentage of connections
> sort of justifying fudging it out. But if one judges from the
> volume (of just the initial bursts) it may not be insignificant.
>
> A good compromise may be to track only connections with burst
> sizes = IW, or>= IW minus some threshold, relying on the fact
> that the ones below IW minus some threshold have been verified
> before so it's not too risky not to continue to validate them.
>
> This does unfortunately require some code, although minimal, to be
> added to the fast path to check the initial burst size. In some
> implementations it may not even be easy to accurately delineate/
> compute the "burst" size, but one can always fudge in that case.
>
> 4. The proposal makes an assumption that the choice of IW is to
> be blamed if the percentage of connection experiencing pkt loss
> during its IW is greater than some threshold. But this may not be
> true. A host may experience high loss percentage regardless of IW.
> E.g., IW3 performs as bad as IW10.
>
> If the losses are spread across a large percentage of connections,
> then one can argue it's moot whether an optimal IW is picked because
> performance will suck for most connections anyway (assuming a high
> loss threshold of, e.g., 5%). But if the losses concentrate on only
> 5% of remote peers and these peers suffer real high pkt loss rate,
> this false positive will forbid IW to ever grow, and hence deprives
> the rest 95% connections of the benefit of a larger IW.
>
> I have no idea how often this false positive case might exist but
> if it turns out to be a large percentage then the proposal will be
> worthless in terms of addressing the small IW problem.
>
> 5. A possible fix may be to use two sets of counters rather than one.
> One tracks the loss rate of all the initial burst sizes<= 3. The
> other tracks the rest upto IW. Then compare the two, and allow IW to
> increase if the loss rate of the second set is within some threshold
> (either absolute or some percentage) of the first one.
>
> But one problem with the above is that the number of the base case
> (IW3) may have been negatively affected by other hosts moving up to
> a higher IW.
>
> 6. Regarding te sequence number wrap around problem, one can add a
> simple check to reset IWnotchecked when seqno goes beyond IW but
> this requres adding a check to the past path. I agree this issue is
> tiny that may be ignored and just allow the false positive.
>
> 7. [Ch10] - misses N. Dukkipati as a co-author.
>
> Jerry
>
> On Thu, Dec 23, 2010 at 3:38 PM, Joe Touch<[email-masked]>  wrote:
>> Hi, all,
>>
>> A preliminary draft has been posted:
>>
>> http://www.ietf.org/id/draft-touch-tcpm-automatic-iw-00.txt
>>
>> It includes the algorithm and a lot of other details, some fleshed out (the
>> intro), others as a list. The goal is to use this as a basis of initial
>> discussion.
>>
>> (I have not determined the author list yet, so it says TBD right now...)
>>
>> Joe
>> _______________________________________________
>> tcpm mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/tcpm
>>

---

From: "SCHARF, Michael" <[email-masked]>
Date: Tue, 4 Jan 2011 19:57:03 +0100

Joe,

the question of byte-based vs. segment-based window management may
require some further discussion: IMHO, the condition "if (IWnotchecked
&& ((ISN - seqno)/MSS < IW))) {" assumes that all sent segments have
full size.

If an app writes smaller pieces of data (telnet-like), this expression
will wrongly count a retransmission that occurs many RTTs after the
connection setup, even even if the IW was never fully used.

Also, if the sender sends IW segments smaller than the MSS immediately
after the connection setup (disabling Nagle) and if a loss occurs, this
expression will wrongly not count this retransmission even if IW was
indeed used, at least in terms of packets. This will in particular be an
issue if a bottleneck is packet-congestible.

Therefore, I want to back Jerry's statement that a more careful tracking
of the initial burst size may be required. For instance, after the
connection setup, one could count the packets and bytes until the
arrival of the first new ACK, assuming this to be a definition for the
initial burst. In this case, one can easily determine whether a burst of
IW was indeed sent (either bytes or packets). The statistics would then
consider only the connections that actually used the full IW. But,
unfortunately, this seems to require some additional fast path code.

BTW, in the Quick-Start implementation we wrote some time ago, we had to
implement a similar check in order to determine whether a retransmission
was out of the initial burst. As mandated by RFC 4782, we realized this
exactly by awaiting the first new ACK.

For Slow-Start after idle times, the whole issue is apparently more
complex, due to congestion window validation etc. IMHO it may make sense
to monitor the IW performance only after connection setup, to keep it
simple. In that case, it may be useful to set "IWnotchecked = 0" when
the congestion window is reset to IW after an idle time.

As a side note, IMHO it may make also sense to briefly comment in the
draft on the possibility of spurious retransmissions due to reordering,
even though this is probably a corner case that might just be ignored.

Michael

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Joe Touch
> Sent: Friday, December 24, 2010 12:38 AM
> To: [email-masked]
> Subject: [tcpm] Automatic IW draft posted
> 
> Hi, all,
> 
> It includes the algorithm and a lot of other details, some 
> fleshed out (the intro), others as a list. The goal is to use 
> this as a basis of initial discussion.
>

---

From: Nandita Dukkipati <[email-masked]>
Date: Wed, 5 Jan 2011 17:29:27 -0800

Joe,

I have three high level questions on the draft, mostly aimed at
understanding on why we should expect the algorithm to work well in
practice.

1) Timescale for determining IW impact.

One of the algorithm goals is to adapt to the network feedback (on IW
impact) over long timescales. A lot of things change over periods such as a
month -- network configurations, topologies, routing, server software,
applications, traffic patterns and so on. If the net change due to all the
non-IW reasons is larger than or in the same order as the measurable change
due to IW, then what does it mean to measure the impact of IW?

Please note that this is more then merely choosing the algorithm parameter;
it is a core design issue. My concern is partly rooted from our own
experience in the large-scale IW experiments. To average out the diurnal
patterns and the weekday/weekend differences, it was necessary to run the
experiments (baseline IW3 and experiment IW10) over at least one week, i.e.,
one week of IW3 and one week of IW10. It was an absolute pain to perform an
apples-to-apples comparison while adjusting for all the non-IW changes that
also took place within these weeks. And this was with a large jump from IW3
to IW10; the impact of a smaller IW change -- such as 2MSS increment -- will
be even harder to discern over longer periods.

(Soon after, we solved this problem by moving to a new and better experiment
infrastructure that allowed setting IW based on IP address hash, thus
allowing experimenting with multiple IWs in parallel.)

2) What if losses are bimodal across connections.

A core assumption of the proposal is that increasing IW will increase the
%connections with losses, which is then used to decide on
incrementing/decrementing IW. Again, the real experiments show otherwise.
One of the things we tried hard is to figure out what kinds of
networks/users we are negatively impacting by increasing losses. While we
always observed an overall increased retransmission rate, but not really an
increase in %connections or %HTTP responses experiencing losses.

But... it may well turn out that (if and when) everyone switches to a larger
IW, the above observation is no longer true. However, being a core part of
the proposal, I think the assumption needs better (preferably empirical)
evidence.

3) Convergence to uniform IW.

This is mentioned as one of the goals. With the current specification, my
sense is this goal will be hard to achieve. Depending on the region of users
served, the %connections with losses can be vastly different across the
globe, naturally lower in States/Western Europe, and higher for regions like
Africa/India. Since the threshold (5% or whatever number you choose) is a
constant, obviously the IW may never get a chance to grow beyond MinIW for
certain regions and can grow to MaxIW for others. Do we really need
convergence to a uniform value?... something to think about.

Thanks,
-Nandita

---

From: Joe Touch <[email-masked]>
Date: Wed, 05 Jan 2011 17:45:46 -0800

Hi, Nandita,

Overall, what I'd say is that:

- if the IW value fluctuates too much, dampen it, or just live with the 
fluctuations
	in specific, a fixed IW will be just as susceptible to
	network changes as well anyway

- if IW increases don't result in an increased number of connections 
with losses, then the IW isn't creating the loss
	if that happens, then increasing the IW is consistent
	with the behavior seen

- it's not clear that the IW will converge to a globally uniform value
	it might be more accurate to say that we hope it will
	converge over sets of connections that share resources

Joe

On 1/5/2011 5:29 PM, Nandita Dukkipati wrote:
> impact) over long timescales. A lot of things change over periods such
> as a month -- network configurations, topologies, routing, server
> software, applications, traffic patterns and so on. If the net change
> due to all the non-IW reasons is larger than or in the same order as the
> measurable change due to IW, then what does it mean to measure the
> impact of IW?
>
> Please note that this is more then merely choosing the algorithm
> parameter; it is a core design issue. My concern is partly rooted from
> our own experience in the large-scale IW experiments. To average out the
> diurnal patterns and the weekday/weekend differences, it was necessary
> to run the experiments (baseline IW3 and experiment IW10) over at least
> one week, i.e., one week of IW3 and one week of IW10. It was an absolute
> pain to perform an apples-to-apples comparison while adjusting for all
> the non-IW changes that also took place within these weeks. And this was
> with a large jump from IW3 to IW10; the impact of a smaller IW change --
> such as 2MSS increment -- will be even harder to discern over longer
> periods.
>
> (Soon after, we solved this problem by moving to a new and better
> experiment infrastructure that allowed setting IW based on IP address
> hash, thus allowing experimenting with multiple IWs in parallel.)
>
> A core assumption of the proposal is that increasing IW will increase
> the %connections with losses, which is then used to decide on
> incrementing/decrementing IW. Again, the real experiments show
> otherwise. One of the things we tried hard is to figure out what kinds
> of networks/users we are negatively impacting by increasing losses.
> While we always observed an overall increased retransmission rate, but
> not really an increase in %connections or %HTTP responses experiencing
> losses.
>
> But... it may well turn out that (if and when) everyone switches to a
> larger IW, the above observation is no longer true. However, being a
> core part of the proposal, I think the assumption needs better
> (preferably empirical) evidence.
>
> This is mentioned as one of the goals. With the current specification,
> my sense is this goal will be hard to achieve. Depending on the region
> of users served, the %connections with losses can be vastly different
> across the globe, naturally lower in States/Western Europe, and
> higher for regions like Africa/India. Since the threshold (5% or
> whatever number you choose) is a constant, obviously the IW may never
> get a chance to grow beyond MinIW for certain regions and can grow to
> MaxIW for others. Do we really need convergence to a uniform value?...
> something to think about.
>
> Thanks,
> -Nandita
>
> On Thu, Dec 23, 2010 at 3:38 PM, Joe Touch <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Hi, all,
>
>     It includes the algorithm and a lot of other details, some fleshed
>     out (the intro), others as a list. The goal is to use this as a
>     basis of initial discussion.
>
>     [email-masked] <mailto:[email-masked]>

---

From: Yuchung Cheng <[email-masked]>
Date: Wed, 5 Jan 2011 18:22:49 -0800

On Wed, Jan 5, 2011 at 5:45 PM, Joe Touch <[email-masked]> wrote:
> - if IW increases don't result in an increased number of connections with
> losses, then the IW isn't creating the loss
A higher IW may result in creating more losses for certain types of
networks/connections, hence the ratio of connection experiences remain
constant but the overall loss rate increase. We observe that in our
live traffic experiments. Her point is that % of connections
experiencing losses alone may not be sufficient to indicate an overly
sized IW.

---

From: Mark Allman <[email-masked]>
Date: Wed, 05 Jan 2011 22:54:00 -0500

> next reboot the latest IW is lost. Does it start from scatch then?

This seems like an excessively minor point to me.  Stuffing a few
counters and a config setting into non-volatile storage every so often
is pretty trivial.  E.g., think about doing it twice a day or something
so at most you lose 12 hours of state.  

Basically, if a host doesn't want to take the bother to try to track
this information over time then they can deal with the implications of
not using a larger initial cwnd because they are not tracking things
well enough to ensure they are doing so safely.

I just can't see this as some sort of big issue ...

> than 20*1460 = 30KB. Assuming IW <= 20 is all good but IW > 20 is

Right... this proposal turns on having a tolerance for "badness".  If we
want to meaningfully increase IW then there will be badness.  The
adaptive scheme sets up a way to manage that badness.  Can we construct
scenarios that are still bad?  Sure.  But, that is always the case.
E.g., IW=10 might be a fine general value, but it might always hurt
performance to some given host.

allman

---

From: Mark Allman <[email-masked]>
Date: Wed, 05 Jan 2011 22:54:08 -0500

> due to all the non-IW reasons is larger than or in the same order as
> the measurable change due to IW, then what does it mean to measure the
> impact of IW?

I don't follow this at all.  The IW is never the problem.  The
characteristics of the path are the problem.  So, yeah, the path changes
and the traffic mix changes and configs change and so the dynamics
change.  We are just trying to find a reasonable---not optimal---initial
setting for the context the host is operating in.  We are not trying to
somehow tease apart root causes here, but rather just trying to arrive
at a roughly OK initial setting for the vast majority of the cases.

> the proposal, I think the assumption needs better (preferably empirical)

I don't follow this.  The proposal is to *use* empirical evidence to
arrive at a reasonable value for an initial parameter setting.

> convergence to a uniform value?... something to think about.

I agree that this seemed like a bogus bit in Joe's draft to me.  We
don't need to converge to anything but a reasonable value for whatever
context a particular host operates in.

allman

---

From: Jerry Chu <[email-masked]>
Date: Wed, 5 Jan 2011 22:06:44 -0800

> so at most you lose 12 hours of state.

I think you've missed my main point (really #2 if you read on), which is not
about stuffing a few counters in some persistent storage (only needed before
system shutdown), but about whether the local storage is even available and
usable for this purpose.

This requirement will likely rule out many client devices, which is a shame
IMO.

> performance to some given host.

I understand the proposal strives to achieve two goals,
1) to prevent an overly aggressive IW from causing too much badness
(subject to some agreed-upon tolerance) on a very large (e.g., global)
scale,
2) to find an automated way to grow IW for the foreseeable future (e.g.,
over
the next decade)

and I'm being very realistic about it, i.e., not clinging to edge cases.
But I just haven't convinced myself the likelihood that the proposed
algorithm
actually works (i.e., achieving the two goals above) is high. What if the
scenarios I (and perhaps Nandita too) described turn out not to be the edge
but the norm? The algorithm will either not catch enough badness until IW
grows too large to cause real damage, or generate too much false positive to
prevent IW from ever growing. Neither is the outcome we want to see.

Jerry

---

From: Nandita Dukkipati <[email-masked]>
Date: Thu, 6 Jan 2011 00:09:08 -0800

> at a roughly OK initial setting for the vast majority of the cases.

I believe we all agree that the goal is to find a reasonable (not
necessarily optimal) value. The algorithm measures %connections with losses
between epochs T1 and T2, and decides to adjust IW based on this feedback;
my point is the measured feedback could be an artifact of non-IW related
changes over T2-T1, and yet it is used to adjust IW. I understood Steps 3
and 4 of the algo. as if it is measuring the impact of IW change, but looks
like that's not entirely the case. It seems like you care about %connections
with losses in the context that the host is operating regardless of whether
those losses are IW related. Is that correct?

> larger

Here's an example from experiments to clarify the question.
IW3: 96% connections have 0 losses. 4% connections experience losses.
IWN (N>3): 96% connections have 0 losses. 4% connections now experience even
further losses.

The point being: %connections with losses is one metric, but clearly doesn't
tell the whole story on the impact of IW. Why is it OK to just rely on
%lossy-connections as a measure of impact?

---

From: "Ilpo Järvinen" <[email-masked]>
Date: Thu, 6 Jan 2011 14:49:11 +0200 (EET)

On Thu, 6 Jan 2011, Nandita Dukkipati wrote:

>       otherwise.

It is not supposed to. Per destination problems and problems that affect 
all destinations are separate problems. And this proposal is not to 
address the first one. Yet, I'm again puzzled, you came to IETF to 
tell that larger IW does no harm, or at worst, hurts only oneself (more 
losses yes, but where's the harm?). Why would it now suddently turn out
to be something that should be avoided in a way that hurts 
performance of those 96%?

-- 
 i.

---

From: Mark Allman <[email-masked]>
Date: Thu, 06 Jan 2011 09:13:41 -0500

> necessarily optimal) value. The algorithm measures %connections with
> losses between epochs T1 and T2, and decides to adjust IW based on
> this feedback; my point is the measured feedback could be an artifact
> of non-IW related changes over T2-T1, and yet it is used to adjust
> IW. I understood Steps 3 and 4 of the algo. as if it is measuring the
> impact of IW change, but looks like that's not entirely the case. It
> seems like you care about %connections with losses in the context that
> the host is operating regardless of whether those losses are IW
> related. Is that correct?

I didn't write Joe's document.

To me you measure whether loss happened in the IW and that determines
whether it "works" or "doesn't work" for that connection.  I.e., your
T2-T1 is the initial window.  I am not entirely sure what you mean by
"non-IW related changes".  There is no need to understand what caused
the loss.  If it happens enough at IW=X then we decide that IW is too
aggressive. 

> The point being: %connections with losses is one metric, but clearly
> doesn't tell the whole story on the impact of IW. Why is it OK to just
> rely on %lossy-connections as a measure of impact?

Not, it isn't % lossy connections.  At least in my thinking it is % with
loss in the IW.  But, that aside.

If you are noting that there is a difference between losing one packet
in the initial window and losing IW-1 packets then I might buy that.
Propose a different assessment criteria if you have one in mind.  E.g.,
is overall loss rate in the IW better?  I.e., retransmits from IW /
packets sent in IW (across zillions of connections) must be within some
threshold?  Or, something?  If you see some better way to do this,
please clue us in.

To me there is simplicity in the binary "loss in IW or not?" notion and
since the goal I have in mind is to find something reasonable it seems
this other stuff is second order.  But, I'd be open to thinking about
other proposals.

allman

---

From: Mark Allman <[email-masked]>
Date: Thu, 06 Jan 2011 09:32:55 -0500

Jerry-

> I think you've missed my main point (really #2 if you read on), which
> is not about stuffing a few counters in some persistent storage (only
> needed before system shutdown), but about whether the local storage is
> even available and usable for this purpose.
> 
> This requirement will likely rule out many client devices, which is a
> shame IMO.

Maybe my mental model is busted but I am struggling with the notion that
there are "many" (what is that?) "client devices" (what are these?) that
both have no non-volatile storage and would benefit greatly from larger
IWs.

It does not seem onerous to me to use a design principle that says to be
more aggressive you have to track the implications of that
aggressiveness and reduce it if they are too great.  

If you have a situation whereby a host cannot track the implications
because it has no non-volatile storage and it doesn't stay up long
enough to get a reasonable sample then that host should not be
aggressive.

> over the next decade)
> and I'm being very realistic about it, i.e., not clinging to edge
> cases.  But I just haven't convinced myself the likelihood that the
> proposed algorithm actually works (i.e., achieving the two goals
> above) is high. What if the scenarios I (and perhaps Nandita too)
> described turn out not to be the edge but the norm? The algorithm will
> either not catch enough badness until IW grows too large to cause real
> damage, or generate too much false positive to prevent IW from ever
> growing. Neither is the outcome we want to see.

I just disagree.  

First, if we make only modest increases in the IW at fairly gross
intervals (e.g., bump by 1 packet at most once / month) then it seems to
me the probability of "causing real damage" is quite low.  Certainly
lower than increasing a static value from 3 to 10, say.

Second, I don't buy the notion of a "false positive".  If you have a
case where 50% of the connections use IW=X and work fine and 50% of the
connections that use IW=X see loss then that is not a reasonable default
IW to use.  Even if the 50% that don't work all go to one host across
some ratty link.  You can say that is holding down the growth
unnecessarily if you want, but my view is that is the wrong way to look
at it.  You can call that a "false positive", but there is nothing false
about it.  It is an empirical observation about half the traffic.  The
right way to look at it is as a *default* IW X is causing issues for
half your traffic and so it is not a good default.  If you want to be
more savvy and segregate this traffic because it is holding down the
global default for everyone else then that seems fine to me.  I.e., use
IW=3 for this ratty host and run everything else through the adaptive
procedure.  You can always go get more savvy about caching information.

But, I disagree that such a ratty link holding down the *default* IW is
somehow wrong.  It might be suboptimal.  It might be conservative.  But,
its probably reasonable.  We're not shooting for optimality.  And, even
if we were we'd never get it.

allman

---

From: Nandita Dukkipati <[email-masked]>
Date: Thu, 6 Jan 2011 10:46:05 -0800

> doesn't

You understand me incorrectly. I am trying to get a sense of how the
algorithm will work in practice, and hence these questions. Obviously, a
dynamic algorithm that bounds damage (if any) and lays out a growth path for
IW is better than a static setting. And I agree with that.

---

From: Jerry Chu <[email-masked]>
Date: Thu, 6 Jan 2011 11:28:40 -0800

> both have no non-volatile storage and would benefit greatly from larger

Joe's proposal aims at solving the IW growth problem for the next decade (or
two).
Can anyone predict what the future devices and traffic pattern will look
like?
It's also better to have less external dependencies.

> enough to get a reasonable sample then that host should not be

So do they get stuck at the bottom (i.e., IW3) forever? IW10 would be a much
preferred value to get stuck with. Otherwise I'd prefer to give them a path
to
grow, e.g., by figuring out a way to monitor the usage of IW and its effect
(if
feasible), and periodically publish progress reports giving green light to
some IW.

BTW, the proposal is not clear on hosts that never last longer than the
check
period due to system shutdown and inability to save counters. They don't get
to grow nor shrink the IW. But what's the IW to use to begin with (MaxIW or
MinIW??)

> lower than increasing a static value from 3 to 10, say.

There is a MaxIW based on year so it'd take years for IW to grow. But my
point was if the algorithm just fails to catch badness, although it may take
a long time to grow IW but it will get there at some point to cause damage.

> if we were we'd never get it.

Like I said repeatedly this is not about some minor optimization. It'd be
silly
and futile to try to seek optimization in the grand scheme of things. My
concern
is that some seemingly edge cases might turn out not to be so as described
in
my bullet 4 previously (cut&paste) below. The threshold of 50% you used
above
is unreal and trivialized the case. A realistic threshold might be between
5-10%.
Again picking a good threshold is another difficult task. Too loose, you
aren't
catching any badness. Too tight, you shutdown IW growth completely. Is it
even
possible to have a single, threshold that is "good enough" (no, I'm not
shooting
for optimality) for global use?

(quote from previous comments)
"4. The proposal makes an assumption that the choice of IW is to
be blamed if the percentage of connection experiencing pkt loss
during its IW is greater than some threshold. But this may not be
true. A host may experience high loss percentage regardless of IW.
E.g., IW3 performs as bad as IW10.

I have no idea how often this false positive case might exist but
if it turns out to be a large percentage then the proposal will be
worthless in terms of addressing the small IW problem."

Jerry

---

From: Mark Allman <[email-masked]>
Date: Thu, 06 Jan 2011 15:21:43 -0500

> There is a MaxIW based on year so it'd take years for IW to grow. But
> my point was if the algorithm just fails to catch badness, although it
> may take a long time to grow IW but it will get there at some point to
> cause damage.

Sure... the algorithm has one definition of badness (loss in the initial
cwnd).  If you have a better definition of badness please lay it on us.

And, yes, ultimately we are making an engineering judgment that says the
progression of the IW is conservative enough that we do not believe it
will cause big issues.  Is that true?  I don't know.  But, (a) I think
the progress is pretty conservative and so the magnitude of any damage
is likely to be modest at worst and (b) there is a built-in way to
observe the issues and back the IW off.

This is, IMO, very similar to making an engineering judgment of some
static IW (like IW=10).  What we have is a collection of anecdotes that
say (to some of us) its likely OK.  Do we know it will be OK when
everyone uses it?  Do we know it won't be Quite Bad for various pieces
of the network?  Do we know it won't "cause damage"?  We don't.

> A realistic threshold might be between 5-10%.  Again picking a good
> threshold is another difficult task. Too loose, you aren't catching
> any badness. Too tight, you shutdown IW growth completely. Is it even
> possible to have a single, threshold that is "good enough" (no, I'm
> not shooting for optimality) for global use?

Well, I don't expect to agree here, but I think anything is 5--10% is
fine.  I also don't care if the arrived at IW is 7 or 9 or 11.  This is
an initial parameter.  We should strive to simply have it set somewhere
within the right ballpark.

The adaptive scheme, I believe, flips the decision from something we
don't care about (the initial cwnd value) to something we do care about
(the implications of TCP's initial settings).  The arguments about
IW=10, say, have not been "well, that's not a power of 2 and so I hate
it" or "I love it because my birthday is the 10th" or whatever.  Rather,
the discussion has focused on how that IW impacts the traffic and the
network.  Nobody cares about the number.

So, let's just directly focus on something we care about.  If our
tolerance is X% then what we're saying is that we want to get the best
possible performance with the constraint that <= X% of the connections
should experience loss within the initial cwnd.  That statement says
nothing about how we want to evolve the IW parameter.  That statement is
an objective statement of a property of our traffic.  That will drive an
IW choice.  But, the IW choice is implementing the policy statement and
therefore whether it grows or shrinks or whatever is immaterial.
Rather, what we care about is the policy statement.  I think we're still
getting too hung up on "what will the IW be?".

And, if you want to be smarter about things and happen to notice its one
host (say) causing much of the loss in the initial window and want to
drop the IW for that host and let the IW adapt higher for the remainder
then that's fine.  In the aggregate you're still <= X%.  So, depending
on how smart you want to be you don't have to let the tail wag the dog.
But, if you don't want the complexity then you may have to live with a
less optimal result.  But, that's *your* choice as an implementer /
operator.

allman