Subject: Comments on Controlled Load Specs

From: "Roch Guerin ((914) 784-7038)" <[email-masked]>
Date: Sat, 20 Jan 96 13:54:43 EST

John,
Sorry for not sending this earlier, but the usual LIFO stacking rule
kicked in (need to fix my scheduler... ;-)

In any case, the specs read much better and most of my comments are
probably only clarification issues, except for one that has actually
been the subject of recent discussions on the mailing list, i.e., the
policing and dropping/delaying at network elements.

Let me start with the more generic comments.

1. The first one has to do with the description of the end-to-end
behavior (Section 2) and related material in Section 4 that specifies
the Network Element Data Handling Requirements.  Specifically, in
Section 2, the end-to-end delay behavior is required to ensure that:

   > - The transit delay experienced by a very high percentage of the
   > delivered packets will not greatly exceed the minimum transmit
   > delay experienced by any successfully delivered packet (the
   > "speed-of-light delay").

The word *greatly* certainly leaves things reasonably open-ended, but I
think you may want to make clear that the user should not expect to see
close to zero queueing delay, i.e., have most packets experience only the
propagation delay.  This is sort of hinted in the note at the top of
page 3 on the definition of *unloaded*, but may be worth stating more
explicitly.  In particular, in the setting of, say, a campus network with
network elements using some kind of WFQ schedulers, you can experience
queueing delays significantly beyond the propagation delay (something of
the order of b/R where R is whatever rate the network elements have
allocated to the flow, e.g., can have R=r), and I think such a behavior
is acceptable from the point-of-view of the goals of thee controlled-load
specs.

On a similar note, on page 4 where the expected behavior of controlled
load service is further defined, the delay characteristics is described
as follows:

   > - Little or no average packet queueing delay over all timescales
   > significantly larger than the "burst time". The burst time is
   > defined as the time required for the flow's maximum size data burst
   > to be transmitted at the flow's requested transmission rate, where
   > the burst size and rate are given by the flow's TSpec, as described
   > below.

Again, what is probably more accurate is that a flow may expect to experience
*An average packet queueing delay up to of the order of a burst time, where
the burst time is defined as the time required for the flow's maximum size
data burst to be transmitted at the flow's requested transmission rate,
where the burst size and rate are given by the flow's TSpec.*

2. At the bottom of page 7 and in the second paragraph of page 11, split
points are mentioned as places where non-conformance with the TSpec may be
expected.  Maybe, I misunderstand the definition of *split point*, but
it seems this would bee more of a problem at *merge points*.  Also, this
is probably something that is to be expected pretty much everywhere as
soon as the flow has been through a certain number of network elements.
This is because, as Curtis mentions in his note, an initially conformant
flow will quickly become non-conformant as it can get more and more
perturbed as it crosses successive network elements (more on this later
in the context of policing).

3. In the middle of page 8, it is stated that

 > Other actions, such as reshaping the traffic stream (delaying packets
 > until they are compliant), are not allowed.

I think that the goal is really to ensure that if you can send a packet
because the link is idle, you wont delay it unnecessarily in order to meet
some compliance check.  In other words, you'd like the service to be
work conserving.  Now, this may be difficult to enforce or check.  In
addition, this is also closely coupled with what you end-up doing with
non-conformant traffic (again more on this below).

4. On page 11, it is stated that

 > Network elements supporting the controlled load service should
 > implement some mechanism in their best-effort queueing path to
 > discriminate between classes of best-effort traffic and provide
 > elastic traffic with protection from inelastic best-effort flows.

I think this is a very important point, but it may be a toll order to
require from a network element that it be able to distinguish between
elastic and non-elastic flows.  In many cases, it may not even have
any awareness of individual flows.  In the context of the sharing of
resources between best effort traffic and *overflow* (non-conformant?)
controlled traffic, what you probably need to have at a minimum is
some fairness mechanism (WFQ or other) that ensures that this overflow
traffic cannot take away more than a certain fraction (50% ??) of the
residual bandwidth on the link.  This should limit, though not eliminate,
the impact of this overflow traffic on the best effort traffic.

5. Let me now come to the trickier issue of policing.  The specs
currently state that an arriving packet is considered non-conformant
if it does not pass the token bucket test.  The problem is that this
is likely to occur routinely for many packets even if the source
had policed itself at the edge.  This means that we are now in a
situation where a user is behaving properly, but will be receiving
a level of service way below the *best effort service under unloaded
condition.*  Even with WFQ and most other schedulers, this will
happen because under normal congestion conditions (backlogged flows),
the *burst* size will typically increase as the flow crosses network
elements.  Assuming there is also best effort traffic contending for
the link, all the packets of the flow that are deemed non-conformant
will see poor service or be dropped.  This means that we don't
really satisfy the stated goals of the Controlled Load.

In order to *fix* this, the policing at intermediate network elements
should NOT be strictly based on the TSpec.  Curtis stated this in his
note, and there is also a similar mention on page 8, i.e.,

 > If resources are available, it is desirable for network elements at
 > points within the interior of the network (but *not* at edge traffic
 > entry points) to enforce slightly "relaxed" traffic parameters to
 > accommodate packet bursts somewhat larger than the actual TSpec.

but I think this is more than desirable, it is necessary in order to
ensure to meet the goal of providing *a level of service similar
to that of best effort service under unloaded condition.*

Thee tricky part is then to determine by how much to relax the policing,
and how to tie this in with the call admission that needs to be
performed.  Assuming that the rate the network elements guarantee
is the one given in the TSpec, this amounts to determining how much more
buffer is needed in order to accommodate the potentially larger burst
size that can be received.  As pointed out in the notes on page 8,
this most likely requires further study, but I think it is important
that the specs acknowledge that the policing is not done rigorously
on the TSpec.  For example, one could say that the network element
should attempt to buffer another b (the bucket size) amount of data
before starting marking packets as non-conformant.  This means that
it should assume a burst size of around 2*b when computing buffer
requirements at call admission (this is not a hard value as there
will clearly be some statistical muxing, and the actual value is
implementation dependent).

This should provide enough tolerance so that you can indeed give a flow
a level of service similar to that of best effort service under unloaded
condition.  This still leaves open the questions of what to do with
non-conformant packets and their impact on best effort, and that's
another tricky issue which also requires further study...  But I think
that getting real experience on what work and what does not, is needed
in order to understand better this issue and identify a solution.

Well, that's about all for now.  Got to run.

Roch