Subject: differential services bof

From: Juha Heinanen <[email-masked]>
Date: Fri, 15 Aug 1997 20:42:58 +0300

the purpose of this message is to open a discussion on the various
differential service proposals in order to make rapid progress in
getting the spec done.

after hearing in munich the presentations of clark, jacobson, and
kilkki in the differential services bof, it seemed to me that the
differences in these proposals are not that big.  

the main argument seemed to be between van and the other two.  van said
that an out-of-profile packet should be discarded at ingress (in order
to keep the network free of excess packets that may be later dropped),
whereas david and kalevi proposed to increase its discard priority and
(resources permitting) accept the packet to the network.

then there were minor questions on how many bits should be used to code
the discard priority and if there should be different profiles for
different classes of service (eg. delay sensitive, delay tolerant, etc.)
or only one profile.

i count myself to the camp of van and kalevi regarding the treatment of
out-of-profile packets.  it is much more user friendly to let the excess
packets to the network than to kill them immediately at ingress.
accepting the excess packets is safe, since it will not cause
in-of-profile packets to be dropped.  they may cause some throughput
loss for cross out-of-profile traffic, but i consider that a smaller
disadvantage than totally removing the change of out-of-profile traffic
trying to get through.  it should be up to the network operator (not the
rfc) to decide the access policy as long as the in-of-profile packets
are properly delivered.

what comes to the number of drop priority bits and the number of service
categories (profiles), the solution should be easy: allow the use of any
number of bits and service categories within the limits of the available
header bits.  for example, if one network only supports one priority bit
and the other supports three, then a mapping can easily defined between
these two (priority values 0-3 map to 0 and values 4-7 map to 1).  the
same applies to the service categories.  

the maximum values for ipv4 are 3 priority bits and 5 service category
bits.  ipv6 seems to have a 4 bit priority field and no tos field.  the
priority field would then need to be split into two in order to support
more than one tos.  by the way, the use of the word "priority" in the
ipv6 spec is quite misleading. the proposed codings specify service
categories rather than priorities.  it should be the task of this wg to
get those bits (as well as the ipv4 ones) redefined.

-- juha

---

From: David Clark <[email-masked]>
Date: Fri, 15 Aug 1997 19:02:39 -0500

Juha,
     I think it is good to get this discussion started.  Here are some
thoughts.

     There is one other specific but significant technical issue that came
up. Kalivi and I argue that the meaning of the preference bits should be
drop preference, Van argued that the meaning of the preference bits should
be priority queues.  This is not a minor difference, but reflects (I think)
a basic difference in the goals of the two approaches.

     My argument in favor of drop preference is that it keeps the packets
in order.  In Van's "hard drop if out of profile", he eliminates the option
of having a "multi-level priority flow", so out of order packets in a
single flow do not arise.  But that specific mechanical difference may just
mask the more basic difference in the goals of the two mechanisms.

     Here is the difference as I see it between drop preference and
priority queues  -- Van can tell me if/how I am wrong  :-).

      With priority, the more preferred packets always leave first. So they
get the lowest possible delay, subject only to the yet higher priority,
which can totally freeze out the lower priorities.  So priority does a
strong job of giving different levels of service a different "feel".
     Drop preference is intentionally a FIFO discipline among the
preference levels. As I use it, its goal is not to create "different
services", but one service (for example, our old friend best effort), and
to give more or less of it to one or another user.  To illustrate what I
mean,  if there is a queue in place, and then some higher preference
packets show up, they sit in the back of the queue behind the lower
preference packets. The dropper will soon come into play and start killing
new lower preference packets as they arrive, but the ones in the queue will
still be sent (unless you add a "drop at exit of queue" function as well as
my "drop at arrival to queue" function).
     I also said, in my talk, that I agree that we might need a different
service with a different feel, as Ed proposed concerning "SNA-style
transaction traffic",  In my approach, this would be a different queue,
selected by the ToS field, with drop preference optionally within the
queues.
     So both Van and I have two sorts of mechanisms -- different queues for
different services, and a way to regulate how much of the service different
users got.  Van uses priority as a mechanism inside the network to give
different services with different feel, and then regulates how much each
user gets of each service by hard-dropping at the edge.  I use different
queues in the network (without specifying the inter-queue service
discipline) to give different services with different feel, and use drop
preference to regulate how much each user gets.

    If this analysis is correct, then you cannot just modify Van's scheme
by saying that you prefer the idea of letting the "out traffic into the
system" and controlling it at the point of congestion. You also have to
specify how to allow different users to get different quantities of the
same service. If you start using the priority queues to regulate how much
traffic each user can send, they don't work at the same time to provide
different services with different look and feel.

       If you have different queues for different services (e.g. "normal"
and "low delay" best effort) you may sometimes want to pick different
service disciplines to control how the two are serviced. If you pick
priority, as Van did,  then the "low delay" can totally starve the
"normal". If you pick WFQ, then if you over-provision the "low delay"
queue, it will normally see real low delays, but if something gets out of
whack, and you get much more "low delay" than you thought, then the
"normal" queue can still get a share. Sometimes you may want one, and
sometimes the other. So I am not sure that we should standardize the
implementation approach if we define different services such as "normal"
and "low delay" best effort, just as we did not define the implementation
of "controlled load" real time. Instead, we described multiple
implementation approaches, described the benefits and limitations of each,
and left it as a matter for product differentiation.

     If you believe the discussion to this point, then adding more than two
levels of preference has a very different impact on the various schemes.
Now let me put words in Kalivi' mouth, so that _he_ can tell me I am
wrong...

    As I read Kalivi's ID, he is somewhere between me and Van, in that he
talks about his scheme delivering different QoS for different preference
levels. But the different QoS is primarily drop rates, I think.  (And, of
course, for good TCPs, drop rates translate into overall sending rate
differentiation.)  I would like to have him clarify if he believes that his
scheme can deliver different QoS's that differ in other significant ways. I
don't know how to do that with drop preference. It is possible that his
scheme and mine are not far apart.

    If you add more levels of drop preference to my scheme, you get a
finer-grained ability to express different profiles that apply at different
levels of congestion. In economics-speak, you are coding the user's utility
curve into the packet.  Kalivi sets them according to a fixed algorithm,
and I have not figured out exactly what utility curve that algorithm
expresses. We need some argument that it expresses a function that users
will find useful.  If you extend my scheme, one approach would be to let
the profile meter set the preference levels with some flexibility.  My
concern is that most users don't know their utility curve in any detail, so
it is an experiment to see if we can figure out how to use the bits.   In
the military context, where you might get major swings in provisioning, it
could be very helpful. Experimentation is needed.

    But this difference is minor compared to what happens if you add more
levels to Van's scheme. There, the consequence is to create more than two
services with different feel. And I would argue that having multiple levels
of priority is not helpful.  I don't think that we can construct more than
about two levels of best effort service that have a significantly different
"feel". We went through the same exercise with real time. Initially, we
proposed to have three levels of real time service, that would be
distinguished by the "feel" of the delay. The delays had to be
substantially different, to make the distinction worth having, and after we
screwed around with a range of offered loads and queue service disciplines,
we finally realized that if we forced the three levels of delay far enough
apart so that they really seemed different, the bottom two were worse than
best effort. So we bagged the whole idea, and just went for the very
simple, single level "controlled load". I suspect that the same will prove
true for best effort service as well -- I doubt that you can pull the
traffic apart into more than two queues, and still end up with queues that
actually have much of a difference in the serice they provide.

     There is also an implementation cost issue.  Priority queues require
multiple real queues, with a service discipline. So the complexity seems to
go up as the number of queues.  Multi-level drop preference has constant
(more or less) complexity -- you just run one of N copies of a RED
dropper,extending my proposal, where you executed one of 2 copies.

     So -- to summarize.

1) You cannot just change Van's scheme by removing the "drop if out of
profile" rule, since you leave his scheme with no way to regulate how much
of each service the users get.

2) Adding multiple levels of preference to my scheme is not a big deal, but
I am not convinced that you need it, so long as the provisioning is stable.

3) I think that adding more priority levels to Van's scheme is not useful.
It has a very different consequence.

4) I think that any scheme we use to allocate more or less of one service
to different users should keep the packets in order. Both drop preference
and "hard drop if out of profile" do this, but my scheme is more general,
and lets you offer a wider range of service profiles. With my scheme, you
could still create a sub-set of the profiles that did hard policing at the
profile meter.

Having said all that, let me put a question about which I am puzzled. Sally
Floyd objected to drop preference, on the following grounds.  If you have
responsive flows that adapt their sending rate, it may work fine. But if
you have hostile flows that just keep sending, you get the network full of
useless "out" packets, which flow part way through the net, messing up
other out packets on the way, only to be discarded. Sally  catagorizes this
as a form of congestion collapse.

Of course, this is the way that the Internet works today. Today,
non-responsive flows can flood the net with useless packets, killing off
well-behaved TCPs and each other.

If you take Van's approach, and "drop if out of profile" at the edge, then
(at least for the premium ToS) you absolutely elimiate this sort of stupid
flooding of the net by uncontrolled sources.

In my scheme, I have no cure for this silly behavior. My single service
class, regulated by drop preference, works just like the Internet today, in
that the outs make it all the way to the point of congestion before being
dropped. In my simulations, we demonstrated that we can discard massive
"out floods" without disturbing the "in" flows, but it is still true that
the "outs" are flowing part way through the net, messing each other up.

So now let me put words in _Sally's_ mouth...  I think what she said was
that if we used priority rather than drop preference, this would get
better. That implies to me that we are intending to use the priority to
regulate how much of the one service each person gets, rather than to
create different services. But (and I don't know how to use priority to do
a good job of that) if we are taking that approach, does priority queuing
help control the "flood of outs" problem?  I don't see how.

 (Note, hard policing to a fixed rate does not motivate adaptive sources,
but quite the opposite. It motivates sources that run at a fixed rate,
which is just below the profile they have been given. If it is fixed, why
build in adaptation?)

Enough, already...

Dave

---

From: Juha Heinanen <[email-masked]>
Date: Sat, 16 Aug 1997 11:53:49 +0300

dave,

thanks for your response.  i didn't realize that van was proposing real
priority with different queues rather than drop preference within the
same queue.  in that case the difference between your scheme and van's
would definitely be a major one.

to summarize once more, i would like to have one or more (maybe, as you
suggest later, two is enough) types of service (that map to one or more
profiles and queues) plus within each type of service at least two drop
preferences, which (if i have understod correctly) is quite close to
what you and kalevi have proposed.

   in order.  

keeping packets that belong to same flows in order is definitely a
dood idea and follows automatically from the above.

exactly.

   I use different

yes, the relationship between the tos queues doesn't necessarily need to
be strict priority.  for example, one can imagine visiting a delay
sensitive queue "often enough".

but, as i said in my earlier message, these "out" packets can only mess
up other "out" packets, not any "in" packets.  also

that is a too strict behaviour, hard to sell to a customer, and may lead
to network underutilization. 

   In my scheme, I have no cure for this silly behavior. 

real bad abuse can be prevented by having a peak rate parameter as part
of the profile: as long as the user is within the profile, the packets
get low drop preference, if out of the profile but below the peak rate,
the packets get a higher drop preference, and if above the peak rate,
the packets will get dropped already at the ingress.

-- juha

---

From: Paul Ferguson <[email-masked]>
Date: Sun, 17 Aug 1997 08:58:41 -0400

>same applies to the service categories.  

Is a binary (0 and 1) level of distinction enough?

I'm not convinced that it is, and I really like the idea of using the
IP precedence bit field, allowing 8 (0-7) levels of distinction, so that
something similar to WRED can provide for differentiated drop in the core.
This scheme could also be used in a such as that the lower 7 (0-6) could
be used to simply do preferential drop distinction and the highest value
could be used to indicate something else entirely.

Of course, this also comes back to an architectural model, where packets
are policed at the edges, and administration is done in the core.

- paul

---

From: Subramaniam Vincent <[email-masked]>
Date: Sun, 17 Aug 1997 11:51:00 -0700 (PDT)

> to network underutilization. 

Dave,

I have followed this discussion with a lot of interest. (and read the ID
ID in some detail). I have not been able to read van's draft yet, and 
I'd appreciate if someone can refer me to the source for it. 

> the packets will get dropped already at the ingress.

The issue here is the objection to carrying of 'out' packets which anyway 
get dropped. By its very nature the basic scheme involves carrying packets 
that are tagged 'out' - that is how user is able to express a drop 
preference. By adding another level of drop preference through a peak 
rate parameter to the traffic profile, we cannot do away with carrying 
of the 'out' packets. If all the abusers decided to collaboratively send 
out of the profile but below respective peak rates, we'd still be carrying 
lots of 'out' packets - we could now call it 'collaborative bad abuse'. 

Please correct me if I am wrong in this comment. In your scheme 
the drop preference can be used to (with more experience and  levels) to 
encode some preferred utility function into a packet stream from a user. 
To deliver this utility to the user it seems as though the  network 
*must* carry all packets UNTIL it really needs to drop. So by the 
very definition, this is the way utility is delivered to the user. In 
contrast, if we dropped all 'out' packets at edge - this is a more rigid 
scenario where 

a) we're assuming all 'out' packets will cause congestion and hence need to 
be dropped right now (edge)
b) we're assuming the profiles and user expectations are well matched to the 
user application, so that 'out' packets will really really be drop worthy. 
c) there might be far fewer real bad abusers than the overall number of 
users; so we might as well hard drop the packets. (section 6.4.2 
approaches this)

The above seem to be very specific scenarios. Your scheme, as I 
understand, seems to be a simpler extension to the best effort paradigm 
than the other schemes. (I have to qualify this by saying I have not yet 
read Van's scheme, but only your comparison of it with your scheme in the 
previous mail). Until more experimentation with profiles happen,  
and more users come to understand the profiles they need to set, I think 
we will be having a lot of 'out' packets to carry and drop. 

Furthermore, if a few more levels of drop  preference are added per  
single "feel" of service (one TOS queue), it does not seem any  easier to 
develop a useful hard drop policy. 

Having said that, I *think*, IFF the real concern is here is the wastage 
of resources in carrying 'out' packets that get dropped, then perhaps a 
different disincentive approach to 'out' packets needs to be explored. One 
approach I  wanted to think aloud was the possibility of using pricing for 
dropped 'out' packets. Since dropped 'out' packets are a cost to the 
network, (nuisance cost under congestion, at least), perhaps the pricing 
disincentive is worth thinking about. Is this too strict too ? 

It actually seems to get more complex/scary. For example, just like what is 
done for 'in' packets, the 'out' packets would need to be measured at 
service boundaries. If 'out' packet drops at congestion points are 
measured, a  proportion wise cost could be slapped to the contributors 
at the boundaries, by the dropping provider. The specific pricing scheme 
can of course vary from one provider to another. At the very least, 
'out' packet measurements at all boundaries are needed, along with some 
pricing scheme. I don't know if this will work.. do you think its worth 
exploring ? 

Even if this were worth considering  there are other problems; one is 
the suitability to receiver based profile control. Another problem with this 
kind of approach is that it is a long  term means of control. A (short 
term) flood of "out traffic" can still happen.... 

Once again, someone kindly point me to the draft paper for van's scheme. 
Dave raised some interesting questions on the other schemes. I'm eager to 
see responses to those questions as well. 

Vince

---

From: Paul Ferguson <[email-masked]>
Date: Sun, 17 Aug 1997 18:13:41 -0400

>develop a useful hard drop policy. 

I would argue that there is no difference between 'hard' or 'soft'
drop -- when a packet gets dropped, it gets dropped.

What really scares me here is the discussion of developing a
'light-weight' RSVP, when all that some people want is a method
to provide differing levels of best-effort.

If someone under-engineers their network, you can't do anything
about it anyway. The granularity provides just that, granularity.

The problem here is one of policy. Someone needs to define how (if there
are more than two levels of distinction) different adminsitrative domains
will either interpret different values, or simply define a standard
method of interpretation.

And lastly, some of this boils down to fiscal issues between providers,
since if they are not fundamentally incentivized, they'll not pay
attention the bits anyway.

- paul

---

From: Subramaniam Vincent <[email-masked]>
Date: Sun, 17 Aug 1997 18:01:37 -0700 (PDT)

> >develop a useful hard drop policy. 

Paul, I am sorry about the unclear language I used :). I referred to 
dropping the 'out' packets at the edges (as early as possible), as hard drop.

Vince

---

From: Wu-chang Feng <[email-masked]>
Date: Mon, 18 Aug 1997 15:36:25 -0400

We have done some work directly related to Dave Clark's proposal for a
drop preference bit.  The following paper describes mechanisms for
using the drop preference/priority bit for providing end-to-end
per-flow or per-flow group bandwidth guarantees in a network-friendly
manner.

W. Feng, D. Kandlur, D. Saha, K. Shin, "Adaptive Packet Marking for
Providing Differentiated Services in the Internet" IBM Research Report,
August 1997.

The paper explores two mechanisms for using this priority bit.  The
first is based on an external packet marking gateway which adaptively
marks a fraction of a connection's packets until its target bandwidth
is achieved.  Key advantages of this gateway is that it can be easily
deployed and has the ability to control the bandwidth usage of flow
groups.  The second mechanism is based on a source-integrated packet
marking scheme.  Key advantages of this is that it can provide the
bandwidth guarantees with an optimal (minimal) amount of priority
marking and it can detect and back-off when no service differentiation
is being done in the network.

Additional information and related research reports can be found at
  http://www.research.ibm.com/people/d/debanjan/pages/reports.html

Detailed simulation results and the paper can be found at
  http://www.eecs.umich.edu/~wuchang/pmg/

Comments would be greatly appreciated.

Wu

---

From: David Black <[email-masked]>
Date: Mon, 18 Aug 1997 18:53:28 -0400

> I am not convinced that you need it, so long as the provisioning is stable.

Adding another level of preference may be useful for availability, but this
would correspond to a different level of provisioning.  Specifically, in
addition to the IPI (In Profile Indicator) that corresponds to network
provisioning under normal circumstances, how about a BRI (Backhoe Resistant
Indicator [see http:// www.23.com/backhoe/ for the :-) ]) that corresponds to
network provisioning under after an arbitrary single disastrous (backhoe)
failure scenario (backhoe cuts an important trunk line).  The economic model is
that there are some users (e.g., Air Traffic Control) that will pay for the
value of not losing service when a physical failure (e.g., backhoe incident)
occurs.

Enforcing this probably requires independent third party auditing.  In Munich,
questions were raised about how to hold providers to normal service (IPI-based)
contracts, with the answer being that one could catch cheaters by looking at
drops that shouldn't have happened.  High availability (BRI-based) contracts
are more difficult in this regard because there will always be failure
scenarios that are out of scope.  In the limit, if every link is down, nobody
gets service, and provisioning against two backhoe failures may already be
excessive, provided that MTTRs aren't excessive.  In addition to the basic
provisioning problem, there are physical configuration problems -- if two
redundant links run through the same conduit group, they have a common failure
mode.  A third party auditor can verify that between the ISP and the
communications carriers that sufficient provisioning exists to meet the high
availability contracts under the defined failure scenarios.

--David

============================================================================
David L. Black            THE OPEN GROUP     Voice: +1 (617) 621-7347
The Open Group                                 Fax: +1 (617) 621-8696
Eleven Cambridge Center      RESEARCH       E-Mail: [email-masked]
Cambridge, MA  02142        INSTITUTE         http://www.opengroup.org/~dlb/
============================================================================

---

From: Paul Ferguson <[email-masked]>
Date: Mon, 18 Aug 1997 20:31:47 -0400

>would correspond to a different level of provisioning.  Specifically, in

I'm curious as to what your definition of 'provisioning' is.

More than two levels of abstraction would simply mean that you would
need to ensure that drop preference really did mean that each level
were probalistically more susceptible to drop as queue depth
increase -- the depth thresholds, of course, for each level should
be tunable on a hop-by-hop basis.

>
>Enforcing this probably requires independent third party auditing. 

Enforcing? One would assume that this is an issue to between
provider and subscriber, and provider and provider.

The real insidious issue, which Geoff Huston hinted at when asked
the same question from the floor in Munich, was how does a customer
measure whether he is actually getting what he expects? Especially
when there is no congestion?

>In Munich,

Let's please use another acronym which isn't already taken. Things get
confusing enough already. :-)

Measuring traffic between providers is easy enough -- by volume, or by
flow, with the appropriate precedence bit set. There is quite a bit of
granularity here. The problem is from the subscriber end.

- paul

---

From: Juha Heinanen <[email-masked]>
Date: Tue, 19 Aug 1997 07:50:27 +0300

yes, the implementation would need to have several queue thresholds and
once you are above a given threshold, then all traffic that belongs to
the corresponding or higher drop priority would be discarded.

you start sending traffic to the net and as long as it is in-profile,
every packet should go through at high propability. when some of the
packets are out-of-profile then you should get at least the profile's
worth of packet through plus your current fair share of the
out-of-profile packets (which you have no means to figure out in
advance).

-- juha

---

From: Juha Heinanen <[email-masked]>
Date: Tue, 19 Aug 1997 13:43:17 +0300

Well, wait a minute. Let's say that I have an 'in' threshold of
   256kbps -- all traffic within this bound is considered 'in' --
   but a local port speed of 1.5Mbps. I can burst up to port speed,
   just all traffic in excess of 256kbps is marked as 'out' of
   profile, similar in nature to a CIR, if you will.

   If the network is uncongested, end-to-end, then I will see no
   difference between 'in' and 'out' of profile traffic.

that is correct.  you only see the difference when the network is
congested, which currently seems to be the common case.

-- juha

---

From: Paul Ferguson <[email-masked]>
Date: Tue, 19 Aug 1997 06:38:40 -0400

Juha,

I guess the measurement portion of this is somewhat
orthogonal to the mechanics, but this was the situation.

- paul

---

From: Paul Ferguson <[email-masked]>
Date: Tue, 19 Aug 1997 06:46:06 -0400

>congested, which currently seems to be the common case.

Well, there are certainly other large-ish networks which are not
the Internet.  :-)

- paul

---

From: Wenjia Fang <[email-masked]>
Date: Tue, 19 Aug 1997 12:02:31 -0400 (EDT)

> when there is no congestion?

I see a pretty straightforward solution here: the user can write a
simple connection-oriented program and installed it on the source and
destination hosts. After the initial handshake, the sender part simply
sends a Constant Bit Rate (CBR) stream, and the receiver simply
counts the number of bytes it receives. Assuming the sender-based
scheme, then it is quite easy for the customer to verify how many
packets are being tagged as "In"s and how may packets actually went
through. Whether there is congestion or not in the network, the
received rate should be no less than what the ISP had promised. 

or am I missing something here? 

Wenjia Fang
LCS/Princeton

---

From: David Black <[email-masked]>
Date: Tue, 19 Aug 1997 12:27:31 -0400

> I'm curious as to what your definition of 'provisioning' is.

The word 'dimensioning' has also been used in this forum.  The fundamental
principle of all of the proposals is that the provider has a responsibility to
provide adequate network capacity so that `good' praffic (In Profile [Clark] or
Premium [Jacobson] or within NBR [Kilkki]) is delivered under normal
circumstances regardless of what else is going on (SIMA doesn't intend this as
a firm guarantee, but it's similar).  The additional availability level I have
in mind matches to a different provider responsibilty -- provide adequate
network capacity so that `high availability' traffic is delivered under
single catastrophic failure conditions (fiber bundle cut by backhoe is one
example, loss of a site and equipment there could be another).

> be tunable on a hop-by-hop basis.

Agreed, the point being that the tuning for the `better than best effort'
preference levels is matched against expected network capacity utilization
so that the traffic that is supposed to get through does get through.

> provider and subscriber, and provider and provider.

That's what I meant, perhaps `ensuring' is a better word -- I was thinking of
the subscriber (or whomever is paying the $$$s) as needing a way to verify that
high availability service really is being delivered.  I think that deliberately
crippling a network to test/verify that it doesn't drop high availability
traffic in disaster scenarios is not a good idea.

> when there is no congestion?

For high availability, take that paragraph and substitute `disastrous failure'
for 'congestion'. I think that a third party auditor can verify that a
provider's network has the capacity and redundancy to support the high
availability contracts that it has entered into.  This is akin to the sort of
supervision that banks get from the government to ensure the safety of
deposits, but the auditor need not be a government regulator.

--David

---

From: Paul Ferguson <[email-masked]>
Date: Tue, 19 Aug 1997 13:11:59 -0400

>deposits, but the auditor need not be a government regulator.

This is sounding more like 'ISP certification' and I think we should
intentionally stay away from anything of this nature, at least in
defining a framework or technology which relies on this within the IETF.

The point I was trying to make, and admittedly it is somewhat orthogonal
to the issue at hand, is what incentivizes the subscriber to either (a)
pay for a 'premium' service level and allow an upstream provider to
police & mark traffic, or (b) mark some arbitrary amount of traffic
as 'premium' if himself if he cannot discern the difference in service
he might get in doing so? If there is congestion, it is clearly easier
to understand how one might see the difference, depending how traffic
was marked. But in cases where there is no congestion, this becomes a
warm, touchy, feely service level agreement thing, and subscribers
generally like to see, touch, and feel the difference beforehand.

I suppose this is really an issue for IPPM, but it has me thinking
nonetheless.

So, never mind.  :-)

I still think that there is a compromise somewhere here between
all of these proposals, and it looks something like using the
Ip Precedence as the bit field for differentiation, and a
'weighted' RED in the backbone. I do not see the need for a
bandwidth broker, since this is inextricably related to how
organizations engineer their capacity, not to a mechanism which
brokers what might be available for the service.

The issue then becomes much simpler -- as all that needs to be
determined is what are standard transitive values for this bit
field.

- paul

---

From: Ben Laurie <[email-masked]>
Date: Tue, 19 Aug 1997 20:06:29 +0100

> received rate should be no less than what the ISP had promised.

If the customer has contracted to get x b/s and the ISP actually gives
.9x b/s + best effort, if there is no congestion, the customer will
still see x b/s. In fact, until congestion is severe, the customer will
see x b/s. So how can they tell they've been ripped off?

Add the likely possibility that SLAs will actually be of the form "I'll
give you x b/s 99% of the time", and the possibilities for abuse become
large. Of course, the moot point is whether the ISP has done anything
wrong if they actually deliver their SLA (by whatever means).

Cheers,

Ben.

-- 
Ben Laurie            |Phone: +44 (181) 994 6435|Apache Group member
Freelance Consultant  |Fax:   +44 (181) 994 6472|http://www.apache.org
and Technical Director|Email: [email-masked] |Apache-SSL author
A.L. Digital Ltd,     |http://www.algroup.co.uk/Apache-SSL
London, England.      |"Apache: TDG" http://www.ora.com/catalog/apache

---

From: David Black <[email-masked]>
Date: Tue, 19 Aug 1997 15:27:49 -0400

> defining a framework or technology which relies on this within the IETF.

I basically agree on the IETF role, but I'm not sure the conclusion that high
availability is to be avoided follows.  Let's return to the single bit for
marking traffic that should be immune to congestion.  The ease of verifying
this service depends on the nature of the traffic and guarantee. As has already
been pointed out, for the simple case of a fixed bandwidth unicast connection
between two points, this is easy to verify by counting marked packets in vs.
packets out.  That's an uninteresting case, because RSVP and the Integrated
Services framework already solve it (and hence we shouldn't need any new
technology).  The more interesting cases involve aggregation, for example,
buying congestion free bandwidth for all of an ISP's customers to access my
commercial web site (up to a specified bandwidth limit, where bigger limits
cost more $$$s).  Verifying this is more difficult because this is essentially
bandwidth into a cloud, and hence the endpoints are variable and the guarantee
has to be more statistical in nature.  While complex test cases could doubtless
be devised, a third party auditor might turn out to be very useful in practice.

In Munich, it was observed that the IETF can and should avoid such issues --
just specify the bit and what it means, while leaving issues of dimensioning/
provisioning, contract terms, and verification of contract deliverables up to
the contracting parties (who may or may not want to use a third party auditor).
A similar approach can be applied to the 'high availability' preference level,
but my opinion is that the increased difficulty of verifying this via traffic
under normal conditions (the failures being protected against should be rare)
will make a buyer of such service want to be able to audit the provider's
network availability in the presence of failures (or know that some third party
that the buyer trusts has done so) to avoid nasty surprises if/when a failure
happens.

> generally like to see, touch, and feel the difference beforehand.

Building on some of David Clark's comments in Munich, one might think of the
'premium' service level as 'congestion insurance.'  Observe that in cases where
there is no congestion, never has been, and never will be, a rational
subscriber will pay $0 for congestion insurance because there is no risk. Given
that we're having this discussion, congestion clearly is a real concern, and
subscribers who are interested in buying the sort of services under discussion
believe that congestion (or the risk thereof) is a serious enough risk/problem
that it's worth spending $$$s to do something about it.  An insurance model
roughly fits David Clark's notion that the price of premium access to an
uncongested network ought to be zero (because $0 is the rational price for
congestion insurance in situations where congestion is impossible).  An
insurance model ought to work for providers from a financial standpoint,
because it provides the kind of predictable cash flow needed to pay for network
capacity (by contrast to some sort of metering approach that charges only if
congestion actually occurs). The result is a "use it or lose it" approach to
marking packets -- having paid for the insurance, I'll mark the packets that
the insurance covers just in case congestion does occur. The flip side of this
is that absence of congestion will not get my current congestion insurance
refunded any more than absence of an accident will get my current auto
insurance refunded :-).

--David

---

From: Paul Ferguson <[email-masked]>
Date: Tue, 19 Aug 1997 16:12:05 -0400

>availability is to be avoided follows.  Let's return to the single bit for

I didn't imply that high availability shouldn't follow, but rather high
probability of not being dropped.

>Services framework already solve it (and hence we shouldn't need any new

This is indeed a concern, since the whole reason we are discussing these
issues of differentiation is because of the scalability problems of
deploying RSVP in the global Internet. Many large ISP's, for the most part,
won't deploy it. Not meaning to sound inflammatory here, but that was
one of the bullet items mentioned at the onset of the meeting in Munich,
if you recall, which was originally mentioned in Memphis.

(This is one of the reasons why I think that this may also not be
an appropriate mailing list for these types of discussions.)

>be devised, a third party auditor might turn out to be very useful in
practice.

That's all well and good, but I believe it is outside the scope of the
as yet unchartered working group. I'm principally concerned about allowing
loose differentiation of services. Compromises need to be made, somewhere
in the middle of the road between pervasive state maintenance and traditional
pervasive best-effort.

- paul

---

From: Jon Crowcroft <[email-masked]>
Date: Sun, 31 Aug 1997 09:46:00 +0100

>would definitely be a major one.

after a conversation with john wroclawski immediately after the
session, i reckon dave and john's scheme can implement van's - the key
"difference" is that van has two motivators for his simpler proposal
which amounts to the mandating of dropping packets out of profile:

1/ in a multi-provider environment, and without complex marking, there
is a good reason for dropping packets early in the flow rather than
later (spatially equivalent of RED which is temporal early drop....:-)
- you don't know traffic conditins downstream, so why risk
exacerbating them?
2/ business case: if an ISP commits to a rate (hmmm, sounds very
FR:-), then they like the user to stay to it too - this allow s them
(and other ISPs they peer with for premimium service) to dimension the
net (and the call acceptance which in this canse is a subscribe time
algorithm rather than rsvp-path/resv time...probably) - the fact that
an idle net does NOT et a preimium customer get more service than
their committed rate is an INCENTIVE for them not to use the premium
bit EXCXEPT when the net is congested....

dave and john's scheme is nicer for current TCP (you can ask for a TCP
rate rather than for a "circut equivalent" rate, which with curret
tcp's will not get you te same thing, but you can always measure it
and see what you can get)....although it seems to me that there is
still some research into exactly what queueing/shape/policer you can
implement at speed that will implement the MIT scheme.....(key word:
at speed!)

so the approaches are complimentary.....imho......we can deploy van's
first (implementing it in pretty much the same framework....)
cheers
jon

---

From: Subramaniam Vincent <[email-masked]>
Date: Mon, 8 Sep 1997 15:51:59 -0700 (PDT)

> to provide differing levels of best-effort.

Paul, I know you sent this message quite sometime back. But would 
appreciate if could you elaborate on what you meant by the above ? 

Vince

---

From: Paul Ferguson <[email-masked]>
Date: Fri, 12 Sep 1997 09:16:14 -0400

[cleaning up old e-mails]

>appreciate if could you elaborate on what you meant by the above ? 

Apparently I overlooked this message earlier -- apologies.

What I meant by the above was the fact that there are clearly
ways to provide differentiation without state maintenance. A
couple of examples have been discussed, but basically a method
to mark packets at network ingress with the appropriate IP
precedence, coupled with a preferential/weighted RED mechanism
in the core, could provide differentiation -- varying levels
of best effort, if you will.

Of course, this is just an opinion.

- paul