Subject: Minutes for INTSERV meetings in LA

From: John Wroclawski <[email-masked]>
Date: Fri, 22 Mar 1996 17:07:45 -0500

Minutes of the INTSERV WG meetings held March 5th, 1996.  (These
minutes do not cover the joint INTSERV/RSVP/IPATM/ROLC meeting held
March 6th, 1996).

TUESDAY AFTERNOON MEETING

This meeting was devoted to new topics in INT SERV.

INT SERV MIB PROGRESS REPORT - Fred Baker

JJ (John Krawczyk) and Fred have developed a MIB (6 objects).  Key
issue is how to divide INT SERV information with RSVP and ST-II
information.  MIB just tells you information about interface
properties (what link capacity, etc).  Stuff in RSVP and ST-II MIBs
tells you how the capacity is allocated.

By interface, you can.....

	* configure total bit rate that can be allocated

	* configure max bit rate for any one flow

	* display currently allocated bandwidth

	* current delay being experience by traffic (has been deleted)

	* add display parameter indicating bit rate allocated per service

	* currently allocated buffer is in the MIB (sum of burst sizes)

Fred expressed interest in getting this MIB done soon, so please send him
suggested objects.

INT SERV DATA OBJECTS - John Wroclawski

Internet-draft (draft-int-serv-data-formats-00.txt) describes some
requirements and a data format for I-S data objects within protocol
messages.  RSPECS & TSPECS, etc.

John presented a data object library:
 - Treats I-S data as opaque to setup protocol or similar
 - Supports:
   - creation and deletion of objects
   - ordered set of per-service information fields within each object
(ordered so RSpec, etc can express priority of different services in
request.
   - Addition and deletion of service-specific info from the set.
   - Listing of which service-specific parameters are actually present
for each service.
   - Creation, modification, and deletion of the parameters given for
each service.
   - Some simple error handling and checking (e.g., that IEEE
floating-point numbers are valid.)

Implementation is available as
ftp://mercury.lcs.mit.edu/pub/intserv/libintsrv.shar

SERVICE DISCRIMINATION FOR BEST-EFFORT TRAFFIC - Dave Clark

David Clark gave a talk on the topic of explicit allocation of best
effort traffic.  This represents a somewhat new topic for the
integrated services working group.  The initial goal for INT SERV was
to add support for new services like voice and video.  But discussions
on the mailing lists have shown that there is also a need to give the
ability to control best-effort quality of service too.  This talk
described one possible way to provide multiple levels of best-effort
traffic service.

The goal of the enhancement proposed by Clark was to allow different
users to be explicitly allocated different capacity service during
times of congestions. Capacity need not be measured using a simple
model such as a fixed minimun rate, but could be characterized in a
broad range of ways, which could model traffic with different degrees
of burstiness at different time frames.  The approach was to provide
each user with a usage profile, and tag traffic entering the network
as to whether it is in or out of this profile. At any point of
congestion, the router preferentially discards packets marked as being
out of profile. The result of this discard is that the sending TCP
perceives an indication of congestion and slows down. It will continue
this process until all of its packets are marked as in profile. At
that point, different TCPs may be getting very different services,
depending on what their profiles are.

The talk presented two schemes, one driven by controls at the sender,
and one by controls at the receiver. It also included some initial
simulations to suggest that somewhat accurate control on usage was
actually possible.

This talk reported ongoing work, not material being proposed for
standarization at this time.INSERT DAVE'S TALK

QUESTIONS: How complex are profiles?  Dave says quite complex.

Juha - Doesn't profile scheme depend on source and destination
profiles?  DDC notes that yes, if you've got a big profile but the
destination has small pipe, you won't get full throughput.  Matter of
education (folks need to be aware of Internet heterogeneity).

Floor: Suggest moving profile out to the hosts (some hosts are big pigs,
or some folks in corporation have different profiles -- CEO
vs. others).  DDC says he agrees.

Floor: Does IP ToS field on WFQ give similar service?  DDC not quite
(difference between precedence and in/out bits).

Floor: Question regarding pricing scheme. If ISPs are only charging
during congestion, doesn't that give them motivation to be congested?
DDC -- currently ISPs charge you for capacity (or modem hours).  ISPs
currently have this conundrum that there's no motivation to add
capacity except fear that competitor has better capacity and will
attract one's user.  And if one looks at congestion portion of fee, it
is pretty small -- and note that expectation scheme means we actually
don't charge during congestion (we charge for a promise you won't be
congested).

Greg Minshall: How does in/out marking work at multiple check points
(MIT, NEARNET, MCI all check)?  DDC's idea is that this is fine, and
that one can mark packets at out at multiple spots (and indeed, ISPs
can observe traffic marked in by MIT that the ISP needed to mark out
and suggest to MIT that they increase their expectation).

Abel Weinrib: If someone gave me TurboTCP that didn't back off.
Doesn't this fall apart?  DDC, says if we are overloaded, in packets
still get through, and outs gets nailed (and trying to be greedy just
ensures you get marked out).

MAPPING INT-SERV ONTO LEVEL 2 TECHNOLOGIES - JJ (John Krawczyk)

Points out world is more than just point-to-point link and ATM links.

Frame-Switched media problems.  LAN switches do simple fifo, frame
relay has no delay characterizations.

Current work, IEEE 802.1p (signalling via management and GARP -- no
admission control or policing -- just destination and source MACs and
priority scheduling)

Proposes that we do some work on implementing services (how-to guide
with current technologies and discussion of adaptation protocols that
we use over layer 2 protocols).

Question -- are vendors thinking about adding guarantees in MAC
switches (Ethernet)? Fred Baker points out that MAC switches sell not
on features but cost per port, so guarantees better be very very cheap
to include.

DON HOFFMAN AND RAJ YAVATKAR -- Mother May I protocol for Level 2.
Services that does admission control for 802.3.

Don, protocol he's developed using softstate and reservation messages
(MMIP).  Multicast protocol used to find subnet bandwidth manager, and
unicast protocol to talk to the manager (Mother).  No enforcement (but
some policing at hosts, who cooperate).

Raj Yavatkar is writing a spec for the protocol.

MINUTES OF THE TUESDAY EVENING MEETING

There were four items on the agenda: Discussion of the Guaranteed
Service (G) specification ID. Discussion of the Controlled Load (CL)
specification ID. Presentation on the Protected Best Effort
Service. Presentation on the issue of dealing with heterogeneity of
QoS service offering within the Internet.

The major objective of the evening was to reach resolution on the G
and CD specs.

Craig Partridge presented the material on the G ID. At the last
meeting, the sense was that the ID could go forward as a proposed
standard once a specific set of issues were resolved. After the last
meeting, Roch Guerin revised the spec to include the new concept of
the "slack term". This was circulated on the list, and no objections
and some support was detected.  The conclusion of this meeting was to
proceed with the spec in the form that includes the slack term.

Two small issues were noted prior to the meeting:
1)The draft is ambiguous on where reshaping point are, that is, places
where traffic is made to conform to a token bucket. The answer is that it
is done for two purposes:
a) done to restore shape to conformant flow; OR b) force merged flows into
a particular shape and perhaps discard excess traffic.
The draft will be revised to make this clear. The draft does not say what
to do when splitting reservations. The draft will make clear that reshaping
is to be done at that point as well.
2) The draft needs to be more clear on exactly how to compute the C and D
terms of the delay bound formula.
The answer is that D is precomputed when a box is configured, a property of
the box, while C is more complex, computed for each flow. The draft will so
indicate.

There was discussion of where to put an expanded discussion of how the
delays terms is computed. The conclusion was to put the meat of
discussion in an Informational RFC, but add some material to the
example implementation section of the standards document, and make
that document refer to the Informational RFC.

The proposed action, confirmed by the meeting, was to revise the spec
to meet the above objectives, and submit it for consideration as a
proposed standard. The Informational RFC would be prepared by Craig
Partridge and sent to the list for comment.

There was a final point of discussion as to whether the intent of the G
spec was that implementation over all media types was mandated. The answer
is that implementation is only anticipated over an appropriate subset of
media types used in the Internet.

Craig Partridge then chaired the next part of the meeting, in which
John Wroclawski presented the status of the CL spec. He presented a
prepared talk with slides, which is summarized below.

The status of the CL spec is that it was first presented in Dallas,
where the sense of the meeting is that it should proceed towards
proposed standard after seeing what comments arose on the net. There
were in fact a few substantive isues raised between that meeting and
now, which were summarized in this talk. There were three sorts of
issues. 1) Some wording was unclear. These points are noted and will
be fixed. 2) There was some uncertainty, or possible disagreement,
about the goals of CL. The talk reviewed the issues here, but the
assumption was that the goals were correct as stated. 3) There were
two technical questions as to how the features of the CL specification
meets those goals. The talk clarified these issues, and presented
specific design choices for review.

The goals of the CL spec, by way of review: 1) Flows within profile
(conforming to their proposed Tspec, should see the behavior
associated with an "unloaded" network. 2) Especially in the case of
multicast, the CL service and the Best Effort (BE) service must
coexist. Use of CL service for a flow should not make the service
worse than BE for that same flow. 3) (Not a CL goal but more a more
basic service issue:) Non rate-adaptive CL flows must not be able to
kill rate adaptive (TCP) traffic. This implies that a router needs to
support a sharing model other than brute force wins.  This point has
always been around, but CL emphasizes the need to deal with it, since
CL legitimizes non rate adaptive traffic.

The first technical point is whether the CL service is allowed to
reorder traffic that is out of profile. Some implementation schemes
reorder under overload. The following points are germane: 1)
Reordering may be easier for the network element implementor. 2)
Reordering is _never_ preferable for the application, since CL does
not have packet selectors to control which packets get delay, and CL
has no marking to show which packets have been out of profile at
previous hops on the path. Thus, there is every reason to think, in a
reordering scheme, that not just some but essentially all of the
packets may be treated as BE and reordered by the time the traffic
crosses a multihop network. However, reordering may be acceptable for
one class of apps -- reordering -insensitive, like vat. There was some
discussion as to the degree to which this is true.  3) There is no
reason to offer both schemes. If reordering is never preferable, if
implementing both, just do the better one.

Currently, the CL spec allows both reordering and non-reordering
implementations of the overload behavior. This range of options was
included to permit product differentiation and ease of implementation
in different circumstances. The alternative that was proposed on the
list was to preclude reordering implementations.

The discussion in the meeting raised the following points, positions
and issues. 1) The Internet today reorders packets. Its a fact. Since
Internet does reorder, leave the spec as is. 2) Reordering can hurt
voice as well.  But it was noted that reordering can only occur if
traffic is out of profile, not in the normal case. Further, it was not
clear that reordering implies a worse delay bound. 3) Due to possible
distortion of traffic shape as it passes through routers, traffic that
conforms to its profile at the edge may be non-conformant in the
middle of the net.

After discussion of several options -- preclude reordering, or say
"should not", or allow both, there was a first sense of the meeting to
prefer "should not". The point was reconsidered, and the final
conclusion was to put an extended discussion of the issue in the
implementation discussion in the spec, and to note the benefits of not
reordering in the evaluation criteria section.

The next technical issue raised was the role of the "B" parameter and its
relation to buffer allocation, for both real time and TCP-style flows.

Non rate-adaptive flows such as real time can exeed their reservation
for two reasons: first, multiple use of a shared reservation, and
second, a transient due to burst aggregation. The minimum necessary to
deal with this is a reasonable amount of buffering in the switch, but
a better approach is to drop packets when the delay gets too high. One
must not drop just because the traffic exceeds its Tspec, but only
when actual queues develop.  This queue management discipline is in
fact the same as the discipline suitable for rate-adaptive flows such
as TCP. Thus there is no recognized need to have two modes of buffer
management in CL, for flows whose rate can or cannot adapt. Adding two
explicit modes raised the further issue of insuring that one cannot
cheat and get better service by asking for the "wrong" mode. So the
proposal, since there is not a clear need for two modes, is not to
define a means, such as the "TCP" flag, to select or specify explicit
buffer management disciplines.

It was further noted, in response to questions from the floor
concerning the meaning of the B parameter for TCP, that if you are
asking for a reservation for TCP, with the goal that the TCP really
achieve the reserved rate in the Tspec, the reservation must tell the
router to allocate a specific number of buffers, not just link
capacity. Otherwise, packets of a typical TCP burst will get lost. In
the absence of a specific calculation of the expected TCP burst size
(as much as one round trip of traffic), one could try setting B to 2*
MTU, which is sufficient to deal with the TCP behavior in which two
packets are produced in response to one ack. Without this level of
buffer assurance, the source will have to pace its sending of packets,
which is difficult with today's systems and hardware. The question was
raised whether the CL spec stated that the buffers specified in the
Tspec must be explicitly allocated and dedicated to this flow. The
answer is no; the router is permitted to allocate both buffers and
capacity on a statistical basis. But all the Tspec parameters should
be considered by the admission control algorithm.

After some discussion, it was the sense of the meeting that trying to
distinguish the treatment that TCP and that real time flows need is
more complex than could be justified. The current form of the spec,
which does not make any distinction, was affirmed.

There was no discussion from the floor at the meeting of revising the
basic goals of CL, as they were presented by Wroclawski.

The action approved by the meeting was to clean up certain wording in
the spec, to leave the technical matters discussed above as they are
currently treated in the draft, and send the draft in for
consideration as a proposed standard.

The next item on the agenda was a presentation by Juha Heinanen on
Protected Best Effort (PBE) Service.

The motivation of this service is the claim that CL is too
complex. What is needed is to protect BE users from non-adaptive
flows. We need to replace Van Jacobson in his current role as real
time traffic police. So a PBE service is proposed, as a way to allow a
user to ask for a fair share of BE capacity. This request could be
used to control IP routing, etc.

The specific features of the PBE service are as follows.

PBE is intended for elastic, time insensitive applications. It may not
be suitable for some real time traffic. It provides an isolated BE
flow with optional min. BW. All unprotected BE traffic is considers as
a single PBE flow. The end to end behavior is that a flow gets a fair
share or minimum requested bandwidth (MRE), whichever is larger. There
should be fair transit delay among PBE packets on same path. There
should be ordered delivery of PBE packets as long as the path remains
the same. There is no guarantee of delivery of packets at higher then
the min. rate.

Why PBE? The motivation is to support all elastic non real time users
and only them. It support apps that don't know their traffic
characteristics, and protects the apps against greedy internet
users. It may provide an upper bound on transfer times.

The TSpec: min. requested BW, min. policed unit, max. packet size, and
might add peak rate.

A PBE flow is nonconformant if it exceeds the fair share or minimum
requested bandwidth, whichever is larger. Fair is defined according to
the max-min concept.

Policing: nonconformant traffic may be discarded, but should not as
long as conforming traffic can be protected. It must be discarded if
buffer resources would not be adequate. Under this approach, one can
calculate how much buffer space each flow uses, so we don't need RED.

Evaluation criteria: fairness can be measured with a parking lot algorithm.
Delay should depend only on active flows. Bandwidth should not depend on
the number or behavior of unprotected BE flows. Each flow should get its
minimum rate.

Following this presentation, there was discussion that raised the
following issues and concerns. 1) Is fairness a good thing? It seems
initially appealing, but is not obviously the right thing. 2) the
max-min definition of fairness may not be suitable in practice, and is
not at all simple to achieve. So this specification may not in fact be
simple. 3) The proposal provdes "fairness" only for those flows which
make a reservation (flow setup). Movement towards per-flow setup for
best effort may be a bad direction for the Internet. 4) The simplicity
of the service may come at the cost of constraining the implementation
options to essentially one approach, a per flow WFQ. This may not be
an acceptable constraint for the range of routers in the Internet.

It was noted that by relaxing the definition of fair allocation, one
might permit a wider range of implementations. However, this might
also remove some of the appealing initial simplicity of the service
goal.

It was agreed that this proposal would be further discussed and refined.

The next and last item on the agenda was a presentation by Lee Breslau
on Partial Deployment of the Service Model.

The three questions under consideration in this talk were 1) whether
we can expect all services to be deployed at all IS routers? If not,
2) can end to end services be composed out of the different
heterogeneous services that are found at each router, and if so, 3)
should the end node be allowed to control this?

Can we expect universal deployment if advanced Integrated Services
(IS)?  The answer seems obviously no. Implementation is hard on some
subnets, it is left to service providers to decide what to install,
etc. So how can we deploy and utilize real time services in a
heterogeneous environment with varying subnet characteristics? One
option is only to define services that are implementable on all subnet
technologies. This would imply a baseline service weaker than CL. This
reduces to too much a low common denominator.

Services at some but not all hops may be very useful. Even today, we
tunnel between non-RSVP nodes, which has same effect. And if service
is in place at critical (e.g. congested) nodes, a flow can get most of
the benefit. If we do not allow service variation, then the only
solution is just to reject all non-comforming requests. But if we
allow composition of heterogeneous service at different nodes, then we
must decide how to compose. One question is will apps want to know if
the service they get does not exactly conform? Some can just work
without knowing, but some will need/want to know the nature of the
actual service.

One mechanism to let apps know what is going on is to add a flag to
the request message saying that an app will or won't accept
replacement service at nodes. In multicast, merging is a problem for
this flag. Another mechanism is for routers to advertise availability
of services. This allows apps to know the degree to which a request
will achieve the service, but needs extra mechanism for advertising.

Service replacement might work as follows: when a router does not
offer a service, it offers some replacement, such as BE. When it gets
a service request, it maps it to the local replacement. Replacements
are characterized as reliable or unreliable. BE is a replacement
(perhaps unreliable) for anything. Replacement decisions are local.

Adspecs can be augmented to indicate to the requester what the degree of
replacement is as follows. Include in the adspec counts of the number of
routers that offered reliable and unreliable replacement as well as how
many implented the actual service. Each router can then export additional
substitute parameters, such as an estimate of D for guaranteed service.

In addition to the three questions at the beginning of the talk, there
are two more. 4) should we consider service-specific solutions for
this problem, and 5) do we want extensible solutions?

Following this presentation, the following issues were raised in the
discussion.

It was observed that this approach, if kept simple, is a good idea,
but it could be a real morass of complexity. Does thedetailed
information really matter to the end service? One extreme is to set
one bit in adspec saying a replacement occurred, but this cannot (for
example) distinguish between reliable replacement and BE
replacement. However, another approach noted was to use the bit to
signal the end node that some replacement occured, which could then
use diagnostic tools to find out what actually happened at each
node. This might permit the actual notification mechanism to be very
simple. There was some disagreement as to whether an adspec of three
integers is simple or complex.

Another concern was that since different services have different
parameters, replacement may be limited by wrong parameter
set. However, today this does not seem to be a real issue, since we
have a limited number of services, and this is not a fact with
them. Mostly today the replacement service would be BE, and in that
case we never need additional info.

A question was asked as to whether there a problem of how admission control
works in the presence of various sorts of service replacements?

There is a practical issue of what the individual vendor chooses to
call a true implementation of a service, a reliable replacement, and
an unreliable one. There may be marketing issues here. There is also a
technical issue as to when a replacement is reliable. For example, if
a BE service is currently very underloaded, it might at the moment
represent, in practice, a reliable replacement even for G. The current
proposal would preclude this sort of replacement as reliable, unless
the underloaded condition was known to persist for a very long
time. The determination might be administrative, rather than based on
measurement.

Craig Partridge offered, since he wanted to move forward on this aspect of
the problem, to will work with Lee to make this happen.

9:17 adjourn.