Subject: design notes

From: Scott Shenker <[email-masked]>
Date: Thu, 16 Nov 1995 15:05:02 PST

Internet Engineering Task Force                   Integrated Services WG
                                                           Scott Shenker

           Design Notes on the the Specification of Services

The current set of service specifications and related documents reflect
one set of design choices.  In most cases there were a range of possible
options and, in some cases, a wide range of opinions about the
appropriate design choice.  This memo describes some of the thinking
behind these design decisions, and particularly addresses the recent
changes in the drafts and highlights remaining open issues.  Many of the
issues mentioned below will be brought up for discussion at our Dallas
WG meeting.  This document is intended to provide the background for
those, and subsequent, design discussions.

GENERAL CHANGES

 New Document

   There is now a draft describing "general" characterizations, that is
   those characterizations that are not associated with any particular
   service; examples include latency, bandwidth, and hop count.  See
   below for a discussion of one particular characterization parameter,
   the IP hop count, which is in flux.

 Predictive Service

   The Internet Draft describing Predictive service, which had expired,
   has now been resurrected.  Predictive service is very closely modeled
   on controlled delay, and in fact the service specification is largely
   copied from there.  The only salient difference is that predictive
   service provides a delay-bound at each hop, and controlled delay does
   not.  The end-to-end delay bound provided by predictive service can
   be known by the endpoints only if characterizations are made
   available by the setup protocol.  Thus, the difference between

Shenker                                                         [Page 1]
Design Notes                                           November 15, 1995

   controlled delay and predictive service is significant only in
   contexts where characterizations are made available to endpoints.

 Organizational Changes in the Template Document

   The "Resulting Service" section was changed to the "End-to-End
   Behavior" section and moved to immediately follow the introduction.
   The "Resulting Service" and "Motivation" sections had considerable
   overlap, much of which has been trimmed.

 Editorial Changes

   An effort was made to harmonize the presentation in the various
   documents so that they did not use different language to communicate
   the same concept.  There is substantial reuse of text between the
   various documents.

 Heterogeneity

   Not all network elements will offer the same set of services; for
   instance, some elements might offer only controlled delay service,
   and others might offer only guaranteed service.  There is a new
   document describing one proposal for dealing with this heterogeneity
   in services offered by network elements.  While there is not yet
   consensus on this or any other particular proposal for dealing with
   the inevitable heterogeneity, we are distributing this document now
   to spur discussions on this important topic.  We expect to devote
   some time to this issue at the Dallas meeting.

TECHNICAL ISSUES

 Policing: What to Do

   A general design principle for the addition of real-time services
   into the Internet is that they should not obstruct the adequate
   delivery of best effort traffic.  More precisely, the presence of a
   reservation for a flow should not cause the flow to receive worse

Shenker                                                         [Page 2]
Design Notes                                           November 15, 1995

   service than if there were no reservation in place.  Previous
   versions of the service specifications called for the dropping of
   nonconformant packets.  This leads to a situation where a receiver
   (assuming receiver-oriented reservations as in RSVP) reserving with
   too small a rate can cause the vast majority of packets to be
   discarded.  For instance, if a source generated a traffic stream was
   best described by a token bucket with r=1mbps and b=50kb, but some
   receiver made a reservation with a token bucket with r=1kbps and
   b=1kb, then almost all of the packets would be discarded.  This can
   be considered a (perhaps unintentional) denial-of-service attack on
   receivers that wished to receive the data via best effort service.
   This is widely seen as an unacceptable vulnerability and in violation
   of the aforementioned general design principle.

   An alternative approach is to treat the nonconforming packets as if
   there were no reservation state in place; i.e., use the default
   service for nonconforming packet.  Currently, this means treating
   nonconforming packets as best effort traffic. (In the future, there
   will hopefully be a coherent approach to using ToS bits and/or other
   mechanisms to determine the service delivered to packets for which
   there is no reservation state, but currently things are pretty
   primitive.  Nothing in our policing proposal should be read as
   restricting the variety of future "reservation-less" services.)  This
   allows receivers who desire best effort service to see less
   degradation in their service due to the presence of reservations for
   the same traffic; some degradation might still occur in that the
   resulting delay distribution would be changed because some packets
   were serviced as best effort and others as reserved, and perhaps the
   resulting delay distribution would be harder to adapt to.

   This alternative, however, is not without problems.  The policing
   rules of the currently contemplated services -- where one only
   polices at the edge of the network, at heterogeneous branch points,
   and at source merge points -- leads to the problem that the
   nonconforming packets may not be policed again at the next hop.  For
   instance, in the example above, most of the packets in the 1mbps
   stream of traffic would be relegated to best effort service at the
   edge of the network where the flow was first policed.  However, at
   the next hop, if it is not a heterogeneous branch point or a source
   merge point, the flow would not be policed.  Thus, a severely out of
   conformance flow will be entering a network element that is relying
   on upstream policing to protect it.  This could perhaps interfere
   with the delivery of the reserved service at that hop.  As we discuss
   below, one could invoke policing everywhere to prevent this, but
   entails its own problems.

   A better solution would be to "mark" the nonconforming packets at the
   policing point, so that all subsequent network elements would know to

Shenker                                                         [Page 3]
Design Notes                                           November 15, 1995

   treat the packets with the default service and not according to the
   reserved quality of service.  This allows all traffic to continue
   flowing through the network (as opposed to dropping it when
   nonconformant) while still protecting network elements from severely
   out of conformance flows.  We are investigating such a marking
   facility now.

   In the meantime, it seems better to expose the real-time traffic to
   some risk (as results from treating nonconforming packets as best
   effort) rather than exposing best effort receivers to denial-of-
   service attacks.  Thus, we have chosen to police by relegating to
   default behavior (which now is just best effort).  However, this
   design choice does mean that we need to make progress on defining a
   marking facility.

   Another choice is to perhaps try to limit the allowable TSpecs (i.e.,
   one cannot make a reservation for a TSpec smaller than what the
   source has declared).  This introduces a new architectural construct
   into the service specifications: the source's description of its own
   traffic, as opposed to the TSpec which can come from the receiver (as
   in RSVP) or whoever installs the reservation state.  This avenue
   imposes more requirements on the setup protocol, and for the time
   being we are not pursuing this approach.

   Yet another approach is to have policing done at all routers.  This
   would remove the need for marking; each network element would test
   for conformance and give out-of-conformance packets default service,
   and would not need to be protected by the upstream policing.  What
   are the problems with this approach?  Most services will introduce
   jitter into traffic streams, and so policing everywhere in a naive
   fashion would result in packets that were conformant upon entering
   the network possibly being judged nonconformant at some interior
   router.  This would lead to unacceptable performance.  Thus, all
   interior network elements would have to reshape in addition to
   policing; that is, when packets arrived that would have been judged
   out-of-conformance, they would be held in a buffer until they met the
   TSpec requirements.  Reshaping at all routers may introduce
   significant additional queueing delays for streams that were
   conformant on entry.  Moreover, reshaping everywhere, if reshaping
   allowed any significant buffering, would certainly result in
   significantly increased delays for flows that are out-of-conformance
   upon entry, and thus would lead again to a mild form of denial of
   service (in that making a reservation for an inappropriately small
   TSpec would lead to increased delays for the previously best-effort
   flow).  But the tradeoffs here at not clear.  We don't really know
   how much delay reshaping would typically introduce.  For cases with
   heterogeneous TSpecs, or source merge points, there will be some
   interior policing no matter what.  It is not entirely clear that

Shenker                                                         [Page 4]
Design Notes                                           November 15, 1995

   policing everywhere is significantly worse than policing at some
   interior points.

   A final point is that the ill effects of policing/reshaping
   everywhere could be mitigated by using somewhat relaxed policing
   parameters in the interior routers.  However, it is not clear how to
   set these relaxed parameters, and to what extent there needs to be
   uniformity in this degree of relaxation.

   The various design choices, and some related issues, should be
   discussed at the WG meeting.  At this time, this appears to the
   single most vexing open problem.

 Policing: Where to Do It

   Language was added about policing at heterogeneous branch points.
   More precisely, consider a router where there are several downstream
   links for a flow (i.e., a split point or branch point).  If the
   reservations on the various links have different TSpecs, then the
   upstream link with have a larger TSpec than at least one of the
   downstream links.  The links with smaller TSpecs must then police the
   flow, because the policing done upstream does not ensure that the
   traffic stream conforms to their TSpec.  If all reservation requests
   for a given flow have the same TSpec, there are no heterogeneous
   branch points.

 Packet Processing and Header Overhead

   Many network elements are limited not just by the bandwidth of the
   associated link but also by their ability to process packets.  Also,
   some subnet technologies append their own link-layer headers, which
   add additional per-packet bandwidth overhead.  A token bucket
   specification of traffic describes the bandwidth usage, but not the
   likely rate of packet arrivals.  A minimum policed unit, m, was added
   to the TSpec.  Incoming packets of size smaller than m are treated by
   the policing function as if they were of length m.  This limits the
   maximal packet arrival rate.  An arrival process conforming to an
   (r,b) token bucket in terms of bandwidth conforms to a (r/m,b/m)
   token bucket in terms of packets.  This allows the network element to
   compute a (perhaps extremely conservative) worst-case estimate of the
   link and packet processing overhead.  While a more accurate estimate
   would be desirable, we see no practical way to obtain one.  Moreover,
   in guaranteed service one must use the worst-case computation, and in
   predictive and controlled delay the admission control decisions are
   likely to rely heavily on measurements and less on the worst-case

Shenker                                                         [Page 5]
Design Notes                                           November 15, 1995

   computation.  Thus, we expect that the negative impact of having only
   rough estimate of the resulting overhead to not be a serious
   disadvantage.

 Fragmentation

   Packets larger than a link's MTU must be fragmented.  Fragmentation
   both imposes extra processing overhead on the network element, and
   also changes the nature of the traffic stream for downstream network
   elements.  In particular, fragmentation causes an increase in the
   number of packets in the flow.  As observed above, some routers have
   limitations on their packet processing rate and thus would need to
   know about this increase before knowing if they could support the
   requested service with this fragmented traffic stream.  Also, the
   increase in the number of packets may cause a higher proportion of
   link-layer overhead (in terms of headers) and again this may affect
   the network element's ability to service the flow.

   An initial design point is to disallow fragmentation for the proposed
   services.  Each reservation request's TSpec includes a specification
   of the maximum packet size M of the flow.  Network elements must
   reject the request if their MTU is less than this maximum packet
   size.  In addition, MTUs are considered part of a network element's
   characterization parameters and can be made available to the
   endpoints.  In this way, endpoints will know in advance if the path
   supports the source's maximal packet size.

   This design choice is attractive because the alternatives are so
   ugly; determining the exact nature of the flow after fragmentation is
   extremely difficult.  One could have the service module compute the
   worst case TSpec that would result after a given TSpec was
   fragmented.  This new TSpec will be extremely conservative.
   Moreover, this new TSpec would have to be used at all network
   elements downstream of the fragmenting router and arranging for this
   rewriting and distribution of the new TSpec is difficult.  For
   instance, in RSVP the reservation messages travel upstream so that
   the TSpec, once rewritten, travels in the wrong direction.  There are
   other ways to distribute a rewritten TSpec (such as distribute in
   RSVP PATH messages) but they seem architecturally inappropriate (the
   TSpec contained in a service request is specified by the receiver in
   RSVP) and overly constraining (in that all receivers would have to
   use the same TSpec).  One could provide a characterization parameter
   which reflects the end-to-end rewriting of the traffic descriptor
   provided by the source, which could then be used by the receiver in a
   service request.  However, that results in the use at all hops of the
   conservative rewritten TSpec, even at hops upstream of the

Shenker                                                         [Page 6]
Design Notes                                           November 15, 1995

   fragmentation.

   A different set of options is to require that the fragmented stream
   be reshaped or policed to conform to the original TSpec (this
   reshaping/policing could be done at the fragmenting network element
   or at hops immediately following fragmentation).  If fragmentation is
   significant, then it is likely that this reshaping will cause
   substantial delays.  To compensate, the receiver could reserve with a
   TSpec larger than that needed for the original flow, but this TSpec
   would be used along the entire path, not just over the fragmented
   portion.  Whether or not these, or other, options are worth adopting
   will depend on the severity and ubiquity of the fragmentation
   problem.  For now, the decision is to disallow fragmentation.  Note
   that one might allow fragmentation for P and CD service, since they
   are more tolerant of failures.  However, a nonpoliced but fragmented
   flow might impose significant penalties on other flows, and so for
   now we disallow such fragmentation for all the currently contemplated
   services.

   The policing function exercised on packets that exceed the requested
   maximal packet size M is that they are relegated to best effort
   service (or whatever the default is).  If best effort service
   fragments oversize packets, then these packets are fragmented and
   delivered; if best effort service drops oversize packets, then these
   packets are dropped.

 Latency

   Latency is one of the general characterization parameters.  Latency
   is defined to be the minimal possible packet delay along the path
   taken by the flow.  This parameter allows applications using
   guaranteed or predictive service to calculate the maximal and minimal
   delays.  In the typical case, latency is just the speed-of-light
   delay of the associated link.  More generally, this delay may result
   from "speed-of-light" propagation delay, or from packet processing
   limitations, or both.

   There are some subnet technologies for which determination of the
   latency is difficult.  In particular, on multi-access subnets the
   latency between different pairs of routers may vary by several orders
   of magnitude, as in an ethernet bridged via satellite to another
   ethernet.  The existence of multiple next hops raises the question of
   how a network element can report a single meaningful latency value
   for a flow.  Eventually, this situation might best be handled by
   allowing characterization parameters to be provided by either
   upstream or downstream routers.  We do not yet know how to

Shenker                                                         [Page 7]
Design Notes                                           November 15, 1995

   incorporate this ability into the architecture, and so do not attempt
   to do so in the initial version of the specifications.

   The initial version of the characterization specification calls for
   conservative estimates of these delays.  In this context,
   conservative refers to the minimal delays experienced, as latency is
   intended to provide a lower bound on delays experienced by packets.
   All other delays (that is, delays larger than this minimal amount)
   should be described by characterizations reflecting variable or per-
   service delay at the network element.  For example, in predictive
   service the difference (if any) between the minimal and maximal
   speed-of-light delays crossing a particular subnet should be included
   in the characterizations of the delay bound s. Thus, the latency
   parameter represents a lower bound on packet delay and the latency
   plus the delay bound parameter bounds the maximal packet delay.

   This is, in some sense, no different than any case where the service
   delivered by a subnet technology is not easily characterized.  The
   best one can do is use a conservative characterization.  There are
   likely to be some subnet-specific methods to reduce the degree of
   uncertainty in these characterizations.  We do not address such
   methods here, but eventually they may be described in documents
   describing how to implement integrated services over various subnet
   technologies.

 Delay Guidelines for Predictive and Controlled Delay Service

   The invocation of these services involves the specification of a
   logical service level.  Different network elements can map that
   service level into very different levels of delay.  It is not clear
   to what extent this is a problem.  Extreme heterogeneity in the
   absolute delay control offered by different network elements for the
   same logical service level might lead to inefficient use of the
   network resources, in that it is of little use to require very tight
   control of delay at one element if another exercises very loose delay
   control.  However, if the service levels reflect a "level of effort"
   then perhaps the most efficient allocation of network resources is to
   spread this effort equally, regardless of the associated delays.

   If we determined that heterogeneous delays were a significant
   problem, then one approach is to offer an alternative form of service
   invocation involving target delays.  This, however, makes the
   characterization of service extremely difficult (it appears
   impossible to efficiently provide characterization parameters
   describing the delays resulting from an arbitrary target delay).
   Another approach is to provide guidelines for the deployment of these

Shenker                                                         [Page 8]
Design Notes                                           November 15, 1995

   services, so that extreme heterogeneity in service levels is largely
   avoided.  It is unclear whether such guidelines should be embedded in
   the service specifications, or should arise from notions of "accepted
   practice" as we gain experience with the services.

   At present, no guidelines are included in the service specifications.

 Packet Loss in Controlled Delay

   There was some question about how losses should be treated and to
   what extent loss was present in controlled delay service.  To
   partially address this, the service specifications make clear that
   discarded packets are not counted in the delay measure, and that the
   levels of loss can vary from implementation to implementation.
   However, we are left with the question of whether these loss rates
   are characterized in a characterization parameter.  The assumption is
   that such losses will rarely be noticeable, and so quantifying them
   would be of little use.  The same reasoning applies to the
   possibility of quantifying delay bound violations in predictive
   service.

 Validity Flag

   The following language was added to the template:

     There is a Validity Flag associated with each characterization
     parameter.  If this flag is set the characterization parameter
     should be considered invalid.  In cases where network elements are
     unable to provide the required characterization information, the
     network element should set the Validity Flag and, if composition is
     called for, should leave the characterization parameter unchanged
     (i.e., apply no composition rule).

 Delay Characterizations for Predictive and Controlled Delay

   There was some discussion last time about what delay quantities to
   export for controlled delay (and hence predictive) service.  There is
   a tension here.  On the one hand, we are not really sure what the
   most useful quantity would be, and in fact it is likely that
   different applications would want different delay characterizations.
   On the other hand, if we merely leave this open for vendors to
   experiment with different delay characterizations there is no way for
   the endpoints to know how to interpret the characterizations they
   receive.  Thus, it seems best to settle on a specified set of delay
   characterization even if there are many roughly equivalent choices.

Shenker                                                         [Page 9]
Design Notes                                           November 15, 1995

   The last version of the drafts measured the maximal delay over time
   intervals T, where T was 1 second, 60 seconds, and 3600 seconds.
   There was some concern expressed that we really don't want the
   maximal values, but the 99.9 percentile (or 99.99 percentile, or some
   other nonmaximal amount).  Unfortunately, we are not aware of any
   stable algorithms to compute such percentiles other than the brute
   force method of recording all of the data over the whole time
   interval; this seems like a substantial burden for little incremental
   return (99.9 percentile versus the maximal).

   Another possibility is to provide the averages of the maximal values.
   That is, rather than report only the value of the maximal delay over
   the last time period T, report the average value of this quantity
   over some number of previous time periods, each of length T.  More
   precisely, for a given length T, the network element divides the time
   into periods of length T and then records the quantities Qi, where Qi
   is the maximal delay seen over the i'th time period of length T.  The
   value the network element reports is some average over these
   quantities Qi.  Separate values are kept for each of the three time
   periods T of 1 second, 60 seconds, and 3600 seconds.  This method of
   reporting an average of these maximal values, rather than just the
   last maximal value, will provide a better indication of where
   adaptive algorithms will, on average, end up placing their playback
   point.  The different time periods will reflect the tendencies of
   adaptation algorithms with different degrees of conservativeness.
   The longer time periods will reflect more conservative (i.e., less
   willing to tolerate packets arriving late) adaptive algorithms.

   We need not specify the averaging algorithm used on these maximal
   values, since it seems to make little difference whether it is a
   rolling exponential average or a straight arithmetic one over some
   number of previous data points.  Language has been inserted into the
   text to reflect this change.  This should be discussed at the next WG
   meeting.

 Peak Rate and Slack Term for Guaranteed Service

   Guerin et al. distributed to the WG two proposals for modifying the
   Guaranteed service specification.  The first called for the inclusion
   of a peak rate term in the TSpec.  The second called for the
   inclusion of a slack delay term in the RSpec.  We have included the
   peak rate in the latest specification of guaranteed service, but have
   not included in the slack term. We hope to discuss these two
   proposals at the Dallas WG meeting and reach a decision about whether
   or not these changes should be adopted.

   Note that we are including the peak rate term in the guaranteed

Shenker                                                        [Page 10]
Design Notes                                           November 15, 1995

   TSpec, but not in the predictive and controlled delay TSpec.  This is
   because those services don't need, nor can they typically make use
   of, the peak rate.  Moreover, if a request for controlled delay or
   predictive is mapped into a guaranteed request (as in our
   heterogeneity proposal), then the peak rate can be set to the default
   value of infinity without any harm besides the fact that admission
   control will be slightly more conservative.

 "Killer Reservation" Problem

   As has been observed in several forums, merging of flows can present
   a problem that has come to be known as the "killer reservation
   problem" in the RSVP WG.  Consider the case where one receiver's
   reservation for CD service level 3 is successful end-to-end.  Another
   receiver attempts to make a reservation for CD service level 1.  At
   some point the reservations for these two receivers merge, and become
   a reservation for CD service level 1.  Assume that upstream of this
   merging point the reservation fails.  Has the original receiver, who
   previously had a successful reservation with service level 3, been
   denied service?  Several people have suggested that we should address
   this problem in the service specifications.  However, this problem
   has already been successfully addressed in the RSVP WG.  The solution
   requires remembering that there used to be a successful reservation
   of service level 3 at the failure point, and then forwarding this
   reservation on.  The solution does not require any modification to
   our service definitions, but it does require some support from the
   interfaces in the service module, in that a failed reservation
   request should leave the current reservation in place (rather than
   resulting in no reservation state).

 Modularity and Characterizations

   Characterizations involve two distinct steps.  One is providing the
   basic data, and these clearly must come from the network element.
   The other is applying the composition rule specific to that
   characterization parameter.  Because we want to allow setup protocols
   to be independent of the services offered, we have defined services
   in such a way that this composition is performed by the service
   module.  One can imagine other, more service-aware setup protocols
   that do this composition themselves, but it is important that
   characterizations do not require setup protocols to do the
   composition.

   Furthermore, the service definitions do not require any specific
   behavior on the part of a setup protocol.  They merely make available
   a set of service module interfaces (such as characterization

Shenker                                                        [Page 11]
Design Notes                                           November 15, 1995

   parameters, invocation, etc.).  In particular, characterizations need
   not be provided to the endpoints.  For instance, the service
   requirements do not require that RSVP provide "advertising".
   However, the usefulness of the service may depend on the degree to
   which these characterizations are made available to endpoints, and to
   network elements along the path.  Such dependencies should be clearly
   spelled out.

   The characterization parameters in controlled delay service may be
   used to select the service levels, but we expect many applications
   would be able to use this service without any characterizations.  The
   delay bounds provided by predictive service, however, are of little
   use if the characterizations of the delay bound are not made
   available to endpoints.  In such a setting, predictive service is
   essentially indistinguishable from controlled delay from an
   application's perspective.  In both controlled delay and predictive,
   the characterizations are used only by the endpoints, and not by the
   network elements along the way.  Any order of composition along the
   path yields the same results.

   Characterizations play a much more central role in guaranteed
   service.  First, the delay bounds, which are the crucial aspect of
   guaranteed service, cannot be determined by the endpoints without
   characterizations.  Moreover, the intermediate network elements along
   the path need characterizations to calculate the appropriate buffer
   capacities.  Thus, the composition along the path should proceed from
   sender to receiver.

 Modularity and Reservation Styles

   Reservation styles allow the reserving entity (e.g., a receiver in
   RSVP) to specify how the reserved service is shared among various
   flows (i.e., do the flows from various sources share the same
   reservation, or do all have independent reservations?).  RSVP offers
   several different reservation styles, such as wildcard, shared
   explicit, and fixed.  Other setup protocols offer other reservation
   styles (such as the in the Tenet architecture).  These reservation
   styles are not addressed in the service specifications.  The intended
   modularity is that the service specifications address the nature of
   the service being offered at a particular network element.  The setup
   protocol dictates whether the reservation is made for a shared flow,
   or for independent flows, based on the requested reservation style.

   For RSVP's shared reservation styles (wildcard and shared explicit),
   the TSpec used in the reservations is based not only on the
   receiver's request.  Instead, it is based on the minimum of (1) the
   requested TSpec, and (2) the sum of all relevant (i.e., in scope)

Shenker                                                        [Page 12]
Design Notes                                           November 15, 1995

   upstream TSpec's.  Note that the first quantity is specified by the
   receiver in the service request, while the second quantity comes from
   the specification of the source's TSpec carried in the PATH message.
   From an architectural perspective, this is somewhat confusing in that
   the source's TSpec is specified without reference to a particular
   service, while the receiver's TSpec is in the context of a particular
   service.  At present, since all the services discussed so far use
   basically the same TSpec, this is not a problem (note that guaranteed
   service includes the peak rate in the TSpec while the controlled
   delay and predictive don't, but this is not a serious issue).

   To compute this reservation request, either RSVP or the service
   module must compute two quantities: sum of n Tspecs and the minimum
   of two Tspecs.  The sum of n TSpecs is defined to be a TSpec that
   would judge as conformant any possible superposition of flows that
   each conform to their respective TSpec; one wants a least upper bound
   to such "sum" TSpecs.  The minimum of two TSpecs is a TSpec that will
   only judge as noncompliant a flow that failed at least one of the two
   TSpecs.  Note that because the current form of TSpecs is
   multidimensional, one cannot define a perfect "min" TSpec that will
   judge a flow as noncompliant if and only if it fails at least one of
   the constituent TSpecs.  That is, a flow may not conform to one of
   the 2 Tspecs but it will not be judged noncompliant by the min Tspec.
   Thus, we seek only a smallest member of the set of Tspecs that never
   judge a flow noncompliant if is is compliant with both TSpecs.  The
   term "smallest" relies on the ordering relationships spelled out in
   the service documents (and a smallest member is one where there is no
   other smaller member in the set).  For the current TSpecs, we propose
   the following:

   Sum:  minimum of the minimum policed units m, the maximum of the
   maximum packet sizes M, the sum of the peak rates p, the sum of the
   token bucket rates r, and the sum of the token bucket sizes b.

   Minimum: minimum of the minimum policed units m, the maximum of the
   maximum packet sizes M, the minimum of the peak rates p, the minimum
   of the token bucket rates r, and the maximum of the token bucket
   sizes b.

   The choice of minimum is rather arbitrary.  We should discuss whether
   other choices are better (e.g., one could pick the maximum of the
   rates r and the minimum of the buckets b instead).  However, unless
   we are policing everywhere, these choices need to be uniform (i.e.,
   all elements make the same choices when there is ambiguity in the
   min).

   Since these definitions are not completely specific to a service
   (since the source specified TSpec aren't), there is some confusion

Shenker                                                        [Page 13]
Design Notes                                           November 15, 1995

   about where this discussion (about sums and minimums of TSpecs)
   should go in our documentation.  Your suggestions are solicited.

 IP Hop Count

   An integrated services (IS) aware network element is one that
   conforms to the various requirements described in the service
   specification documents (it need not offer those services, but if it
   does it supports and characterizes the services in conformance with
   the relevant specification).  At the request of the RSVP WG, one of
   the general characterization parameters included in the current
   drafts is the number of IP hops between two IS network elements.
   This is so the endpoints can know if there are hops along the path
   that aren't responding to the service request.  This characterization
   requires the service module to know the number of IP hops between the
   last (i.e., upstream) IS network element and this one.  Observe that
   since the network element may not be aware of its neighbor on the the
   upstream path, *the local parameter must be provided to the network
   element by the setup protocol* (possibly by tracking the TTL value).

   There are two questions to address.  First, do we need the actual
   number of hops, or just a bit that indicates that a non-IS router was
   traversed?  Second, should this issues be handled by a
   characterization parameter, or by the setup protocol directly?  One
   can argue that the set up protocol should manage this count, since
   the knowledge of the number of IP hops between IS network elements
   must come from the setup protocol anyway (in contrast to the other
   characterization parameters which are service module related).
   Another point is that ultimately, what we are aiming for is to know
   at how many hops an application will be able to obtain an enhanced
   quality of service, and at how many it won't (or, more simply,
   whether or not it will get the enhanced QoS at all hops).  The
   application's ability to get the enhanced quality of service at a
   router depends both on whether the router offers the service, as well
   as on the application's ability to transmit a request to that router
   (e.g., with a setup protocol).  What if both RSVP and another setup
   protocol are used on a router, but one of them has crashed.  Then the
   value of the characterization parameter exported by the service
   module depends on who is asking.

   We should discuss this question at Dallas.

   [note added in proof:  this parameter was added at the request of the
   RSVP WG, but apparently the thinking there has shifted recently.  So
   it is very likely that this parameter will deleted entirely.]

Shenker                                                        [Page 14]
Design Notes                                           November 15, 1995

 Open Issues

   There are, of course, many open issues.  However, the ones that seem
   in most need of immediate closure are:

    (1) The general issue of policing, and the specific issue of how to
    implement marking in IPv4 and IPv6.

    (2) Delay characterization in CD and P service.  There are several
    roughly equivalent choices here.  We should make a decision and move
    on.

    (3)  Do we need delay guidelines for the different levels of CD and
    P service?  [current drafts do not include such guidelines]

    (4) How to account for the differing delays between different
    endpoints on shared media.  To what extent is having the
    characterization parameters filled in on the other end a matter that
    needs addressing in the service specifications.

 Issues not Covered

   The current service specifications do not describe the associated
   MIBs.  They also do not describe the content of error messages for
   service request denials.  More generally, there is no description of
   the abstract interfaces that a service module must support.  The
   current RSVP spec contains an operational definition of these, but we
   should make progress in providing a more general and abstract
   discussion of the interfaces.

 Lossy Services

   The current set of proposed services (guaranteed, predictive, and
   controlled delay) all provide low or no loss service.  There is a
   widespread belief that there are applications that, while they
   require some form of delay control, can tolerate non-negligible
   levels of loss (at least for some substreams in the flow).  Their
   needs are not efficiently met with the current set of services.
   Thus, another important class of services to explore are those that
   include significant levels of loss.  Note that lossiness isn't an end
   in itself; allowing losses is valuable because it enables the network
   to operate at higher levels of utilization for the same level of

Shenker                                                        [Page 15]
Design Notes                                           November 15, 1995

   delay control.  One form of "variable loss" service would offer some
   delay control (e.g.  equivalent to controlled delay or predictive)
   and several levels of lossiness.

   While the immediate focus of the WG at Dallas, and for a period
   thereafter, will likely be on the finalization of the current
   proposed services, we expect at some later point to shift at least
   some of our attention to discussion of these lossy services.

Shenker                                                        [Page 16]