Subject: BBN, work on Resource Coordination Objects (overview)

From: Julio Escobar <[email-masked]>
Date: Tue, 02 Nov 93 23:53:33 -0500

As I mentioned in todays MMUSIC meeting,
this is the overview document that BBN sent to Dartnet in July 93
describing our work on Resource Coordination Objects under the Real
Time Multicast Communications project at BBN.  We are working on
releasing the draft document "Resource Coordination Objects: a state
distribution mechanism," which we will post on this email list.  

Unfortunately this overview is written in Latex, but should still be
readable.   

JULIO

------- Forwarded Message

- ----------------------------------------------------------------------------
% project overview paper for Real-Time Multicast

\documentstyle[11pt]{article}
\newcommand{\rem}[1]{$<<${\em #1}$>>$}

\title{Real-Time Multicast Communications \\
Project Overview}
\author{Walter Milliken \\
Advanced Networking Department \\
BBN Systems and Technologies}
%\date{}

\begin{document}

\maketitle

\section{Introduction}

The Real-Time Multicast Communications project at BBN has been
concerned with defining and implementing new communications services
for use in the rapidly-growing area of distributed real-time
applications.

There are four services being developed under this project:

\begin{description}
\item[Anycasting] provides a logical addressing mechanism for
obtaining services from one of a set of equivalent hosts.

\item[Multi-level flows] allow a single multicast group to be used to
transport several sub-flows, and allow hosts and routers to optimally
propagate these sub-flows.

\item[Shared streams] allow several flows to share network resources
under a single resource reservation.

\item[Resource Coordination Objects] provide a general mechanism for
distributing, storing, and accessing network-related information among
entities involved in distributed communications.

\end{description}

These concepts are discussed in more detail below.  For each, there is
a description of the service, the technical approach, and the
technology transfer issues.

Our general approach is to develop near-term deployable IP versions of
these concepts, so that the utility of the ideas can be assessed.  We
are also looking at the longer-term implications of the general concepts,
to guide future work in these areas.

\section{Anycasting}

\subsection{Description}
This capability is intended to serve as an alternative to directory
services and multicasting for locating and using distributed servers
(a group of equivalent hosts providing an identical service at a
logical address).

The basic anycasting service has been implemented using ordinary IP
protocols, and is operational.  We are currently investigating
possible demonstration applications for this service, e.g., video
server, NTP, etc.

\subsection{Approach}
The service is implemented by advertising a route from each
anycast host to the anycast address, using standard routing
protocols.  Normal IP routing then resolves multiple routes to the
anycast address just as it handles multiple network routes.  The
anycast client application in each host can optionally supply an
initial metric to be used in the host's routing advertisement, to
influence route choice based on some application-specific
requirement.

Our current implementation uses the standard "gated" routing demon
to advertise the route.  Recognition of the anycast address by the
host for input is done by creating a new socket address family that
is a duplicate of the IP family (AF\_INET).  This family (the
virtual Internet family) allows creation of virtual network
interfaces bound to anycast addresses.  The server application then
opens sockets of this family, bound to the anycast address.
Since each virtual network interface socket looks like a standard
network interface to the kernel, gated automatically advertises
them.

The version of gated used so far uses EGP, which requires the
assignment of a class C IP address to each anycast address.  We may
also try using OSPF instead of EGP would allow use of ordinary host
addresses.

\subsection{Tech Transfer}
Anycasting is compatible with existing routers, and
requires only minimal changes to host networking software (to allow
the anycast address to be recognized by the host).  Our
implementation can be easily added to existing BSD-based systems,
by compiling the virtual network interface module and linking it to
the standard kernel, and similar implementations should be simple
to add to other platforms.

Anycasting is most efficiently supported by routers using mask-based
IGPs, such as OSPF.  Efficient inter-domain anycasting may require
new inter-domain routing protocols, which would be a topic for
further study.

\section{Multi-level Flows}

\subsection{Description}
This service allows multicast flows to be labeled with
sub-flow identifiers, and provides the capability for receivers to
request only some of the sub-flows.  This allows the network to
deliver only requested data down the various branches of the
multicast routing tree.

\subsection{Approach}
We are planning to augment the IP multicast routing
implementation, mrouted, to support multi-level flows.  This will
include upward-compatible extensions to IGMP to identify which
sub-flows the receiver desires.

We are looking at using the IP precedence field for labeling
sub-flows, as this appears to match well with most anticipated
application uses for sub-flows, which tend to involve data streams
with a clear priority ordering.

This approach should be more scalable than the alternative of using
multiple IP groups per application (one per sub-flow), which can easily
multiply the number of IP multicast group routes in routers by a factor
of 2-4.  (One technique we've studied for handling large-scale
distributed simulation applications might require several thousand
multicast groups, each with several sub-flows.)  Our approach will
only add on the order of 10% to the size of the routing entry for
each multi-flow group address.  It will also perform better in the
face of network congestion, at least when there is a clear priority
ordering among sub-flows.

\subsection{Tech Transfer}
The code modifications to support multi-level flows in
router and host implementations that already support IP
multicasting appear to be minimal, so that this service should be
readily transferable to existing multicast-capable routers and hosts.

Use of the IP precedence field will generally be consistent with
router behavior -- those routers that examine the field will deliver
the more important sub-flows in preference to less-important ones.
Only if an application arose where sub-flows had no clear precedence
would normal treatment of IP precendence cause gratuitous behavior,
and no applications we've considered should suffer this problem.

The ability to set IP precedence on application packets isn't
supported by the standard BSD UDP socket implementation.  We would
provide a send-side UDP library routine to send precedence-labelled
UDP datagrams using the BSD raw socket facility.  This would be more
portable than modifying the BSD socket implementation in the kernel
to support an IP precedence socket option,

This design allows interoperation with unmodified IP multicast hosts
and routers, though optimal delivery of sub-flows may not occur when
unmodified routers are involved.

\section{Shared Streams}

\subsection{Description}
This service can be viewed as an extension of resource reservation
mechanisms such as Fair Share and RSVP, permitting multiple flows to
share a single resource reservation.  This allows distributed
applications to make use of bandwidth multiplexing.  For example, a
conference using floor control might specify that all conference flows
could share a single sender's worth of bandwidth, since only one can
send at a time.

\subsection{Approach}
We are implementing shared streams based on the Fair Share code, which
we already have source for.  Fair Share was originally designed to
allow fair bandwidth sharing of links to support policy routing
(IDPR).  It does this by identifying and separating different flows,
and then scheduling output according to the policy specifying the
bandwidth allocation for each group of flows.  Fair Share's
flow-separation and scheduling implementation is general enough to
support shared streams, so we will implement shared streams in IP by
replacing Fair Share's administrative policy interface with a
user-controlled mechanism for associating groups of flows with
reservation information.

We will use an early version of {\em Resource Coordination Objects}
(see below) to describe the flows, and to provide a user interface for
making reservations.  We plan to use {\em flowspecs} (RFC 1363) to
pass generic reservation information, which will then be adapted to
the requirements of the resource enforcement mechanism.

\subsection{Tech Transfer}
The shared stream service assumes a pre-existing flow-based resource
enforcement mechanism.  Since most of the shared stream implementation
will be outside the kernel (interfaced through a system call to the
reservation enforcement mechanism), it should be readily portable to
any system supporting such a service.  However, all such systems are
currently experimental.  When such systems become more widely
available, this work should be directly applicable to them.

\section{Resource Coordination Objects}

\subsection{Description}
RCOs are a general technique for distributed applications
to communicate information about aggregates of network-related
resources among hosts and network nodes.  They can be used to support
both session-layer services and network-layer services, and can be
used to coordinate auxiliary information about flows with network
routing.  RCOs appear to be a rich general mechanism for
incrementally adding services to networks.

RCOs are typed blocks of state information labelled from a global
namespace.  Hosts can create RCOs and pass them to the network, or,
given an RCO label, obtain them from the network.  RCOs can be
associated with a flow or set of flows in the network, in which
case they are propagated along the route(s) of the flow(s), for
possible use by routers.  Other RCOs may be stored in servers in
the network, and obtained only on demand of a host.  (These might
be used to store session information, for example.)  A third
possible type of RCO would be one which hosts (or network nodes)
would register interest in, and any changes in the RCO would be
automatically propagated to the set of interested parties.

There are a wide variety of potential uses for RCOs, including
distributed session management, network resource reservation,
multicast access control, and intelligent resource preemption.  Under
the current funding, we plan to work on the infrastructure and a few
example services using RCOs, including shared streams (above).

\subsection{Approach}
The details of the technical approach to RCOs are still
evolving.  The current top-level design calls for a user interface
to RCOs in hosts, probably UDP based, links to routing update
processing in routers to support propagation of flow-related RCOs
along data flow paths, and a distributed server process to store
RCO information not directly related to flows.

Each RCO has a type field (whose values would probably be
registered with a central authority), a propagation type
(flow-related, server-stored, or sent to an update list), a
flow list association (for flow-related types only), and a block of
opaque data, whose structure is determined by the type field.

Thus the basic RCO service is a means of establishing blocks of
typed state data at particular points in a distributed system,
either along a route, in a central (or distributed) repository
visible to all comers, or with a list of interested parties.

Since RCOs are transported opaquely, not all network components
need support any given type of RCO.  Only those components using
a particular type of RCO need to be modified to provide support for
that RCO.

\subsection{Tech Transfer}
RCOs should be readily transferable between platforms
- --- they will be transported using standard UDP/IP, and the support
libraries and servers will be built on vanilla BSD capabilities.
These could easily be adapted to different network APIs -- all that
is required is basic UDP support.

Flow-related RCOs require minor router modifications to supply
routing update "hooks" to the flow-related RCO propagation code.
RCO propagation will require a UDP send/receive capability in the
router, but this is typically available in order to support SNMP.

Once the basic infrastructure support for RCOs becomes commonly
available in routers and hosts, it becomes easier to add new
network services, using RCOs as a general extension mechanism.
One simply defines a new RCO type, decides how it is propagated,
and implements support for that RCO type in systems that wish to
support the new feature.  Other systems will propagate the RCO
properly, but will otherwise ignore it.

\end{document}

- ----- End of forwarded messages

------- End of Forwarded Message