Subject: comments on the service template

From: [email-masked] (Sally Floyd)
Date: Tue, 28 Mar 95 11:16:46 PST

I have been reading the internet drafts for the various service
templates, and in general it looks quite nice.  I have some
comments about the "Evaluation Criteria" section.

The tests in the Evaluation Criteria section seem to be stress-tests
designed to verify that a router has competently implemented a
particular service, for a router that could be considered as a black box
sitting on a bench in the laboratory.

I have been talking with the organizers of the IP Provider Metrics
BOF, which has as its goal "to provide the Internet community with
tools to help customers select IP providers, and thereby encourage the
providers to `do the right thing'".  From this perspective, it might
also be helpful to include in the Evaluation Criteria a discussion of
the performance metrics that would be of interest to users.  These
would be metrics concerning the service in fact delivered to an
Internet user.

I have in mind consumer-reports-type metrics such as the following:
Average delay and packet drop rates (e.g., for the last month ...).
Call rejection rates.  
Worst-case reports (e.g., most connections using this service
  might have received acceptable performance, but on this day this
  connection received performance so bad that the service was
  unuseable...).

This is NOT to suggest that the service templates should outline
procedures by which these metrics would be obtained, or should provide
the tools to obtain these metrics.  But the service templates could
discuss the relevant metrics that customers are likely to be interested
in.  (Partly as a reminder that actual users will care about call
rejection rates as well as about the delay and packet drop behavior.)

- Sally