Subject: [ippm] Surveyor implementation report for RFC 2679-2680

From: Matthew J Zekauskas <[email-masked]>
Date: Sun, 01 Aug 2004 01:01:20 -0400

I don't believe we attempted to implement RFC 2678, and certainly
not RFC 2681 for Surveyor.

I'm also trying to get an implementation report on the metrics
for the Internet2 OWAMP implementation (I believe we also implement
periodic streams, although we haven't used them in practice to
date).

--Matt

- - - - -

General
=======

Name of Implementation:      Surveyor

Organization:                Advanced Network & Services, Inc.

Contact details:             (original)
                             Advanced Network & Services, Inc.
                             200 Business Park DR, Armonk, NY 10504, USA

                             Advanced is essentially defunct as of this
                             writing (30-Jul-2004); for details on the
                             implementation see the INET'99 paper
                               Sunil Kalidindi and Matt Zekauskas. "Surveyor:
                                 An Infrastructure for Internet Performance
                                 Measurements."  INET ’99, June 1999.
                             or contact Matt Zekauskas, [email-masked].

                             The Surveyor code and data from 1997-2002
                             was donated to the
                             Wisconsin Advanced Internet Lab; contact is
                             Paul Barford, [email-masked]

Origin of code:              Written from scratch.

Platform:                    BSDI/OS 3.2+
                             TrueTime GPS-PC card, with custom device driver

                             This platform is IPv4-only; while the code
                             should be IP-address independent, this was
                             never verified, and would likely require some
                             tweaking.

Metrics
=======

RFC2679/Type-P-One-way-Delay
----------------------------

Parameters:                                 Implemented?
 + Src, the IP address of a host            Yes
 + Dst, the IP address of a host            Yes
 + T, a time                                Yes

Units:
 + dT                                       Yes

Methodology:
 + According to section 3.6 of RFC 2679
 + Synchronization via GPS time cards
 + Timestamping in software (and not in the network driver, although
    we did have an experimental version that did this)

Comments (Type-P):
 + UDP packets are being used.
 + The port numbers for test traffic varied (and could not be specified)
    among sessions, although for any given session port numbers were fixed
 + User data length was 12 bytes (32 bit sequence number, 64 bit time value)
 + In general, no special IP bits are set, but there was a version of the
    code which allowed the user to set the TOS/DSCP values.
 + If a packet is duplicated, any duplicates are ignored (the delay of
    the first packet to arrives is used).
 + Corrupted packets are generally counted as lost; they will
    fail either a hardware CRC check, or IP or UDP checksum verification.

Features included:
 + If either end lost synchronization with GPS (as reported by the
    timing card) the session was broken and measurement ceased.

Features not implemented:

Reporting the metric:
 + Because measurements ceased when GPS synchronization was lost, and
    the errors (typically 50 to 100 microseconds with our densest
    measurement mesh) were much smaller than instrumented paths,
    negative values were not reported, but flagged as a possible software
    error.
 + The loss threshold was 400 received packets in our implementation.
   For typical paths where two packets per second (on average) were sent,
   the loss threshold was about 200 seconds, or 3 1/3 minutes.
 + In our implementation the distinguished value 10,000,000 was used
    to signify infinite delay (a loss).(10 seconds -- although in
    practice I don't believe we saw anything greater than ~3 sec.)
 + All systematic errors are removed from the timestamps before reporting
   the metrics
 + Paths are being determined by running the well-known traceroute tool
   approximately 6 times an hour (on a Poisson schedule with average
   10 seconds, but also clamped to 10 seconds maximum).  Paths were stored
   independent of delays, but displayed along with delays in our graphical
   displays.

Tests performed:
 + Back-to-back tests of typical machines in the field was performed
   for calibration.  For our worst-case fan-out (due to the software
   implementation, error became worse the more paths that were measured),
   80 paths, the calibration error was 100 microseconds.
 + Cross check of measurements against the RIPE-TT implementation
   (reported at the 44th IETF IPPM WG meeting by Henk Uijterwaal).

RFC2679/Type-P-One-way-Delay-Poisson-Stream
-------------------------------------------

Parameters:
 +  Src, the IP address of a host                   Yes
 +  Dst, the IP address of a host                   Yes
 +  T0, a time                                      Yes
 +  Tf, a time                                      Yes
 +  lambda, a rate in reciprocal seconds            Yes

Note: Lambda is reported as packets per second.

Units:
 +  T, a time, and                                                 Yes
 +  dT, either a real number or an undefined number of seconds.    Yes

Report consists of series of measurements for Type-P-One-Way-Delay.
All the Type-P-One-Way-Delay features and comments above apply here.

Features:
 + The receiver knew the send schedule (passed as a seed to the
    random number generated used for the Poisson schedule) so it
    could independently determine if packets were lost.
 + As with the singleton, he loss threshold was 400 received packets
   in our implementation.  For typical paths where two packets per
   second (on average) were sent, the loss threshold was about 200 seconds,
   or 3 1/3 minutes.
 + This 400 packet window was also used to correct for reordering.
 + All delay values were kept to allow computation of arbitrary percentiles
    on demand.

RFC2679/Type-P-One-way-Delay-Percentile
---------------------------------------
By default, the 50th percentile (median) and 90th percentile are calculated
and reported for one minute intervals.  A separate Java-based UI could ask
for arbitrary percentiles over arbitrary intervals (provided the raw data
was on-line) -- typically 6 months of raw data were on-line.

RFC2679/Type-P-One-way-Delay-Median
-----------------------------------
This value is calculated and reported for one minute intervals, although
we did have a separate Java-based UI that could vary the reporting period

RFC2679/Type-P-One-way-Delay-Minimum
------------------------------------
This value is calculated and reported for one minute intervals, although
we did have a separate Java-based UI that could vary the reporting period.

RFC2679/Type-P-One-way-Delay-Inverse-Percentile
-----------------------------------------------
Not implemented.

Our implementation did not calculate any inverse percentiles.  One could
ask for the raw data and calculate the inverse percentile yourself.

Our implementation did, however, provide a histogram of delay values,
by default ranging from 0 to 300mS in 10mS buckets.

RFC2680/Type-P-One-way-Packet-Loss
----------------------------------

Metric Parameters:                                  Implemented
 +  Src, the IP address of a host                    Yes
 +  Dst, the IP address of a host                    Yes
 +  T, a time                                        Yes

Metric Units:
 + Type-P-One-way-Packet-Loss = [0,1]                Yes

Comments:
 + The time is the time the packet was sent, rounded to the nearest microsecond.
 + Packets that arrive at the machine corrupted are generally dropped either
    by the hardware CRC failure, or by IP or UDP checksum failure.  Hence,
    corrupted packets are counted as lost.
 + If a packet is duplicated, any duplicates are ignored.

Further comments:
This metric has been implemented in combination with RFC2679/Type-P-One-way-Delay.
All comments made there apply.   If a packet arrives, it's
Type-P-One-way-Packet-Loss is 0.  If a packet does not arrive in the
specified window (200 seconds by default) the delay is infinite (undefined),
and the Type-P-One-way-Packet-Loss is 1.

RFC2680/Type-P-One-way-Packet-Loss-Poisson-Stream
-------------------------------------------------
This has been implemented as part of RFC2679/Type-P-OWD-Poisson-Stream.
Results consists of a series of measurements for the One Way Delay, with
values of 10,000,000 for packets that were sent but did not arrive.

RFC2680/Type-P-One-way-Packet-Loss-Average
------------------------------------------
By default, the Surveyor reporting mechanisms calculate this on
one-minute intervals.  A Java UI allows calculations on different
intervals.

_______________________________________________
ippm mailing list
[email-masked] 
https://www1.ietf.org/mailman/listinfo/ippm