Subject: RE: [ippm] metric requirements

From: "Alan Clark" <[email-masked]>
Date: Thu, 3 Oct 2002 21:34:50 -0400

This may be getting into too much detail (however is my favorite subject!!!)

We recently compared the burst parameters of multiple large traces using
both the consecutive loss and a Gilbert model.  In addition to analyzing
pure packet loss we also looked at the distribution of discards due to
jitter.  The result showed clearly that bursts of both loss and loss/discard
were typically 20-200 packets in length with loss densities of 20-30%,
whereas the consecutive loss model would have suggested bursts averaging 1.5
packets!!!  This is an easy experiment to duplicate and can be performed
with Sue Moon or Wenyu Jiang's trace files. For those able to make
measurements on core IP networks then it is quite possible that loss would
primarily occur to to route changes or link failures and hence that loss may
often be consecutive, this should not however be used to infer that loss is
always consecutive.

I agree with Chuck that a multi-pronged approach is essential however I
would suggest that a Gilbert/ Gilbert-Elliott or 3/4-state Markov model
based approach would be preferable to a consecutive loss model.  We use a
4-state Markov model which combines the ability of Gilbert-Elliott to
capture long sparse bursts with the ability of a 2-state model to capture
consecutive loss, this is computationally efficient and produces all the
statistics needed.

The E Model modifications that were made to support bursty loss
unfortunately focus only on consecutive loss - this is a definite
improvement however still does not reflect conditions that appear quite
common on some networks (including both Enterprise IP networks and the
Internet).  If the E model is used for planning (it's original purpose) then
you don't know what the distribution of packet loss is - hence the current E
model provides a good approach.  In a monitoring (or simulated) enviroment
then it may be possible to see what actually happened in which case a more
sophisticated approach may be justified.

Regards

Alan Clark
Telchemy

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf
Of Dvorak, Charles A (Chuck), ALASO
Sent: Thursday, October 03, 2002 6:17 PM
To: [email-masked]; stanislav shalunov;
[email-masked]
Cc: [email-masked]; Ash, Gerald R (Jerry), ALASO; Tarapore, Percy S,
ALASO; [email-masked]; [email-masked]
Subject: RE: [ippm] metric requirements

Further comments on packet loss:

Recommendation Y.1541 of the ITU-T defines 6 IP layer QoS classes, NI-to-NI.
All classes except best effort have a 0.1% packet loss objective, measured
over an interval of a minute. One reason for this objective is an
acknowledgment that in real networks, packets occur in bursts. So the
"several percent packet loss" tolerance that many people have stated is
really only for the unrealistic, uniformly distributed random case. When
this 0.1% objective is met, speech quality will almost always be
acceptable--as long as they don't all occur in just one or two bursts.

Just FYI, AT&T Labs proposed a new approach (not adopted yet) that took a
three-pronged approach to setting objectives for loss packets: One objective
keeps the quality as good as a PSTN call; the second objective allows a few
bursts spread out causing some, but not serious, degradation; the third
objective limits the maximum sequential loss, so as to not have any bursts
that in and of themselves cause an unacceptably annoying impairment during
the minute interval.

Finally, the E-model of G.107 was recently modified to accommodate (albeit
crudely) the effects on speech quality of bursty packet loss, which should
dispel the notion that the E-model is not for VoIP use. It is!

Anyway a key message here is that (supporting Alan's message) during high
burtsy losses, one goes from very good speech quality to unusable moments.
Unlike random losses, bursts make you fall off a cliff.

Chuck Dvorak
AT&T Labs

-----Original Message-----
From: Alan Clark [mailto:[email-masked]]
Sent: Monday, September 30, 2002 3:19 PM
To: 'stanislav shalunov'; [email-masked]
Cc: [email-masked]
Subject: RE: [ippm] metric requirements

Fortunately many codecs behave in a consistent way and don't generate triple
the amount of traffic in order to improve resilience!!

Work done within ITU SG12, ETSI TIPHON and a few other groups has
characterized the behavior of a wide range of codecs under packet loss
conditions up to 20-30%.  It can however be a little tricky deducing if
packet loss concealment is in use from the info available in RTP headers!

One key point when considering the effects of packet loss on VoIP is the
distribution of packet loss.  This does not behave, as many would suggest,
as a simple burst channel exhibiting only consecutive loss.  Bursts of loss
are often sparse (e.g. 20%) and extend for periods of seconds - largely
caused by periods of congestion.  You may also see the extreme case of
bursts of consecutive loss extending for hundreds of packets, due to link
failures.
The effect of this is that call quality will be fine for some period of
time, and then experience several seconds of badly degraded quality before
improving again.

Alan Clark

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf
Of stanislav shalunov
Sent: Monday, September 30, 2002 12:10 PM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [ippm] metric requirements

"H.E.Holland" <[email-masked]> writes:

> Where might I find quantitative metric requirements for services
> such as VoIP? E.g. what are recommended values for one-way delay and
> packet loss for a VoIP service?

While others pointed out the ITU documents for one-way delay, I don't
think we answered the `loss' part of the question.

There can possibly be no answer, because codecs have various degrees
of tolerance and redundancy.  Relevant gedanken Experiment: Imagine
that your favorite codec were to send all its packets in triplicate;
that would change the threshold of tolerance for loss, wouldn't it?

--
Stanislav Shalunov		http://www.internet2.edu/~shalunov/

"I didn't attend the funeral, but I sent a nice letter saying that I
approved of it."				-- Mark Twain
_______________________________________________
ippm mailing list
[email-masked]
http://mailhost.advanced.org/mailman/listinfo/ippm

_______________________________________________
ippm mailing list
[email-masked]
http://mailhost.advanced.org/mailman/listinfo/ippm

_______________________________________________
ippm mailing list
[email-masked]
http://mailhost.advanced.org/mailman/listinfo/ippm

_______________________________________________
ippm mailing list
[email-masked]
http://mailhost.advanced.org/mailman/listinfo/ippm

---

From: "Dvorak, Charles A (Chuck), ALASO" <[email-masked]>
Date: Thu, 3 Oct 2002 21:36:48 -0400

Just for clarity, we are not advocating a focus on consecutive loss!

But one does want to know, as a function of codec type, how much consecutive loss you can have before just one such occurrence on a VoIP call makes it unacceptable.

Chuck Dvorak
AT&T Labs

-----Original Message-----
From: Telchemy Standards Tracking [mailto:[email-masked]]
Sent: Thursday, October 03, 2002 9:27 PM
To: Dvorak, Charles A (Chuck), ALASO; stanislav shalunov;

Regards

Alan Clark
Telchemy

-----Original Message-----

Chuck Dvorak
AT&T Labs

-----Original Message-----
From: Alan Clark [mailto:[email-masked]]
Sent: Monday, September 30, 2002 3:19 PM
To: 'stanislav shalunov'; [email-masked]
Cc: [email-masked]
Subject: RE: [ippm] metric requirements

Alan Clark