Subject: [NEXTEL-1] Confidence in Waterfall Curves guides noise analysis in wireless system test

From: [email-masked]
Date: Thu, 21 Aug 2003 01:12:22 -0400

Confidence in Waterfall Curves guides noise analysis in wireless system test
By Scott Siclari, Senior Systems Engineer, Aeroflex Inc. (Plainview, N.Y.), EE
Times
July 23, 2003 (1:01 p.m. EST)
URL: http://www.eetimes.com/story/OEG20030710S0042

Factoring confidence and accuracy into the measurements for wireless systems can
be somewhat complicated. By spending some time before running the test and doing
some analysis afterward, the test effort will be minimized and the test results
should be more reliable and quantitative.

First, before making any measurements, determine any requirements for confidence
and accuracy. Determine the testing limit for bit error rate (BER): this is the
point at which the BER need not be measured any lower for any Eb/No (the ratio of
energy per bit to noise density) value. Apply these to the Eb/No test points to
determine the test time estimates.

Next, make adjustments to the confidence, accuracy and BER limit as needed to
provide acceptable results and to fit into the testing schedule. Use these
parameters to calculate the criteria to stop the measurements. This will be the
number of bit errors and the error-free bits required for each Eb/No test point.

Once the testing is complete, the confidence and accuracy have to be applied to
the measurements to allow reliable analysis of the waterfall curves.

Now, let's take a look at the details.

Evaluating almost any communications systems will require some form of BER
testing, whether it is done for engineering qualification or for production
testing. With wireless systems, the testing is not focused on how long between bit
errors, but at what point the background noise will cause too many bit errors. For
that reason, wireless systems typically require an evaluation of BER in response
to varying levels of simulated background noise, where the noise is expressed in
terms of Eb/No , or bit signal-to-noise ratio.

Simulated background noise is produced by a noise source generating additive white
Gaussian noise (AWGN). The AWGN will cause bit errors on the data stream that can
mathematically be represented as a Gaussian probability distribution. The
relationship between the AWGN and the bit error distribution allows for a much
broader analysis of the BER results. Because of the known properties of the error
probability, comparisons of test results among similar devices can be made
reliably.

Applying AWGN to wireless devices will lead to test results that are normally
represented as BER vs. Eb/No curves, commonly referred to as waterfall curves.
Waterfall curves express the BER response to decreasing background noise. A
waterfall curve plot is typically the end result of the testing and is a
significant factor when wireless devices are being compared.

The goal here is to be able to effectively generate the test results so that there
can be reliable comparisons using the resultant waterfall curve. By understanding
the probability distribution, testing can be performed effectively. Tradeoffs can
be made between the amount of test time that is acceptable and the amount that is
required to attain the desired measurement confidence level and accuracy. When the
testing is complete, the confidence level and accuracy of the measurements can
then be applied to the waterfall curve so that these can be used for reliable,
quantitative comparisons.

Gaussian probability distribution

Statistically, any measurement would have to be taken over infinite time to know
its true BER. Since the measured BER is taken over a shorter, finite, period, the
true BER could be significantly higher or lower. By adding stress with known
properties, testing can be completed in less time and still be reproducible.

For wireless systems, this stress is typically in the form of AWGN. The occurrence
of bit errors due to the AWGN can be represented through a Gaussian probability
distribution. Using the properties of this distribution, the individual
measurements can easily be characterized with confidence levels and accuracy.

Confidence level specifies how often the accuracy, when applied to the measured
BER, will correctly contain the true BER. Accuracy is the difference between the
measured BER and the true BER. Both are typically expressed as a percentage.

Let's take a simple example. Assume that a measurement was performed where 100 bit
errors were recorded. If the confidence level were one standard deviation (or
68.27 percent), the accuracy of the measurement would be ±10 percent. This means
that it is 68.27 percent likely that the true BER would be within 10 percent of
the measured BER. If this assumed test were done 10,000 times, the true BER would
be within 10 percent of the measured BER 6,827 times; the remaining 3,173 times it
would be off by at least 10 percent.

Waterfall curves express the bit-error-rate response to decreasing background
noise and are a significant factor when comparing wireless devices.

Knowing the Gaussian probability distribution provides a several formulas with
which to apply confidence level and accuracy to the BER measurements. The basic
one relates confidence level, accuracy and the number of bit errors. The
calculation can be expressed to solve for any of those three factors using the
remaining two factors. Most often it will be used to solve for accuracy, but it
will also be useful to solve for the number of bit errors required to meet a
specified accuracy and confidence level. Notice that these calculations are
independent of test time and bit rate. The accuracy is based purely on the
measured bit errors.

Waterfall curves

Typically, the test results will end up as waterfall curves, displaying the BER
response to decreasing background noise. There are also waterfall curves that are
the theoretical limits of the lowest attainable BER values for a given modulation
scheme. These theoretical waterfall curves will provide a starting point for
estimating test times.

The theoretical values are the theoretically lowest attainable BER. The BER that
is actually measured is going to be at or above this level. Using real signals,
the measured BER may be close to theoretical or significantly higher. After some
initial testing, this can be measured and, if necessary, a new curve can be made
representing the expected BER.

For each point on the curve that is to be tested, by using the theoretical or
expected BER values as the actual incoming BER an accurate time estimate could be
made for a measurement of that signal. The estimates can then be examined to
determine where most of the test time would be spent. Mostly likely the confidence
will be set to a fixed value, but by adjusting the target accuracy and confident
upper bound BER, the time estimates can be controlled so that the test time is
being spent to the best benefit. Inaccurate estimates of BER will lead to
overestimating test times.

When the test is complete, the measurements can then be analyzed. The calculated
accuracy can be applied to each BER and the upper BER limit can be found for each
error-free measurement. These can then be applied to the test result's waterfall
curve, allowing for more reliable and quantitative comparisons.

Let's assume that a measurement is to be performed using BPSK modulation with no
error correction and is operating at 60 Mbits/second. The Eb/No test points will
be at 8, 10, 12 and 13 dB. The selected confidence level is 99.9 percent. The
target accuracy is ±5 percent and the maximum test time allowed is 20 minutes.

As the background noise level (Eb/No) goes up, the test times go out. The problem
areas are 12 dB at over two hours and 13dB, which pushes test times out to 6.5
days!

But consider the effects of the BER estimate: If the system requirements do not
need any BER measurement better than 5.0e-09, ((IS THAT OK?))the test time is
sliced from days to hours.

This improves the test time considerably, but still too much time is being spent
at 12 dB and 13 dB. To improve the test times enough to satisfy all testing
concerns — to, say, 12 minutes — the accuracy at those points could be widened to
25 percent. This provides an acceptable total test time that is under 20 minutes,
with acceptable confidence and accuracy for results.

References

1) Statistical Treatment of Experimental Data: An Introduction to Statistical
Methods, by Hugh D. Young; McGraw-Hill Book Co., 1962; reissued by Waveland Press
Inc., Prospect Heights, Ill., 1996; ISBN 0-88133-913-X.

2) Modern Elementary Statistics, Fourth Edition, by John E. Fruend; Prentice-Hall
Inc., Englewood Cliffs, N.J., 1973; ISBN 0-13-593475-3.

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for Your HP, Epson, Canon or Lexmark
Printer at Myinks.com. Free s/h on orders $50 or more to the US & Canada. http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/l.m7sD/LIdGAA/qnsNAA/61FxlB/TM
---------------------------------------------------------------------~->

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/