Subject: [PSAMP] PSAMP-INFO IE realtiveError

From: "Zseby, Tanja" <[email-masked]>
Date: Thu, 31 Jul 2008 19:05:48 +0200

Hi Benoit and Paul,

here my suggestions for clarification of the error IEs in PSAMP-INFO.

-    I suggest to rename the fixedError to absoluteError

-    I suggest to introduce CI limits and level to also report
estimation errors

-    If it is still possible I would suggest to make a few small changes
in PSAMP-PROTO for consistency.

-    Upper and lower CI limits can be also specified as provided
absolute or relative limits. So we could also add 2 more IEs (for the
relative CI limits)

New description of IEs:

absoluteError

This Information Element specifies the maximum possible measurement
error of the reported value for a given Information Element. The
absoluteError has the same unit as the information element it is
associated to. The real value of the metric can differ by absoluteError
(positive or negative) from the measured value. This information element
provides only the error for measured values. If an information element
contains an estimated values (from sampling) the confidence boundaries
and confidence level have to be provided instead.

relativeError

This Information Element specifies the maximum possible measurement
error of the reported value for a given Information Element as
percentage of the measured value. The real value of the metric can
differ by relativeError percent (positive or negative) from the measured
value. This information element provides only the error for measured
values. If an information element contains an estimated values (from
sampling) the confidence boundaries and confidence level have to be
provided instead.

upperCILimit

This Information Element specifies the upper limit of a confidence
interval. It is used to provide an accuracy statement for an estimated
value. The confidence limits define the range in which the real value is
assumed to be with a certain probability p. Confidence limits always
need to be associated with a confidence level that defines this
probability p. Please note that a confidence interval only provides a
probability that the real values lies within the limits. That means the
real value can lie outside the confidence limits.

lowerCILimit

This Information Element specifies the lower limit of a confidence
interval. For further information see the description of upperCILimit.

confidenceLevel

This Information Element specifies the confidence level. It is used to
provide an accuracy statement for estimated values. The confidence level
provides the probability p with which the real value lies within a given
range. A confidence level always needs to be associated with confidence
limits that define the range in which the real value is assumed to be.

Changes to PSAMP-PROTO if still possible:

-    Rename fixedError to absoluteError

-    Slightly modify paragraph 2

OLD:

...  The accuracy SHOULD be reported either with the fixedError
Information Element [PSAMP-INFO
<http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-INFO>
], or with the relativeError Information Element [PSAMP-INFO
<http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-INFO>
].

NEW:

... The accuracy for a measured information elelment SHOULD be reported
either with the fixedError Information Element [PSAMP-INFO
<http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-INFO>
], or with the relativeError Information Element [PSAMP-INFO
<http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-INFO>
]. The accuracy for an estimated information element (from sampling)
SHOULD be reported with confidence limits and confidence
level.[PSAMP-INFO]

-    Remove the following paragraph (very important! Otherwise it would
lead to confusion):

For example, the accuracy of an Information Element to estimate the
accuracy of a sampled flow, for which the unit would be specified in
octets, can be specified with the relativeError Information Element with
the octet units.  In this case, the error interval is the Information
Element value +/- the value reported in the relativeError times the
reported Information Element value.

-    Avoid the term error interval

OLD: 

In this case, the error interval is the Information Element value +/-
the value reported in the fixedError.

NEW: 

In this case, the real values lies within the range of the Information
Element value +/- the value reported in the absoluteError.

-    Remove the following paragraph (since absolute or relative error
are just different representations I would not gain something if I
report both)

Alternatively to reporting either the fixedError Information Element or
the relativeError Information Element in the Accuracy Report
Interpretation, both Information Elements MAY be present.  This scenario
could help in more complex situations where the system clock drifts, on
the top of having its own accuracy, during the duration of a
measurement.

Sorry for the late comments, I was quite busy with PSAMP-TECH before...

Kind regards,

Tanja

_______________________________________________
PSAMP mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/psamp

---

From: Benoit Claise <[email-masked]>
Date: Thu, 31 Jul 2008 23:46:09 +0100

Hi Tanja,
> -    I suggest to rename the fixedError to absoluteError
No problem with that, but [PSAMP-PROTO] must follow otherwise we have a 
problem
Can we still change that in AUTH48 maybe?
> -    I suggest to introduce CI limits and level to also report 
I'm wondering whether this is a good idea to add upperCILimit, 
lowerCILimit, and confidenceLevel at this stage.
Because it implies that we need a complete new section in [PSAMP-PROTO] 
(as opposed to just editorial change) similar to "Accuracy Report 
Interpretation" but for the accuracy statement for estimated value. 
Now, the simple solution is to add the information elements in 
PSAMP-INFO and don't discuss the accuracy statement for estimated value 
in [PSAMP-PROTO].
>
> -    If it is still possible I would suggest to make a few small 
> changes in PSAMP-PROTO for consistency.
>
> associated to. The real value of the metric can differ by 
> absoluteError (positive or negative) from the measured value. This 
> information element provides only the error for measured values. If an 
> information element contains an estimated values (from sampling) the 
> confidence boundaries and confidence level have to be provided instead.
>
>  
>
> relativeError
>
> differ by relativeError percent (positive or negative) from the 
> measured value. This information element provides only the error for 
> measured values. If an information element contains an estimated 
> values (from sampling) the confidence boundaries and confidence level 
> have to be provided instead.
>
I like your suggestions for absoluteError and relativeError because 
something that was not clear (neither from PSAMP-PROTO or PSAMP-INFO) is 
that we wanted to quantify the accuracy of the measurement estimation, 
as opposed to the accuracy of the estimated value
> value. The confidence limits define the range in which the real value 
> is assumed to be with a certain probability p. Confidence limits 
> always need to be associated with a confidence level that defines this 
> probability that the real values lies within the limits. That means 
> the real value can lie outside the confidence limits.
>
>  
>
> lowerCILimit
>
> provide an accuracy statement for estimated values. The confidence 
> level provides the probability p with which the real value lies within 
> a given range. A confidence level always needs to be associated with 
> confidence limits that define the range in which the real value is 
> assumed to be.
>
>  
>
>  
>
> <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-INFO>], 
> or with the relativeError Information Element [PSAMP-INFO 
> <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-INFO>].
>
> NEW:
>
> ... The accuracy for a measured information elelment SHOULD be 
> reported either with the fixedError Information Element [PSAMP-INFO 
> <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-INFO>], 
> or with the relativeError Information Element [PSAMP-INFO 
> <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-INFO>]. 
> T
>
To be consistent with my statement above, I would not add the following 
sentence.
>
> he accuracy for an estimated information element (from sampling) 
> -    Remove the following paragraph (very important! Otherwise it 
> would lead to confusion):
>
> octets, can be specified with the relativeError Information Element 
> with the octet units.  In this case, the error interval is the 
> Information Element value +/- the value reported in the relativeError 
> times the reported Information Element value.
>
>  
>
> Alternatively to reporting either the fixedError Information Element 
> or the relativeError Information Element in the Accuracy Report 
> Interpretation, both Information Elements MAY be present.  This 
> scenario could help in more complex situations where the system clock 
> drifts, on the top of having its own accuracy, during the duration of 
> a measurement.
>
I would also change "Accuracy Report Interpretation" into "Measurement 
Accuracy Report Interpretation" in  [PSAMP-PROTO]

Regards, Benoit.

---

From: "Zseby, Tanja" <[email-masked]>
Date: Fri, 1 Aug 2008 10:28:44 +0200

Hi Benoit,

I think we need no new section in PSAMP-PROTO and can only introduce the
CI limits in PSAMP-INFO. We can stick with the example in PSAMP-PROTO
but need to clarify that the example is only for the accuracy of
measured values. That means mainly to remove the 2 sentences that refer
to estimation error. It depends how much we can still change in
PSAMP-PROTO. Is removing 2 sentences o.k.?

Kind regards
Tanja

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Benoit Claise
> Sent: Friday, August 01, 2008 12:46 AM
> To: Zseby, Tanja
> Cc: psamp; Juergen Quittek
> Subject: Re: [PSAMP] PSAMP-INFO IE realtiveError
> 
> Hi Tanja,
> 
> 
> 	Hi Benoit and Paul,
> 
> 
> 
> problem Can we still change that in AUTH48 maybe?
> 
> 
> 
> 
> Now, the simple solution is to add the information elements in PSAMP-
> INFO and don't discuss the accuracy statement for estimated value in
> [PSAMP-PROTO].
> 
> 
> 
> 
> 	This Information Element specifies the maximum possible
> measurement error of the reported value for a given Information
> Element. The absoluteError has the same unit as the information
element
> it is associated to. The real value of the metric can differ by
> 	This Information Element specifies the maximum possible
> measurement error of the reported value for a given Information
Element
> as percentage of the measured value. The real value of the metric can
values
> (from sampling) the confidence boundaries and confidence level have to
> be provided instead.
> 
> something that was not clear (neither from PSAMP-PROTO or PSAMP-INFO)
> is that we wanted to quantify the accuracy of the measurement
> estimation, as opposed to the accuracy of the estimated value
> 
> 
> 
> 
> 
> 
> 	upperCILimit
> 
> 	This Information Element specifies the upper limit of a
> confidence interval. It is used to provide an accuracy statement for
an
> estimated value. The confidence limits define the range in which the
> real value is assumed to be with a certain probability p. Confidence
> limits always need to be associated with a confidence level that
> defines this probability p. Please note that a confidence interval
only
> provides a probability that the real values lies within the limits.
> That means the real value can lie outside the confidence limits.
> 
> 
> 
> 	lowerCILimit
> 
> 	This Information Element specifies the lower limit of a
> confidence interval. For further information see the description of
> upperCILimit.
> 
> 
> 
> 	confidenceLevel
> 
> 	This Information Element specifies the confidence level. It is
> used to provide an accuracy statement for estimated values. The
> confidence level provides the probability p with which the real value
> lies within a given range. A confidence level always needs to be
> associated with confidence limits that define the range in which the
> real value is assumed to be.
> 
> 
> 
> 
> 
<http://tools.ietf.org/html/draft-ietf-
> psamp-protocol-09#ref-PSAMP-INFO> ], or with the relativeError
<http://tools.ietf.org/html/draft-ietf-
> psamp-protocol-09#ref-PSAMP-INFO> ].
> 
> 	NEW:
> 
> <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-
> INFO> ], or with the relativeError Information Element [PSAMP-INFO
> <http://tools.ietf.org/html/draft-ietf-psamp-protocol-09#ref-PSAMP-
> INFO> ]. T
> 
> SHOULD be reported with confidence limits and confidence level.[PSAMP-
> INFO]
> 
> 
> 
> 	For example, the accuracy of an Information Element to estimate
> the accuracy of a sampled flow, for which the unit would be specified
> in octets, can be specified with the relativeError Information Element
> 	In this case, the error interval is the Information Element
value
> +/- the value reported in the fixedError.
> 
> 	NEW:
> 
> 	In this case, the real values lies within the range of the
> Information Element value +/- the value reported in the absoluteError.
> 
> 
> 
> 
> 
> 	-    Remove the following paragraph (since absolute or relative
> error are just different representations I would not gain something if
> I report both)
> 
> 	Alternatively to reporting either the fixedError Information
> Element or the relativeError Information Element in the Accuracy
Report
> drifts, on the top of having its own accuracy, during the duration of
a