Subject: RE: [Speechsc] Small point on verification scores

From: "Pierre Forgues" <[email-masked]>
Date: Thu, 16 Jun 2005 09:39:22 -0400

Very well, I understand and agree with the [-1,1] range.  This actually
fits our model where -1 is considered an imposter, 0 is undecided and +1
is the true speaker.

We should note that the VoiceXML work is considering mapping this to an
integer from 0-100, but that is a different story.

Pierre

-----Original Message-----
From: Ran Zilca [mailto:[email-masked]] 
Sent: Thursday, June 16, 2005 9:33 AM
To: Pierre Forgues
Cc: David R. Oran; [email-masked]
Subject: RE: [Speechsc] Small point on verification scores

Once a system is deployed people do not always read the "fine print" in
the
spec, and there is a concern that people will interpret the scores as
probabilities based on the value range.

Ran

"Pierre Forgues" <[email-masked]> on 06/16/2005 09:28:13 AM

To:    Ran Zilca/Watson/IBM@IBMUS, "David R. Oran" <[email-masked]>
cc:    <[email-masked]>
Subject:    RE: [Speechsc] Small point on verification scores

I agree with the proposed wording, but would rather stay with a range
from 0.0 to 1.0.  What is the rationale/value for changing this to a
different range?

Pierre

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Ran Zilca
Sent: Thursday, June 16, 2005 9:20 AM
To: David R. Oran
Cc: [email-masked] ((((E-mail))))
Subject: Re: [Speechsc] Small point on verification scores

Normalizing the score to fit in a predetermined range has practical
advantages, but unfortunately it does not imply any probabilistic
interpretation. I believe that associating reliable probabilities to
scores
is still an open research problem. Instead, vendors are more likely to
provide users with offline tools that will analyze data and recommend
score
thresholds associated with desired errors (but the threshold values will
not have a direct probabilistic meaning).

To avoid further confusion I suggest to both adopt the new suggested
wording and also change the score range to be [-1,1].

-- Ran

Ran Zilca
Senior Software Engineer
IBM T. J. Watson Research Center
1101 Kitchawan Rd.
Yorktown Heights, NY 10598

http://www.research.ibm.com/CBG

Voice:      (914) 945-3013
Fax:  (914) 945-4490

"David R. Oran" <[email-masked]>@ietf.org on 06/16/2005 08:08:28 AM

Sent by:    [email-masked]

To:    "[email-masked] ((((E-mail))))" <[email-masked]>
cc:
Subject:    [Speechsc] Small point on verification scores

  The text says:

      "During verification, the higher the score the more likely
      it is that the speaker is the same one as the one who
      spoke the voiceprint utterances. During training, the
      higher the score the more likely the speaker is to have
      spoken all of the analyzed utterances. The value is a
      floating point between 0.0 and 1.0. If there are no such
      utterances the score is 0. It should be noted that though
      the value of the verification score is between 0.0 and 1.0
      it SHOULD NOT be interpreted as a probability value."

Ok, I'll bite. How can we say in the first sentence "the higher the
score the more likely is is that the speaker is
the same one who spoke the voiceprint utterances", and then say it
should not be interpreted as a probability value? If not a
probability, then...WHAT?

Are you saying that verification servers can't even normalize their
outputs to some credible statistical model?

That sucks, but if it's really so flakey we should say:

"it SHOULD NOT be interpreted as a probability value in the strict
statistical sense."

Comments?

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

---

From: "David R. Oran" <[email-masked]>
Date: Thu, 16 Jun 2005 09:51:26 -0400

I will change this and the newly-renamed min-verification-score  
header to -1:1 range unless I hear objection with persuasive rationale.

Dave.