On the Relation between Generalized Entropy and the Bayes Decision Error

0351622 - ÚI 2011 RIV CZ eng J - Journal Article
Horáček, Martin
On the Relation between Generalized Entropy and the Bayes Decision Error.
European Journal for Biomedical Informatics. Roč. 6, č. 1 (2010), s. 38-42. ISSN 1801-5603
R&D Projects: GA MŠMT(CZ) 1M06014
Institutional research plan: CEZ:AV0Z10300504
Keywords : generalized entropy * f-entropy * Bayes error * average inaccuracy * power entropy * quadratic entropy * Shannon’s entropy * Emlen’s index * Ferreri’s index * Good’s index
Subject RIV: BB - Applied Statistics, Operational Research

We deal with the relation between generalized entropies (f-entropies) of a discrete random variable and the minimal posterior probability of error (Bayes error) when the value of the random variable is estimated. The tightness of their relation is studied by the means of recently introduced measure called the average inaccuracy. This measure is defined as a standardized average difference between the upper and the lower bound for the Bayes error under given entropy. It can be applied to any strictly concave f-entropy and used to evaluate its relation to the Bayes error. However, due to a complex form of the formula of the average inaccuracy, it is difficult to compare the average inaccuracies of most f-entropies analytically. We propose a smooth approximation of the lower bound for the Bayes error under given f-entropy that simplifies the formula. We show that under this approximation, the quadratic entropy has the tightest relation to the Bayes error among f-entropies.
Permanent Link: http://hdl.handle.net/11104/0191334