Approximation of Unknown Multivariate Probability Distributions by Using Mixtures of Product Components: A Tutorial

0475182 - ÚTIA 2018 RIV GB eng J - Journal Article
Grim, Jiří
Approximation of Unknown Multivariate Probability Distributions by Using Mixtures of Product Components: A Tutorial.
International Journal of Pattern Recognition and Artificial Intelligence. Roč. 31, č. 9 (2017), č. článku 1750028. ISSN 0218-0014. E-ISSN 1793-6381
R&D Projects: GA ČR GA17-18407S
Institutional support: RVO:67985556
Keywords : multivariate statistics * product mixtures * naive Bayes models * EM algorithm * pattern recognition * neural networks * expert systems * image analysis
OECD category: Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Impact factor: 1.029, year: 2017
http://library.utia.cas.cz/separaty/2017/RO/grim-0475182.pdf

In literature the references to EM estimation of product mixtures are not very frequent. The simplifying assumption of product components, e.g. diagonal covariance matrices in case of Gaussian mixtures, is usually considered only as a compromise because of some computational constraints or limited data set. We have found that the product mixtures are rarely used intentionally as a preferable approximating tool. Probably, most practitioners do not „trust“ the product components because of their formal similarity to „naive Bayes models“. Another reason could be an unrecognized numerical instability of EM algorithm in multidimensional spaces. In this paper we recall that the product mixture model does not imply the assumption of independence of variables. It is even not restrictive if the number of components is large enough. In addition, the product components increase numerical stability of the standard EM algorithm, simplify the EM iterations and have some other important advantages. We discuss and explain the implementation details of EM algorithm and summarize our experience in estimating product mixtures. Finally we illustrate the wide applicability of product mixtures in pattern recognition and in other fields.
Permanent Link: http://hdl.handle.net/11104/0272087