Robust Regularized Cluster Analysis for High-Dimensional Data

0431648 - ÚI 2015 RIV CZ eng C - Conference Paper (international conference)
Kalina, Jan - Vlčková, Katarína
Robust Regularized Cluster Analysis for High-Dimensional Data.
Proceedings of 32nd International Conference Mathematical Methods in Economics MME 2014. Olomouc: Palacký University, 2014 - (Talašová, J.; Stoklasa, J.; Talášek, T.), s. 378-383. ISBN 978-80-244-4209-9.
[MME 2014. International Conference Mathematical Methods in Economics /32./. Olomouc (CZ), 10.09.2014-12.09.2014]
R&D Projects: GA ČR GA13-17187S
Grant - others:GA ČR(CZ) GA13-01930S
Institutional support: RVO:67985807
Keywords : cluster analysis * robust data mining * big data * regularization
Subject RIV: BB - Applied Statistics, Operational Research

This paper presents new approaches to the hierarchical agglomerative cluster analysis for high-dimensional data. First, we propose a regularized version of the hierarchical cluster analysis for categorical data with a large number of categories. It exploits a regularized version of various test statistics of homogeneity in contingency tables as the measure of distance between two clusters. Further, our aim is cluster analysis of continuous data with a large number of variables. Various regularization techniques tailor-made for high-dimensional data have been proposed, which have however turned out to suffer from a high sensitivity to the presence of outlying measurements in the data. As a robust solution, we recommend to combine two newly proposed methods, namely a regularized version of robust principal component analysis and a regularized Mahalanobis distance, which is based on an asymptotically optimal regularization of the covariance matrix. We bring arguments in favor of the newly proposed methods.
Permanent Link: http://hdl.handle.net/11104/0236247