On Combining Robustness and Regularization in Training Multilayer Perceptrons over Small Data

0562371 - ÚI 2023 RIV US eng C - Konferenční příspěvek (zahraniční konf.)
Kalina, Jan - Tumpach, Jiří - Holeňa, Martin
On Combining Robustness and Regularization in Training Multilayer Perceptrons over Small Data.
2022 International Joint Conference on Neural Networks (IJCNN) Proceedings. Piscataway: IEEE, 2022. ISBN 978-1-7281-8671-9.
[IJCNN 2022: International Joint Conference on Neural Networks /35./. Padua (IT), 18.07.2022-23.07.2022]
Grant CEP: GA ČR(CZ) GA22-02067S
Institucionální podpora: RVO:67985807
Klíčová slova: feedforward networks * nonlinear regression * outliers * robust neural networks * trend estimation
Obor OECD: Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
https://dx.doi.org/10.1109/IJCNN55064.2022.9892838

Multilayer perceptrons (MLPs) continue to be commonly used for nonlinear regression modeling in numerous applications. Available robust approaches to training MLPs, which allow to yield reliable results also for data contaminated by outliers, have not much penetrated to real applications so far. Besides, there remains a lack of systematic comparisons of the performance of robust MLPs, if their training uses one of regularization techniques, which are available for standard MLPs to prevent overfitting. This paper is interested in comparing the performance of MLPs trained with various combinations of robust loss functions and regularization types on small datasets. The experiments start with MLPs trained on individual datasets, which allow graphical visualizations, and proceed to a study on a set of 163251 MLPs trained on well known benchmarks using various combinations of robustness and regularization types. Huber loss combined with L2 - regularization turns out to outperform other choices. This combination is recommendable whenever the data do not contain a large proportion of outliers.
Trvalý link: https://hdl.handle.net/11104/0334710