Robust Multilayer Perceptrons: Robust Loss Functions and Their Derivatives

0524790 - ÚI 2021 RIV CH eng C - Conference Paper (international conference)
Kalina, Jan - Vidnerová, Petra
Robust Multilayer Perceptrons: Robust Loss Functions and Their Derivatives.
Proceedings of the 21st EANN (Engineering Applications of Neural Networks) 2020 Conference. Cham: Springer, 2020 - (Iliadis, L.; Parvanov Angelov, P.; Jayne, C.; Pimenidis, E.), s. 546-557. Proceedings of the International Neural Networks Society. ISBN 978-3-030-48790-4. ISSN 2661-8141.
[EANN 2020: International Conference on Engineering Applications of Neural Networks /21./. Halkidiki (GR), 05.06.2020-07.06.2020]
R&D Projects: GA ČR(CZ) GA19-05704S; GA ČR(CZ) GA18-23827S
Institutional support: RVO:67985807
Keywords : Neural networks * Loss functions * Robust regression
OECD category: Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
https://link.springer.com/chapter/10.1007%2F978-3-030-48791-1_43

Common types of artificial neural networks have been well known to suffer from the presence of outlying measurements (outliers) in the data. However, there are only a few available robust alternatives for training common form of neural networks. In this work, we investigate robust fitting of multilayer perceptrons, i.e. alternative approaches to the most common type of feedforward neural networks. Particularly, we consider robust neural networks based on the robust loss function of the least trimmed squares, for which we express formulas for derivatives of the loss functions. Some formulas, which are however incorrect, have been already available. Further, we consider a very recently proposed multilayer perceptron based on the loss function of the least weighted squares, which appears a promising highly robust approach. We also derive the derivatives of the loss functions, which are to the best of our knowledge a novel contribution of this paper. The derivatives may find applications in implementations of the robust neural networks, if a (gradient-based) backpropagation algorithm is used.
Permanent Link: http://hdl.handle.net/11104/0309079