The influence of first CNN layer initialization on training convergence

0578258 - ÚT 2024 CZ eng C - Conference Paper (international conference)
Krejsa, Jiří - Věchet, Stanislav - Chen, K.S.
The influence of first CNN layer initialization on training convergence.
Engineering Mechanics 2023 : 29th International Conference. Vol. 29. Prague: Institute of Thermomechanics of the Czech Academy of Sciences, 2023 - (Radolf, V.; Zolotarev, I.), s. 135-138. ISBN 978-80-87012-84-0. ISSN 1805-8248. E-ISSN 1805-8256.
[Engineering Mechanics 2023 /29./. Milovy (CZ), 09.05.2023-11.05.2023]
Institutional support: RVO:61388998
Keywords : convolution neural networks * training * initialization
OECD category: Automation and control systems
https://www.engmech.cz/im/proceedings/show_p/2023/135

During evaluation of convolution neural networks on the task of sign language single hand alphabet classification we have discovered that in small but not negligible number of cases the training of the network does not converge at all. This paper investigates the problem that we believe is independent of the application. While the true cause of training divergence was not discovered, we can offer the reader an easy solution from practical point of view – initialization of the first CNN layer using pretrained networks parameters.
Permanent Link: https://hdl.handle.net/11104/0351917