Training a Single Sigmoidal Neuron is Hard

Šíma, Jiří
Training a Single Sigmoidal Neuron is Hard.
Neural Computation. Roč. 14, č. 11 (2002), s. 2709-2729. ISSN 0899-7667. E-ISSN 1530-888X
Impact factor: 2.313, year: 2002
http://hdl.handle.net/11104/0124828

Cited: 26

--- Hammer B., Villmann T.: Mathematical aspects of neural networks. Proceedings of the ESANN’2003 European Symposuim on Artificial Neural Networks, pp. 59–72, Bruges: D-Side Publications, 2003.
--- Daqi G., Hua L., Changwu L.: On variable sizes and sigmoid activation functions of multilayer perceptrons. Proceedings of the IJCNN’2003 International Joint Conference on Neural Networks, pp. 2017–2022, New York: IEEE, 2003.
--- Igel C., Wiegand S., Friedrichs F.: Evolutionary optimization of neural systems: The use of self-adaptation. In M.G. de Bruin, D.H. Mache, J. Szabados (eds.): Trends and Applications in Constructive Approximation, International Series of Numerical Mathematics, Vol. 151, pp. 103–123, Basel: Birkhäuser Verlag, 2005.
--- SCHMITT, M. Some dichotomy theorems for neural learning problems. JOURNAL OF MACHINE LEARNING RESEARCH. ISSN 1532-4435, AUG 2004, vol. 5, p. 891-912. [WOS]
--- GORI, M. - SPERDUTI, A. The loading problem for recursive neural networks. NEURAL NETWORKS. ISSN 0893-6080, OCT 2005, vol. 18, no. 8, p. 1064-1079. [WOS]
--- WINDISCH, D. Loading deep networks is hard: The pyramidal case. NEURAL COMPUTATION. ISSN 0899-7667, FEB 2005, vol. 17, no. 2, p. 487-502. [WOS]
--- LEONI, P. Long-Range Out-of-Sample Properties of Autoregressive Neural Networks. NEURAL COMPUTATION. ISSN 0899-7667, JAN 2009, vol. 21, no. 1, p. 1-8. [WOS]
--- GAO, D.Q. - LIU, H. - LI, C.W. On variable sizes and sigmoid activation functions of multilayer perceptrons. PROCEEDINGS OF THE INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS 2003, VOLS 1-4. ISSN 1098-7576, 2003, p. 2017-2022. [WOS]
--- Igel C., Sendhoff B.: Synergies between evolutionary and neural computation. Proceedings of the ESANN'2005 European Symposuim on Artificial Neural Networks, pp. 241-252, Bruges: D-Side Publications, 2005.
--- Igel C., Sendhoff B. Genesis of organic computing systems: Coupling evolution and learning. In R. Würtz (Ed.), Organic Computing, Chapter 7, pp. 141-166. Springer-Verlag 2008.
--- Schmidhuber, J. Deep Learning in Neural Networks: An Overview. arXiv:1404.7828 [cs.NE]
--- SCHMIDHUBER, J. Deep learning in neural networks: An overview. NEURAL NETWORKS. ISSN 0893-6080, JAN 2015, vol. 61, p. 85-117. [WOS]
--- Palagi, L. Global Optimization issues in Supervised Learning. Global Optimization issues in Supervised Learning. An overview. Sapienza Universita de Roma, Technical Report n. 1, 2017
--- Maibing, S.F. - Igel, C. Computational Complexity of Linear Large Margin Classification With Ramp Loss. Proceedings of the 18th International Conference on Artificial Intelligence and Statistics (AISTATS) 2015, San Diego, CA, USA. JMLR: W&CP volume 38, p. 259-267
--- Gautier, A. - Nguyen, Q.N. - Hein, M. Globally Optimal Training of Generalized Polynomial Neural Networks with Nonlinear Spectral Methods. in Advances in Neural Information Processing Systems 29, NIPS 2016, p. 1687--1695
--- Anandkumar, A. - Sedghi, H. - Janzamin, M. Generalization Bounds for Neural Networks through Tensor Factorization. arXiv:1506.08473 [cs.LG] 2015
--- Anandkumar, A. - Sedghi, H. - Janzamin, M. Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods. arXiv:1506.08473 [cs.LG] 2016
--- Soudry, D. - Carmon, Y. No bad local minima: Data independent training error guarantees for multilayer neural networks. arXiv:1605.08361 [stat.ML] 2016
--- Du, S.D. - Lee, J.D. - Tian, Y. When is a Convolutional Filter Easy To Learn? arXiv:1709.06129 [cs.LG] 2017
--- Gautier, A. - Nguyen, Q. - Hein, M. Globally Optimal Training of Generalized Polynomial Neural Networks with Nonlinear Spectral Methods. arXiv:1610.09300 [cs.LG] 2016
--- Soudry, D. -Hoffer, E. Exponentially vanishing sub-optimal local minima in multilayer neural networks. arXiv:1702.05777 [stat.ML] 2017
--- Nguyen, Q. - Hein, M. The loss surface and expressivity of deep convolutional neural networks. arXiv:1710.10928 [cs.LG] 2017
--- Nguyen, Q. - Hein, M. The loss surface of deep and wide neural networks. arXiv:1704.08045 [cs.LG] 2017
--- Du, K.L. - Swamy, M.N.S. Perceptrons. In: Neural Networks and Statistical Learning. Springer, London 2014, p. 67-81
--- PALAGI, L. Global optimization issues in deep network regression: an overview. JOURNAL OF GLOBAL OPTIMIZATION. ISSN 0925-5001, FEB 2019, vol. 73, no. 2, p. 239-277. [WOS]
--- LI, Y.Z. - YUAN, Y. Convergence Analysis of Two-layer Neural Networks with ReLU Activation. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 30 (NIPS 2017). ISSN 1049-5258, 2017, vol. 30. [WOS]