Approximation of Classifiers by Deep Perceptron Networks

0572576 - ÚI 2024 RIV GB eng J - Journal Article
Kůrková, Věra - Sanguineti, M.
Approximation of Classifiers by Deep Perceptron Networks.
Neural Networks. Roč. 165, August 2023 (2023), s. 654-661. ISSN 0893-6080. E-ISSN 1879-2782
R&D Projects: GA ČR(CZ) GA22-02067S
Institutional support: RVO:67985807
Keywords : Approximation by deep networks * Probabilistic bounds on approximation errors * Random classifiers * Concentration of measure * Method of bounded differences * Growth functions
OECD category: Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Impact factor: 7.8, year: 2022
Method of publishing: Limited access
https://dx.doi.org/10.1016/j.neunet.2023.06.004

We employ properties of high-dimensional geometry to obtain some insights into capabilities of deep perceptron networks to classify large data sets. We derive conditions on network depths, types of activation functions, and numbers of parameters that imply that approximation errors behave almost deterministically. We illustrate general results by concrete cases of popular activation functions: Heaviside, ramp sigmoid, rectified linear, and rectified power. Our probabilistic bounds on approximation errors are derived using concentration of measure type inequalities (method of bounded differences) and concepts from statistical learning theory.
Permanent Link: https://hdl.handle.net/11104/0343221