Energy Complexity of Fully-Connected Layers

0573359 - ÚI 2024 RIV CH eng C - Konferenční příspěvek (zahraniční konf.)
Šíma, Jiří - Cabessa, Jérémie
Energy Complexity of Fully-Connected Layers.
Advances in Computational Intelligence. IWANN 2023 Proceedings, Part I. Cham: Springer, 2023 - (Rojas, I.; Joya, G.; Catala, A.), s. 3-15. Lecture Notes in Computer Science, 14134. ISBN 978-3-031-43084-8. ISSN 0302-9743.
[IWANN 2023: International Work-Conference on Artificial Neural Networks /17./. Ponta Delgada (PT), 19.06.2023-21.06.2023]
Grant CEP: GA ČR(CZ) GA22-02067S
Institucionální podpora: RVO:67985807
Klíčová slova: Convolutional neural networks * Energy complexity * Dataflow
Obor OECD: Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
https://dx.doi.org/10.1007/978-3-031-43085-5_1

The energy efficiency of processing convolutional neural networks (CNNs) is crucial for their deployment on low-power mobile devices. In our previous work, a simplified theoretical hardware-independent model of energy complexity for CNNs has been introduced. This model has been experimentally shown to asymptotically fit the power consumption estimates of CNN hardware implementations on different platforms. Here, we pursue the study of this model from a theoretically perspective in the context of fully-connected layers. We present two dataflows and compute their associated energy costs to obtain upper bounds on the optimal energy. Using the weak duality theorem, we further prove a matching lower bound when the buffer memory is divided into two fixed parts for inputs and outputs. The optimal energy complexity for fully-connected layers in the case of partitioned buffer ensues. These results are intended to be generalized to the case of convolutional layers.
Trvalý link: https://hdl.handle.net/11104/0343823