Abstract
The energy efficiency of processing convolutional neural networks (CNNs) is crucial for their deployment on low-power mobile devices. In our previous work, a simplified theoretical hardware-independent model of energy complexity for CNNs has been introduced. This model has been experimentally shown to asymptotically fit the power consumption estimates of CNN hardware implementations on different platforms. Here, we pursue the study of this model from a theoretically perspective in the context of fully-connected layers. We present two dataflows and compute their associated energy costs to obtain upper bounds on the optimal energy. Using the weak duality theorem, we further prove a matching lower bound when the buffer memory is divided into two fixed parts for inputs and outputs. The optimal energy complexity for fully-connected layers in the case of partitioned buffer ensues. These results are intended to be generalized to the case of convolutional layers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alwani, M., Chen, H., Ferdman, M., Milder, P.A.: Fused-layer CNN accelerators. In: Proceedings of IEEE/ACM MICRO 2016, pp. 22:1–22:12 (2016). https://doi.org/10.1109/MICRO.2016.7783725
Chen, Y., Emer, J.S., Sze, V.: Eyeriss: A spatial architecture for energy-efficient dataflow for convolutional neural networks. In: Proceedings of ACM/IEEE ISCA 2016, pp. 367–379 (2016). https://doi.org/10.1109/ISCA.2016.40
Mittal, S.: A survey of techniques for approximate computing. ACM Comput. Surv. 48(4), 62:1–62:33 (2016). https://doi.org/10.1145/2893356
Mittal, S.: A survey of FPGA-based accelerators for convolutional neural networks. Neural Comput. Appl. 32(4), 1109–1139 (2020). https://doi.org/10.1007/s00521-018-3761-1
Parashar, A., et al.: Timeloop: A systematic approach to DNN accelerator evaluation. In: Proceedings of IEEE ISPASS 2019, pp. 304–315 (2019). https://doi.org/10.1109/ISPASS.2019.00042
Shao, Y.S., et al.: Simba: Scaling deep-learning inference with multi-chip-module-based architecture. In: Proceedings of IEEE/ACM MICRO 2019, pp. 14–27 (2019). https://doi.org/10.1145/3352460.3358302
Šíma, J., Vidnerová, P., Mrázek, V.: Energy complexity model for convolutional neural networks. In: Proceedings of ICANN 2023. LNCS, Springer (2023)
Sze, V., Chen, Y., Yang, T., Emer, J.S.: Efficient Processing of Deep Neural Networks. Synthesis Lectures on Computer Architecture, Morgan & Claypool Publishers (2020). https://doi.org/10.2200/S01004ED1V01Y202004CAC050
Wu, Y.N., Emer, J.S., Sze, V.: Accelergy: An architecture-level energy estimation methodology for accelerator designs. In: Proceedings of IEEE/ACM ICCAD 2019 (2019). https://doi.org/10.1109/ICCAD45719.2019.8942149
Yang, T., Chen, Y., Emer, J.S., Sze, V.: A method to estimate the energy consumption of deep neural networks. In: Proceedings of IEEE ACSSC 2017, pp. 1916–1920 (2017). https://doi.org/10.1109/ACSSC.2017.8335698
Acknowledgements
The research was partially supported by the institutional support RVO: 67985807 and the Czech Science Foundation grant GA22-02067S. We thank Petr Savický for inspiring discussions in the early stages of this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Šíma, J., Cabessa, J. (2023). Energy Complexity of Fully-Connected Layers. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2023. Lecture Notes in Computer Science, vol 14134. Springer, Cham. https://doi.org/10.1007/978-3-031-43085-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-43085-5_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43084-8
Online ISBN: 978-3-031-43085-5
eBook Packages: Computer ScienceComputer Science (R0)