Počet záznamů: 1
Energy Complexity of Recurrent Neural Networks
- 1.
SYSNO ASEP 0393985 Druh ASEP J - Článek v odborném periodiku Zařazení RIV J - Článek v odborném periodiku Poddruh J Článek ve WOS Název Energy Complexity of Recurrent Neural Networks Tvůrce(i) Šíma, Jiří (UIVT-O) RID, SAI, ORCID Zdroj.dok. Neural Computation - ISSN 0899-7667
Roč. 26, č. 5 (2014), s. 953-973Poč.str. 21 s. Jazyk dok. eng - angličtina Země vyd. US - Spojené státy americké Klíč. slova neural network ; finite automaton ; energy complexity ; optimal size Vědní obor RIV IN - Informatika CEP GAP202/10/1333 GA ČR - Grantová agentura ČR Institucionální podpora UIVT-O - RVO:67985807 UT WOS 000334027800005 EID SCOPUS 84897975813 DOI 10.1162/NECO_a_00579 Anotace Recently, a new so-called energy complexity measure has been introduced and studied for feedforward perceptron networks. This measure is inspired by the fact that biological neurons require more energy to transmit a spike than not to fire, and the activity of neurons in the brain is quite sparse, with only about 1% of neurons firing. In this paper, we investigate the energy complexity of recurrent networks which counts the number of active neurons at any time instant of a computation. We prove that any deterministic finite automaton with m states can be simulated by a neural network of optimal size s=\Theta(\sqrt{m}) with the time overhead of \tau=O(s/e) per one input bit, using the energy O(e), for any e such that e=\Omega(\log s) and e=O(s), which shows the time-energy tradeoff in recurrent networks. In addition, for the time overhead \tau satisfying \tau^\tau=o(s), we obtain the lower bound of s^{c/\tau} on the energy of such a simulation, for some constant c>0 and for infinitely many s. Pracoviště Ústav informatiky Kontakt Tereza Šírová, sirova@cs.cas.cz, Tel.: 266 053 800 Rok sběru 2015
Počet záznamů: 1