Počet záznamů: 1
The EsnTorch Library: Effcient Implementation of Transformer-Based Echo State Networks
- 1.
SYSNO ASEP 0562588 Druh ASEP C - Konferenční příspěvek (mezinárodní konf.) Zařazení RIV D - Článek ve sborníku Název The EsnTorch Library: Effcient Implementation of Transformer-Based Echo State Networks Tvůrce(i) Cabessa, Jérémie (UIVT-O) ORCID
Hernault, H. (CH)
Lamonato, Y. (CH)
Rochat, M. (CH)
Levy, Y. Z. (CH)Zdroj.dok. Neural Information Processing. 29th International Conference, ICONIP 2022, Proceedings, Part VII. - Singapore : Springer, 2023 / Tanveer M. ; Agarwal A. ; Ozawa S. ; Ekbal A. ; Jatowt A. - ISSN 1865-0929 - ISBN 978-981991647-4 Rozsah stran s. 235-246 Poč.str. 12 s. Forma vydání Tištěná - P Akce ICONIP 2022: The International Conference on Neural Information Processing /29./ Datum konání 22.11.2022 - 26.11.2022 Místo konání Indore / Virtual Země IN - Indie Typ akce WRD Jazyk dok. eng - angličtina Země vyd. SG - Singapur Klíč. slova reservoir computing ; echo state networks ; natural language processing (NLP) ; text classification ; transformers ; BERT ; python library ; Hugging Face Obor OECD Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8) CEP GA22-02067S GA ČR - Grantová agentura ČR Institucionální podpora UIVT-O - RVO:67985807 EID SCOPUS 85161666487 DOI 10.1007/978-981-99-1648-1_20 Anotace Transformer-based models have revolutionized NLP. But in general, these models are highly resource consuming. Based on this consideration, several reservoir computing approaches to NLP have shown promising results. In this context, we propose EsnTorch, a library that implements echo state networks (ESNs) with transformer-based embeddings for text classification. EsnTorch is developed in PyTorch, optimized to work on GPU, and compatible with the transformers and datasets libraries from Hugging Face: the major data science platform for NLP. Accordingly, our library can make use of all the models and datasets available from Hugging Face. A transformer-based ESN implemented in EsnTorch consists of four building blocks: (1) An embedding layer, which uses a transformer-based model to embed the input texts, (2) A reservoir layer, which can implements three kinds of reservoirs: recurrent, linear or null, (3) A pooling layer, which offers three kinds of pooling strategies: mean, last, or None, (4) And a learning algorithm block, which provides six different supervised learning algorithms. Overall, this work falls within the context of sustainable models for NLP. Pracoviště Ústav informatiky Kontakt Tereza Šírová, sirova@cs.cas.cz, Tel.: 266 053 800 Rok sběru 2024 Elektronická adresa https://dx.doi.org/10.1007/978-981-99-1648-1_20
Počet záznamů: 1