Argument Mining with Modular BERT and Transfer Learning

0577167 - ÚI 2024 RIV US eng C - Konferenční příspěvek (zahraniční konf.)
Mushtaq, U. - Cabessa, Jérémie
Argument Mining with Modular BERT and Transfer Learning.
IJCNN 2023 Conference Proceedings. Piscataway: IEEE, 2023, č. článku 191330. ISBN 978-1-6654-8867-9.
[IJCNN 2023: International Joint Conference on Neural Networks /36./. Queensland (AU), 18.06.2023-23.06.2023]
Grant CEP: GA ČR(CZ) GA22-02067S
Institucionální podpora: RVO:67985807
Klíčová slova: Argument Mining * BERT * Features as Text * modular BERT * NLP * Transfer Learning
Obor OECD: Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

We introduce BERT-MINUS, a modular, feature-enriched and transfer learning enabled model for Argument Mining. BERT-MINUS consists of: 1) a joint module which embeds the paragraph text, and 2) a dedicated module, consisting of three customized BERT models, which contextualize the argument markers, argument components and additional features given as text, respectively. BERT-MINUS implements two kinds of transfer learning - auto-transfer (transfer from a task to itself) and cross-transfer (classical transfer from one task to another) - via a novel Selective Fine-tuning mechanism. BERT-MINUS achieves state-of-the-art results on the Link Identification task and competitive results on the Argument Type Classification task. The synergy between the Features as Text and Selective Fine-tuning mechanisms significantly improves the performance of the model. Our work reveals the importance and potential of transfer learning via selective fine-tuning for modular Language Models. Moreover, this study dovetails naturally into the Prompt Engineering paradigm in NLP.
Trvalý link: https://hdl.handle.net/11104/0346396