Human Computer Interface Based on Tongue and Lips Movements and its Application for Speech Therapy System

0524713 - ÚTIA 2021 RIV US eng C - Konferenční příspěvek (zahraniční konf.)
Bílková, Zuzana - Novozámský, Adam - Bartoš, Michal - Dominec, Adam - Greško, Šimon - Zitová, Barbara - Paroubková, Markéta - Flusser, Jan
Human Computer Interface Based on Tongue and Lips Movements and its Application for Speech Therapy System.
Electronic Imaging, Visualization and Data Analysis 2020. Springfield: Society for Imaging Science and Technology, 2020, č. článku 389. ISSN 2470-1173.
[IS&T International Symposium on Electronic Imaging 2020. Burlingame, California (US), 26.01.2020-30.01.2020]
Grant CEP: GA TA ČR TJ01000181
Grant ostatní: GA MŠk(CZ) SVV–2017–260452; AV ČR(CZ) AP1701
Program: Akademická prémie - Praemium Academiae
Institucionální podpora: RVO:67985556
Klíčová slova: speech therapy * deep learning * tongue detection * lips detection
Obor OECD: Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
http://library.utia.cas.cz/separaty/2020/ZOI/zitova-0524713.pdf

The novel human computer interface is introduced, based ontongue and lips movements and using video data from a commercially available camera. The size and direction of the movementsare extracted and can be used for setting cursor actions or to otherrelevant activities. The movement detection is based on convolu-tional neural networks. The applicability of the proposed solutionis shown on the ASSISLT system, aimed to support speechtherapy for adults and children with inborn and acquired motorspeech disorders. The system focuses on individual treatment us-ing exercises that improve tongue motion and thus articulation.The system offers an adjustable set of exercises which proper per-formance is motivated using augmented reality. Automatic evaluation of the performance of therapeutic movements allows thetherapist to objectively follow the progress of the treatment.
Trvalý link: http://hdl.handle.net/11104/0309170