Sketch2Code: Automatic hand-drawn UI elements detection with Faster R-CNN

0536724 - ÚTIA 2021 RIV DE eng C - Conference Paper (international conference)
Zita, Aleš - Picek, L. - Říha, A.
Sketch2Code: Automatic hand-drawn UI elements detection with Faster R-CNN.
CEUR Workshop Proceedings : Volume 2696. Working Notes of CLEF 2020 - Conference and Labs of the Evaluation Forum. Achen: CEUR-WS.org, 2020, č. článku 82. ISSN 1613-0073.
[CLEF 2020. Thessaloniki (GR), 22.09.2020-25.09.2020]
Grant - others:GA MŠk(CZ) LO1506
Institutional support: RVO:67985556
Keywords : Computer Vision * Object Detection * Machine Learning
OECD category: Robotics and automatic control
http://library.utia.cas.cz/separaty/2020/ZOI/zita-0536724.pdf

Transcription of User Interface (UI) elements hand drawings to the computer code is a tedious and repetitive task. Therefore, a need arose to create a system capable of automating such process. This paper describes a deep learning-based method for hand-drawn user interface elements detection and localization. The proposed method scored 1st place in the ImageCLEFdrawnUI competition while achieving an overall precision of 0.9708. The final method is based on Faster R-CNN object detector framework with ResNet-50 backbone architecture trained with advanced regularization techniques. The code has been made available at: https://github.com/picekl/ImageCLEF2020-DrawnUI.

Permanent Link: http://hdl.handle.net/11104/0314461