Počet záznamů: 1
Item Difficulty Prediction Using Item Text Features: Comparison of Predictive Performance across Machine-Learning Algorithms
- 1.
SYSNO ASEP 0577144 Document Type J - Journal Article R&D Document Type Journal Article Subsidiary J Článek ve WOS Title Item Difficulty Prediction Using Item Text Features: Comparison of Predictive Performance across Machine-Learning Algorithms Author(s) Štěpánek, Lubomír (UIVT-O)
Dlouhá, Jana (UIVT-O)
Martinková, Patrícia (UIVT-O) SAI, RID, ORCIDArticle number 4104 Source Title Mathematics. - : MDPI - ISSN 2227-7390
Roč. 11, č. 19 (2023)Number of pages 30 s. Publication form Online - E Language eng - English Country CH - Switzerland Keywords text-based item difficulty prediction ; text features and item wording ; machine learning ; regularization methods ; elastic net regression ; support vector machines ; regression and decision trees ; random forests ; neural networks ; algorithm vs. domain expert’s prediction performance OECD category Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8) R&D Projects GA21-03658S GA ČR - Czech Science Foundation (CSF) Method of publishing Open access Institutional support UIVT-O - RVO:67985807 UT WOS 001084249000001 EID SCOPUS 85176471034 DOI https://doi.org/10.3390/math11194104 Annotation This work presents a comparative analysis of various machine learning (ML) methods for predicting item difficulty in English reading comprehension tests using text features extracted from item wordings. A wide range of ML algorithms are employed within both the supervised regression and the classification tasks, including regularization methods, support vector machines, trees, random forests, back-propagation neural networks, and Naïve Bayes. Moreover, the ML algorithms are compared to the performance of domain experts. Using f-fold cross-validation and considering the root mean square error (RMSE) as the performance metric, elastic net outperformed other approaches in a continuous item difficulty prediction. Within classifiers, random forests returned the highest extended predictive accuracy. We demonstrate that the ML algorithms implementing item text features can compete with predictions made by domain experts, and we suggest that they should be used to inform and improve these predictions, especially when item pre-testing is limited or unavailable. Future research is needed to study the performance of the ML algorithms using item text features on different item types and respondent populations. Workplace Institute of Computer Science Contact Tereza Šírová, sirova@cs.cas.cz, Tel.: 266 053 800 Year of Publishing 2024 Electronic address https://dx.doi.org/10.3390/math11194104
Počet záznamů: 1