Bayesian location-scale model for assessing reliability differences with ordinal ratings

0580425 - ÚI 2024 RIV US eng A - Abstrakt
Bartoš, František - Martinková, Patrícia - Brabec, Marek
Bayesian location-scale model for assessing reliability differences with ordinal ratings.
IMPS 2023: Abstract Book: Talks. Psychometric Society, 2023. s. 88-88.
[IMPS 2023: International Meeting of the Psychometric Society. 23.07.2023-28.07.2023, Washington DC]
Grant CEP: GA ČR(CZ) GA21-03658S
Institucionální podpora: RVO:67985807
Klíčová slova: reliability * ordinal ratings * Bayesian location-scale model
Obor OECD: Statistics and probability
https://www.psychometricsociety.org/sites/main/files/file-attachments/imps2023-abstracts.pdf

ZÁKLADNÍ ÚDAJE: IMPS 2023: Abstract Book: Talks. Psychometric Society, 2023. s. 88-88. [IMPS 2023: International Meeting of the Psychometric Society. 23.07.2023-28.07.2023, Washington DC]. ABSTRAKT: The quality of ratings and quantitative assessments depends on the reliability of the rating instrument. Especially important is the measurement error – a high measurement error results in high uncertainty of the resulting scores. Detected systematic differences in measurement error due to applicant/raters-related characteristics might provide guidance on which groups to focus on in interventions designed to lower the measurement error. A flexible approach for detecting differences in measurement error was proposed in Martinková et al., 2023) for cases when scores are assumed to be continuous. In this work, we build on this approach by focusing on ordinal ratings. We highlight cases where treating ordinal rating as continuous might result in biased estimates and outline a Bayesian cumulative probit multi-level location-scale model to mitigate the issue. We use spike-andslab prior distributions to obtain inclusion Bayes factors of individual predictors and model-averaged posterior distributions within a single model fit. We demonstrate the superiority of the proposed ordinal approach with a simulation study.
Trvalý link: https://hdl.handle.net/11104/0349214