Number of the records: 1
Model-Based Reliability to Check for Disparities in Ratings of Internal and External Applicants
- 1.
SYSNO ASEP 0493164 Document Type A - Abstract R&D Document Type O - Ostatní Title Model-Based Reliability to Check for Disparities in Ratings of Internal and External Applicants Author(s) Martinková, Patrícia (UIVT-O) SAI, RID, ORCID
Goldhaber, D. (US)
Erosheva, E. (US)Source Title Abstractband. VIII. European Congress on Methodology. Abstract Book. - Jena : Institut für Psychologie, Friedrich-Schiller-Universität, 2018
S. 102-102Number of pages 1 s. Publication form Online - E Action 2018 European Congress on Methodology /8./ Event date 25.07.2018 - 27.07.2018 VEvent location Jena Country DE - Germany Event type WRD Language eng - English Country DE - Germany Keywords Generalizability ; Linear Mixed Models ; Reliability ; Variance Decomposition Subject RIV BB - Applied Statistics, Operational Research OECD category Statistics and probability R&D Projects GJ15-15856Y GA ČR - Czech Science Foundation (CSF) Institutional support UIVT-O - RVO:67985807 Annotation ZÁKLADNÍ ÚDAJE: Abstractband. VIII. European Congress on Methodology. Abstract Book. Jena: Institut für Psychologie, Friedrich-Schiller-Universität, 2018. s. 102-102. 2018 European Congress on Methodology /8./. 25.07.2018-27.07.2018, Jena. Grant CEP: GA ČR GJ15-15856Y. ABSTRAKT: In this work we address disparities in ratings of internal and external applicants. We develop model-based inter-rater reliability (IRR) estimate to account for various sources of measurement error, their hierarchical structure and the presence of covariates, such as assessed status, that have the potential to moderate IRR. Using dataset of ratings of applicants to teaching positions in Spokane district in Washington, USA, we first test for bias in ratings of applicants external to the district, which is shown to be significant even after including various measures of teacher quality in the model. Moreover, withmodel-based IRR, we show that consistency between raters is significantly lower when rating external applicants. We further address how IRR affects the predictive power of measurement in different scenarios and conclude the work by discussing policy implications and applications of our model-based IRR estimate for teacher hiring practices. Workplace Institute of Computer Science Contact Tereza Šírová, sirova@cs.cas.cz, Tel.: 266 053 800 Year of Publishing 2019
Number of the records: 1