Počet záznamů: 1
Disparities in Ratings of Internal and External Applicants: A Case for Model-based Inter-rater Reliability
- 1.
SYSNO ASEP 0494344 Druh ASEP J - Článek v odborném periodiku Zařazení RIV J - Článek v odborném periodiku Poddruh J Článek ve WOS Název Disparities in Ratings of Internal and External Applicants: A Case for Model-based Inter-rater Reliability Tvůrce(i) Martinková, Patrícia (UIVT-O) SAI, RID, ORCID
Goldhaber, D. (US)
Erosheva, E. (US)Číslo článku e0203002 Zdroj.dok. PLoS ONE. - : Public Library of Science - ISSN 1932-6203
Roč. 13, č. 10 (2018)Poč.str. 17 s. Jazyk dok. eng - angličtina Země vyd. US - Spojené státy americké Klíč. slova internal and external applicants ; inter-rater reliability ; mixed-effect models ; rater consistency ; assessee covariates ; teacher hiring Vědní obor RIV BB - Aplikovaná statistika, operační výzkum Obor OECD Sociology CEP GJ15-15856Y GA ČR - Grantová agentura ČR Institucionální podpora UIVT-O - RVO:67985807 UT WOS 000446545500004 EID SCOPUS 85054464213 DOI 10.1371/journal.pone.0203002 Anotace Ratings are present in many areas of assessment including peer review of research proposals and journal articles, teacher observations, university admissions and selection of new hires. One feature present in any rating process with multiple raters is that different raters often assign different scores to the same assessee, with the potential for bias and inconsistencies related to rater or assessee covariates. This paper analyzes disparities in ratings of internal and external applicants to teaching positions using applicant data from Spokane Public Schools. We first test for biases in rating while accounting for measures of teacher applicant qualifications and quality. Then, we develop model-based inter-rater reliability (IRR) estimates that allow us to account for various sources of measurement error, the hierarchical structure of the data, and to test whether covariates, such as applicant status, moderate IRR. We find that applicants external to the district receive lower ratings for job applications compared to internal applicants. This gap in ratings remains significant even after including measures of qualifications and quality such as experience, state licensure scores, or estimated teacher value added. With model-based IRR, we further show that consistency between raters is significantly lower when rating external applicants. We conclude the paper by discussing policy implications and possible applications of our model-based IRR estimate for hiring and selection practices in and out of the teacher labor market. Pracoviště Ústav informatiky Kontakt Tereza Šírová, sirova@cs.cas.cz, Tel.: 266 053 800 Rok sběru 2019
Počet záznamů: 1