On this blog, I discuss about some techniques and general issues related to the design and analysis of inter-rater reliability studies. My mission is to help researchers improve how they address inter-rater reliability assessments through the learning of simple and specific statistical techniques that the community of statisticians has left us to discover on our own.
Sunday, December 7, 2014
Inter-Rater Reliability in Language Testing
That paper entitled Assessing inter-rater agreement for nominal judgement variables, (alternative link) summarizes a simple comparative study between Cohen's Kappa and Gwet's AC1 for evaluating inter-rater reliability in the context of Language Testing, dichotomous variables, and high-prevalence data. Researchers may find this analysis instructive. I personally found it attractive for its simplicity, and the clarity of the examples used. Those who are new in the area of inter-rater reliability assessment may find it useful as well.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment