On this blog, I discuss about some techniques and general issues related to the design and analysis of inter-rater reliability studies. My mission is to help researchers improve how they address inter-rater reliability assessments through the learning of simple and specific statistical techniques that the community of statisticians has left us to discover on our own.
Tuesday, February 16, 2021
New peer-reviewed article
Many statistical statistical packages have implemented the wrong variance equation of Fleiss' generalized kappa (Fleiss, 1971). SPSS and the R package "rel" are among these packages. I recently published in "Educational and Psychological Measurement" an article entitled "Large-Sample Variance of Fleiss Generalized Kappa." I show in this article that it is not Fleiss' variance equation that is wrong. Instead, it is the way it has been used that is. Fleiss' variance equation was developed under the assumption of no agreement among raters and for the sole purpose of being used in hypothesis testing. It does not quantify the precision of Fleiss' generalized kappa and cannot be used for constructing confidence intervals either.
Subscribe to:
Post Comments (Atom)
PPCexpo has introduced a horizontal gray bar to measure the data from the previous period on a Pareto chart. This allows you to compare data from the most recent period with data from the previous period, which helps to investigate the status of significant contributions in the previous period.
ReplyDeletehttps://ppcexpo.com/blog/how-to-read-pareto-chart