Bootstrapping Crippendorff Alpha

Question

I would like to calculate - by bootstrapping the results of Crippendorf Alpha - the 95% confidence interval for the Crippendorff alpha coefficient for Raters Reliability using the R irr package.

Let us use the “C data from Krippendorff” in the irr and R script package to calculate Alpha Krippendorff once:

# the "C" data from Krippendorff
#rater per row; rated subject per column; NAs allowed
library(irr)
nmm<-matrix(c(1,1,NA,1,2,2,3,2,3,3,3,3,3,3,3,3,2,2,2,2,1,2,3,4,4,4,4,4,
1,1,2,1,2,2,2,2,NA,5,5,5,NA,NA,1,1,NA,NA,3,NA),nrow=4)
kripp.alpha(nmm,"ordinal")

Answer 1

You can use the function bootfrom the package bootto load values. Here I will upload a set of items, but I will not correct supporters:

library(boot)
library(irr)
ka <- function(data, indices) kripp.alpha(nmm[,indices], "ordinal")$value
b <- boot(seq(ncol(nmm)), ka, 1000)

boot.ci 95% ; , ( ?boot.ci):

boot.ci(b, type="perc")
# BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
# Based on 1000 bootstrap replicates
# 
# CALL : 
# boot.ci(boot.out = b, type = "perc")
# 
# Intervals : 
# Level     Percentile     
# 95%   ( 0.4297,  1.0000 )  
# Calculations and Intervals on Original Scale

Answer 2

, bootstrap, josliber, , . , boot() nXm-, kripp.alpha() mXn. , , , , 4 , , (, conf 1,0).

nXm, , , kripp.alpha().

alpha.boot <- function(data,x) {
   d <- t(data[x,])
   kripp.alpha(d,method="nominal")$value
} 

Answer 3

, , , , , (http://web.asc.upenn.edu/usr/krippendorff/boot.c-Alpha.pdf).

Repeating samples do not evaluate either units or units, but “hypothetical reliability data from a matrix of observed matches between pairs of values ​​assigned to units in independent repetitions” (Krippendorff 2016). Thus, the usual bootstrap implementations will not give the answer outlined by Crippendorf.

Best Daniel