Skip to main content

Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Table 1 Weighted Kappa and percentage agreement scores for each session by round of pilot coding, set of transcripts and coding pair

From: Measuring fidelity of delivery of the Community Occupational Therapy in Dementia-UK intervention

Linear weighted Kappa (percentage agreement)
Set of transcriptsCoding pairIntroduction (1)OPHI (2)Ethnographic interview (3)Summary and Goal-setting (4&5)Consultation and advice (6)Evaluation (7)
Initial piloting
 11a0.79 (87)0.66 (68.8)0.48 (56.3)0.5 (58)6: 0.22 (35)
6 (2): 0.44 (65)
0.37 (44)
 11b0.93 (87)0.40 (56.3)0.2 (43.8)0.23 (41.7)6: 0.11 (58.8)
6 (2): 0.01 (41)
0.11 (33.3)
 21b0.65 (66.7)0.27 (43.8)0.57 (62.5)0.28 (50)6: 0.16 (47)
6 (2): 0.13 (35.3)
0.36 (55.6)
 120.40 (60)0.50 (62.5)0.49 (56.3)0.15 (37.5)6: 0.5 (64.7)
6 (2): 0.08 (47)
0.16 (33.3)
 220.50 (53.3)0.68 (75)0.81 (81.3)0.42 (54)6: 0.32 (41)
6 (2): 0.34 (47)
0.54 (66)
After initial piloting
 330.70 (80)0.75 (81.3)0.66 (75)4: 0.78 (82.6)
5: 0.37 (60.9)
0.58 (82.4)0.33 (55.6)
 430.71 (73.3)0.87 (87.5)1.00 (100)0.56 (59.6)0.3 (58.8)0.71 (87.5)
 530.57 (66.7)0.56 (68.8)0.48 (62.5)0.50 (69.6)0.42 (56.3)0.5 (50)
 540.56 (60)0.69 (81)0.67 (75)0.43 (65)0.61 (68.8)0.67 (75)
 550.49 (66.7)0.87 (87.5)0.67 (75)0.4 (56.5)0.31 (62.5)0.27 (37.5)
 640.67 (73.3)a0.87 (87.5)a0.81 (87.5)0.37 (52)0.54 (81.3)1.00 (100)
 8140.7 (80)a0.94 (93.8)a0.55 (68.8)0.46 (60.9)0.63 (75)0.73 (75)
 1240.66 (80)a0.81 (87.5)a0.87 (87.5)0.77 (88.2)0.93 (93.3)a0.58 (75)
 140.57 (70.6)0.59 (70.6)0.66 (73.3)a1.00 (100)
 240.81 (87.5)a0.34 (58.8)0.615 (80)a0.39 (62.5)
 940.66 (75)a0.64 (76.5)a0.74 (75)a
 1040.81 (81)a0.76 (76.5)a0.73 (75)a
 1140.49 (70.6)0.77 (87.5)a
 344: 0.94 (94.1)a
5: 0.82 (88.2)a
 440.44 (82.4)a2
  1. Note: Missing responses were accounted for in the analysis
  2. aIndicates agreement > 0.61 was reached | 1Coding guidelines not changed after coding this set | 2Weighted kappa did not reach > 0.61 however > 70% agreement achieved five times in a row and > 0.6 kappa 3 times in last five sets Kappa low due to distribution of responses: lots of ‘done’ responses, despite only three disagreements
  3. -: Agreement had already been reached and no further sessions needed to be coded until the 10% checks