Table 3

Consensus estimates of interrater reliability using percent agreement and Cohen’s weighted kappa, and consistency estimates using Spearman’s rho and Cronbach’s alpha

Skill/CriterionPercent Agreement Consensus/MeanCohen’s Weighted KappaSpearman’s rhoCronbach’s alpha
R1-R2R2-R3R1-R3MeanR1-R2R2-R3R1-R3R1-R2R2-R3R1-R3
Information exchange
1. Content: dentist’s suggestions70%70%60%67%0.6590.6340.5240.8300.7960.7220.903
2. Content: patient contribution90%90%80%87%0.8720.8780.7560.9010.9010.8290.963
3. Process: checking understanding80%80%80%80%0.5450.5830.6430.6120.8070.7910.870
Shared decision making
4. Process: negotiate treatment plan60%90%70%73%0.5450.8940.6150.9130.9490.9040.942
5. Perceptual: mutuality/respect70%60%90%73%0.6880.5740.8860.8730.8550.9640.939
6. Process: forward planning90%90%100%93%0.8390.8391.0000.8850.8851,0000.973
7. Process: closure90%90%100%93%0.9020.9021.0000.8610.8611,0000.993
8. Process: final check70%80%80%77%0.4290.7140.7440.5740.6450.9860.923
Global communication skills
9. Process: language skills40%80%60%60%0.2860.6670.5240.7910.8250.7950.900
10. Process: listening skills, acknowledging40%70%70%60%0.2860.6050.6590.6260.7020.8750.867
11. Process: perceptual, non-interruption70%80%70%73%0.286−0.1110.2860.408−0.1110.4080.516
12. Process: picking up verbal cues90%50%60%67%0.8210.3060.4740.8490.7600.6430.833
Building relationship skills
15. Process: atmosphere90%80%70%80%0.8080.7060.5000.9860.7450.7140.918
16. Perceptual: respect80%100%80%87%0.6671.0000.6670.8251,0000.8250.934
17. Perceptual: empathy60%60%30%50%0.3060.5240.1020.2820.6740.4880.736
18. Perceptual: acceptance50%70%50%57%0.2420.4440.2860.5270.5330.4970.718
  • Note: Items 13 and 14 (on non-verbal communication) were not part of this analysis.