Consensus estimates of interrater reliability using percent agreement and Cohen’s weighted kappa, and consistency estimates using Spearman’s rho and Cronbach’s alpha
Skill/Criterion | Percent Agreement Consensus/Mean | Cohen’s Weighted Kappa | Spearman’s rho | Cronbach’s alpha | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
R1-R2 | R2-R3 | R1-R3 | Mean | R1-R2 | R2-R3 | R1-R3 | R1-R2 | R2-R3 | R1-R3 | ||
Information exchange | |||||||||||
1. Content: dentist’s suggestions | 70% | 70% | 60% | 67% | 0.659 | 0.634 | 0.524 | 0.830 | 0.796 | 0.722 | 0.903 |
2. Content: patient contribution | 90% | 90% | 80% | 87% | 0.872 | 0.878 | 0.756 | 0.901 | 0.901 | 0.829 | 0.963 |
3. Process: checking understanding | 80% | 80% | 80% | 80% | 0.545 | 0.583 | 0.643 | 0.612 | 0.807 | 0.791 | 0.870 |
Shared decision making | |||||||||||
4. Process: negotiate treatment plan | 60% | 90% | 70% | 73% | 0.545 | 0.894 | 0.615 | 0.913 | 0.949 | 0.904 | 0.942 |
5. Perceptual: mutuality/respect | 70% | 60% | 90% | 73% | 0.688 | 0.574 | 0.886 | 0.873 | 0.855 | 0.964 | 0.939 |
6. Process: forward planning | 90% | 90% | 100% | 93% | 0.839 | 0.839 | 1.000 | 0.885 | 0.885 | 1,000 | 0.973 |
7. Process: closure | 90% | 90% | 100% | 93% | 0.902 | 0.902 | 1.000 | 0.861 | 0.861 | 1,000 | 0.993 |
8. Process: final check | 70% | 80% | 80% | 77% | 0.429 | 0.714 | 0.744 | 0.574 | 0.645 | 0.986 | 0.923 |
Global communication skills | |||||||||||
9. Process: language skills | 40% | 80% | 60% | 60% | 0.286 | 0.667 | 0.524 | 0.791 | 0.825 | 0.795 | 0.900 |
10. Process: listening skills, acknowledging | 40% | 70% | 70% | 60% | 0.286 | 0.605 | 0.659 | 0.626 | 0.702 | 0.875 | 0.867 |
11. Process: perceptual, non-interruption | 70% | 80% | 70% | 73% | 0.286 | −0.111 | 0.286 | 0.408 | −0.111 | 0.408 | 0.516 |
12. Process: picking up verbal cues | 90% | 50% | 60% | 67% | 0.821 | 0.306 | 0.474 | 0.849 | 0.760 | 0.643 | 0.833 |
Building relationship skills | |||||||||||
15. Process: atmosphere | 90% | 80% | 70% | 80% | 0.808 | 0.706 | 0.500 | 0.986 | 0.745 | 0.714 | 0.918 |
16. Perceptual: respect | 80% | 100% | 80% | 87% | 0.667 | 1.000 | 0.667 | 0.825 | 1,000 | 0.825 | 0.934 |
17. Perceptual: empathy | 60% | 60% | 30% | 50% | 0.306 | 0.524 | 0.102 | 0.282 | 0.674 | 0.488 | 0.736 |
18. Perceptual: acceptance | 50% | 70% | 50% | 57% | 0.242 | 0.444 | 0.286 | 0.527 | 0.533 | 0.497 | 0.718 |
Note: Items 13 and 14 (on non-verbal communication) were not part of this analysis.