Abstract
Evidence suggests that involving patients in health care decisions has a positive impact on health care outcomes and patients’ perception of quality. However, the extent to which dental students are trained in communication and shared decision making (SDM) differs, and studies have identified a need for intensified learning and assessment of this competence. A need to more clearly define and operationalize what it means to be proficient in this area has been identified. The aim of this study was to operationalize communicative and relational skills in a comprehensive assessment instrument for SDM. Relevant skills in information exchange, negotiation, communication, and relationship-building were identified through an extensive review of previous research and instruments for assessing communication competence. Indicators for assessing these skills were formulated. The instrument was submitted to a pilot test in 2016 and evaluated on test content, internal structure, and response processes. The Assessment of Shared Decision Making (ASDM) instrument consists of 18 items addressing various aspects of the construct and three types of skills. Findings suggest that the ASDM represents a valid measure of SDM with three major components. The importance of developing the ASDM lies both in the summative assessment of students’ communication with patients and for formative assessment purposes. Once identified, the components essential for SDM can be woven into the curriculum and shared with students. Thus, the ASDM provides a structure that can meet the need for intensified learning and assessment of dental students’ competence in communication and SDM.
- dental education
- assessment
- interpersonal communication
- patient care management
- dentist-patient relations
Previous research has suggested that involving patients in health care decisions has a positive effect on outcomes.1–4 Historically, health care providers chose what to disclose about treatment options to a patient, thereby taking full responsibility for the decision making.1 Recently, as part of patient-centered care, there has been a change towards sharing responsibility for treatment decisions with the patient.5,6 Patients have also become increasingly interested in being involved in negotiating a mutual plan for treatment.7 This change towards greater patient involvement has been supported by a change in legislation and recommendations, giving patients the right to be active participants in decisions regarding their treatment.8,9 Another contributing factor is a decrease in trust in health care providers, in which some members of the public feel that the dentists’ priorities have become more self-centered.10 Patients are also increasingly searching for online information and expect to discuss what they find with their dentists.7
Patient involvement affects the relationship between dentist and patient, and dentists may have problems involving patients in the decision making and negotiating a treatment plan acceptable to both parties.11,12 A prerequisite for negotiation is shared understanding, in which the patient needs to understand relevant parts of the dentist’s expertise.4 Similarly, the dentist needs to have the competence to access and understand the patient’s ideas and feelings,13 as well as to adjust the communication to the patient’s level of knowledge.14
Dental education could be considered a natural site for acquiring the competence required for shared decision making (SDM).15,16 However, the extent to which dental students are trained in communication differs, and Hannah et al. noted even in 2004 a need for intensified learning and assessment of dental students’ communication competence.17 To support student development of professional communication competence as a means to obtain SDM, there is a need to more clearly define what it means to be proficient in this area.18 The aim of this study was thus to operationalize communicative and relational skills in a comprehensive assessment instrument for SDM.
Shared Decision Making
SDM entails collaborative discussion about treatment options between dentist and patient and is a crucial component of patient-centered care.1 SDM is based on interpersonal communication consisting of four important skills: giving information, acquiring information from the patient, involving the patient in decision making, and building a relationship with the patient.19 In SDM, the dentist and the patient interact and share all stages of the decision making process and information flows in both directions.20 This study is based on Charles et al.’s model in which SDM involves at least two participants sharing information with each other.20 In this model, both parties take steps to participate in the process of decision making by expressing and discussing treatment preferences. The process ends with a decision with which both parties agree.
As SDM and its value have been known for the last few decades, one could anticipate that it would be implemented by now. However, a systematic review of 33 studies assessing SDM indicates that this is not the case.21 For patients to be involved in decision making, it is imperative that they receive the information that allows them to participate,1 such as all relevant options including assessment of risk and benefit.22 However, Vernazza et al. found that the scope of information was restricted by the dentist’s assessment of clinical and nonclinical patient attributes, the latter including the patient’s motivation for dental treatment and financial issues.23 Direct user charges, common in many health systems worldwide, make SDM in dentistry particularly complicated since the cost of treatment may influence the options presented by the dentist as well as affect the patient’s decision making.12,23 Dentists also have the responsibility to acquire information from patients and are urged to acknowledge the information given by the patient and discuss its value and accuracy.7,24
When patients are involved in treatment decisions, they can understand the probabilities and their relationship to costs, which leads to more successful patient treatment outcomes.3 As most dentists practice dentistry in solo dental practices or clinics, the responsibility for decisions is usually not shared with other professionals.12,25 The decisions should be based on the best available scientific evidence and dentist expertise as well as the values and preferences of the patient and dentist alike.26,27 Due to cost of treatment and dental care being elective, in combination with patients’ sometimes acting like untrusting consumers, dentists may find themselves in a situation of conflict between professional duty and business practice.12,23
Impediments for achieving SDM have been identified. First, it has been perceived as more time-consuming than paternalistic decision making, and time pressure was found to be the most frequent perceived barrier for implementing SDM.4 Second, SDM was found to alter the power relationship between dentist and patient.28 Azarpazhooh et al. reported that the majority of patients valued collaborative participation in deciding treatment,4 while Elwyn et al. found that some patients thought (and disliked) that the doctor was insecure.29 Third, dentists may lack the skills for SDM and for giving relevant and balanced information on risks.1 Studies found that dentists’ attitudes toward SDM need improvement and that dentists struggle to adapt to this new professional role.11,12
Patients’ evaluation of medical care may depend more on the affective-relational dimension of communication than the medical-technical.13 Increased focus on relational competence may reduce the length of hospital stay and increase perceived quality of service.13,30 Patients have been found to seek relationships with professionals in which they find trust, autonomy, expertise, and caring.18 Nevertheless, building a relationship with patients is frequently forgotten or taken for granted, and Silverman et al. divided the skill to build relationships into three parts, all closely connected to communication competence: paying attention to non-verbal communication, developing rapport, and involving the patient.31
Non-verbal communication (including both communicating and interpreting non-verbal communication) was found in one study to be significant for the construction of empathy and has both direct and indirect effects on patients’ health outcomes.32 Acceptance of the patient’s views and emotions without judgment may be key to developing rapport.31 Techniques for involving the patient in the process, such as asking for the patient’s opinions and using supportive communication, help to legitimize the patient’s perspective, thereby facilitating SDM.33,34
Materials and Methods
As most classroom research in Sweden does not meet the definition of human subjects research, an approval from the Institutional Review Board of Malmo University, Malmo, Sweden, was not required for this study. This study sought to design and evaluate an assessment instrument for SDM that can be used to assess students’ progress for both summative and formative purposes.
Instrument Design
To date, there are no instruments specifically designed for assessing SDM in dentistry. Five instruments for assessing more general communication competence developed and/or evaluated in dentistry and dental education were therefore used as a starting point, and relevant items on information-sharing and relationship-building were identified.19,35–38 As none of these items focused directly on shared decisions, items from relevant instruments in the area of medicine were also included, in order to obtain a comprehensive list of items relevant for SDM.
The skills for information exchange included providing information, ensuring the patient’s contribution of information, and checking that the other party has understood the information provided. Skills for SDM included negotiating a treatment plan while showing respect and acknowledging mutuality, as well as providing closure. These two elements (information exchange and SDM) are sequential. Since communication is essential for SDM, global (i.e., necessary during the entire consultation) communication skills were identified as a third aspect, including using appropriate language (avoiding jargon), listening, and picking up verbal cues. Doctors’ interrupting their patients, resulting in a failure to disclose important concerns, has been identified as a problem.39 Therefore, the author decided to also include non-interruption as a listening skill. The fourth and final aspect consists of skills for building relationships: appropriate non-verbal communication, picking up non-verbal cues, and creating an open atmosphere in which respect, empathy, and acceptance are shown (Table 1). Relevant indicators were formulated for assessing these skills. The intention was to stress the equal importance of the dentist’s and the patient’s view, while still focusing on the dentist’s responsibility for involving the patient.
Aspects/skills, indicators, and scale of assessment instrument for shared decision making
Two dentists who were dental educators reviewed a first draft of the instrument and provided feedback on usability and face validity. This process resulted in two items being modified for increased clarity. All the instruments consulted used ordinal scales ranging from two to seven levels.19,35–38 The scales either focused on perceived quality or appearance. In this study, the author decided to use a scale with three levels with response options from inadequate to good. The use of three levels was based on the fact that communication is a complex task, thus resulting in a range of possible actions. Initially there is a need for reaching a consensus about what is acceptable for a professional dentist in various circumstances, while scoring criteria and refinements of the scale can be pursued in future research.
Communication competence consists of three types of communication skills: content, process, and perceptual skills.31 Competence is defined as encompassing knowledge, skills, and attitudes that need to be integrated and utilized appropriately.40 Content skills define what is communicated and process skills how the communication is carried out. Perceptual skills refer to how the communicating parties are handling and displaying what they are thinking and feeling based on their attitudes and beliefs, as well as how they display their attentiveness of feelings and thoughts of the other party. It should be noted that these three types of skills were identified for analytical purposes, so that the teaching and assessment of these skills can be more specific and efficient. It is not suggested that they should be taught or assessed in isolation from each other. The items in the instrument were categorized according to the three types of skills (Table 1).
Evaluating the Instrument
In this study, focus was on evidence based on test content, internal structure, and response processes. Furthermore, evidence from three sources of validity is presented: traditional content validity, expert review, and pilot testing. The statistical analyses provided evidence based on internal structure and response processes.
Data from workshops on training of communication skills and SDM with dental students were used to pilot the instrument. These workshops were performed in years three and five of dental education at the Faculty of Odontology at Malmö University during the spring semester in 201141 and consisted of role-play situations with a simulated patient, in which student consent was given by completing the course evaluation form. The case used was a situation in which the dentist and the patient had different opinions on a treatment plan. The workshops were videotaped, resulting in 29 role-play performances. The communication in the recordings was transcribed to avoid having the raters recognize the students, and ten of the 29 transcripts were randomly selected for this study. Items 13 and 14 were not part of this evaluation since they include non-verbal communication not visible in the transcripts.
Three raters, all dentists who were dental educators at the Faculty of Odontology at Malmö University, who had not been involved in the role-play situations or in the development of the instrument, were asked to test the instrument. The instrument was briefly introduced to the raters (15 minutes) by explaining the items. Each rater then assessed all ten transcripts individually and marked their ratings in an assessment sheet, resulting in three assessment sheets for each of the ten performances.
Validity evidence based on internal structure was provided by performing a principal component analysis (PCA), in which Quartimax rotation was used to produce the minimum number of factors to explain the variance. Results from the Kaiser-Meyer-Olkin measure for sampling adequacy and Bartlett’s test of sphericity indicated the adequacy of the analysis. Cronbach’s alpha was calculated to estimate the degree of homogeneity of each grouping obtained in the analysis.
In all situations in which raters assessed the degree to which the observed behavior corresponded to certain levels of a competence, skill, or other construct of interest, some subjectivity would be present in the ratings. Thus, an estimate of interrater reliability was important, as it had implications for the validity. Validity evidence was therefore provided by consensus estimates using percent agreement. If raters assessed a behavior with a high degree of agreement, it was considered an indication of the raters’ sharing a common interpretation of the construct. As the scale had only four levels, there was a risk for percent-agreement being artificially inflated, due to the fact that most observations fell into a single category. Thus, an additional consensus estimate of interrater reliability was computed for each item, using Cohen’s weighted kappa statistic.
Consistency estimates using the Spearman’s rank coefficient were computed to evaluate the raters’ consistency in scoring the items. It is not necessary for two raters to agree on scores in absolute terms, but it is important that each rater scores consistently for the instrument to be reliable.42 For the comparison of ratings across all three raters, Cronbach’s alpha was calculated for each item. For an overview of the differences among the raters, a calculation of average rating for each item by the three raters for all ten transcripts was made.
Results
Validity evidence was provided by an extensive literature review of previous research and existing measures of communication and SDM, thereby warranting that all items in the instrument were theoretically connected to the latent variable of SDM. (The list of these sources is available from the author.) The three raters evaluating the instrument found the instrument and the items usable and valid, which further added to the validity.
PCA was performed to provide evidence for the internal structure of the instrument: whether the aspects and items identified related to the concept being measured. Prior to the analysis, the suitability for using PCA was evaluated. Table 2 shows a three-component solution explaining 69.7% of the total variance.
Principal component analysis of the instrument
Cronbach’s alphas (internal consistency) of the factors were as follows: communication 0.912; closure 0.921; and perceptual 0.764. All Cronbach’s alphas exceeded 0.7, which is considered acceptable for research purposes.43 The alpha values for factors one and two were both above 0.9, which indicated a high degree of homogeneity. The instrument was therefore considered to have a satisfactorily high internal consistency.
Consensus estimates (Table 3) were computed to evaluate the level of agreement among the three raters. The results using percent agreement varied between 40% and 100%. A score of 70 or greater was considered as a quality guideline.42 When the mean percent consensus was calculated for the three groups, items 2–8, 11, 15, and 16 reached a level of 70% or greater. Landis and Koch suggested that kappa values of 0.41–0.60 are moderate, while values above 0.60 are substantial.44
Consensus estimates of interrater reliability using percent agreement and Cohen’s weighted kappa, and consistency estimates using Spearman’s rho and Cronbach’s alpha
According to Jonsson and Svingby, consensus agreement usually increases if the rubrics are complemented with exemplars and/or training of the raters.45 In this study, however, no exemplars were used. Instead, the raters were only briefly introduced to the items in the instrument. Increased training of the raters could result in higher consensus estimates, but a disadvantage of excessive training of raters is that it reduces the statistical independence of the assessments and may thereby affect the validity of the resulting scores.46
Consensus estimates of interrater reliability are based on the assumption that raters share a common view of the appropriate performance for a certain indicator in relation to a level on a rating scale. Consistency estimates, on the other hand, are based on the assumption that raters do not need this shared view, as long as each rater scores consistently according to his or her view. In this study, the consensus estimate varied for the items. The calculation of mean statistics (Table 4) for each rater for all items and ten transcripts indicated that rater 1 assigned somewhat lower scores overall, while the mean scores of raters 2 and 3 were fairly similar. Thus, it is relevant to evaluate the consistency estimates.
Calculation of mean statistics for each rater for all items and all ten transcripts
The calculation of Spearman’s rank coefficients (Table 3) indicated that most items had rs values of 0.70 or higher (p<0.05), which is considered acceptable.47 Cronbach’s alpha is an indication of the internal consistency, and the findings suggested that the three raters in this study were addressing a commonly shared construct.
Discussion
The aim of this study was to develop and evaluate an instrument for assessing SDM to be used in dentistry. The resulting Assessment of Shared Decision Making (ASDM) instrument consists of 18 items addressing various aspects of the construct (information exchange, SDM, global communication skills, and building relationship skills), as well as three types of skills (content, process, and perceptual). For each item, there are also indicators and a scale in three levels.
The fact that several items loaded on more than one factor indicated that one construct was being measured. The three components are facets of this construct. Furthermore, estimations of interrater agreement and consistency, as well as internal consistency, were made to provide additional evidence of validity. All items in the ASDM were relevant for assessing SDM. However, it could be suggested that the wording of item 5 is too complex, resulting in difficulties in assessing performance. A Spearman’s rank-order correlation analysis was run to determine the relationship between the scorings of items 5 and 16 as they both related to respect. There was a strong positive correlation between the two items that was statistically significant (rs=0.626, p<0.001). This result could indicate that item 5 is redundant.
The importance of developing the ASDM is primarily for formative assessment purposes, although it may also be used for summative purposes. Once identified, the aspects and skills essential for SDM can be woven into the curriculum and shared with the students. The ASDM can then be used for tracking students’ progress, providing feedback, and forming a basis for self-monitoring and self-evaluation. Thus, the ASDM provides a structure that can meet the need for intensified learning and assessment of dental students’ communication competence.17,18
Furthermore, communication can be assessed from three perspectives: 1) content, reflecting students’ knowledge in a specific context; 2) process, ensuring the skills to provide and receive information, as well as negotiating a treatment plan; and 3) the perceptual dimension, which mirrors the attitude of the student and his or her ability to create an atmosphere of mutuality, respect, and empathy. This partition into types of skills can be a way to identify and communicate a need for improvement and further learning to the students. It may also facilitate the amalgamation of communication skills and relational competence. In this study, the perceptual skills seemed to be the most difficult for the raters to agree on since both consensus and consistency estimates were relatively low for these items. This finding may indicate a need for an increased focus on relational skills, which could be facilitated by using the ASDM instrument.
The use of PCA as validity evidence based on internal structure is not optimal and is a limitation of the study. The small data set does not meet the guidelines with regard to sample size as well as the ratio of subjects to variables.48 However, as part of a pilot study, the author deemed it interesting to obtain a first estimate before applying the instrument to larger data sets. For future research, exploratory factor analyses (EFA) could be used as validity evidence instead, since PCA does not discriminate between shared and unique variance.
One of the major limitations of the current version of the ASDM instrument is that it uses a scale without any descriptions of the levels of quality. This means that, for instance, students and educators may interpret the levels inadequate, adequate, and good differently. A natural progression for developing the instrument is therefore to provide qualitative descriptors for the levels of quality. There are several methods that may be used for such an endeavor, such as the Delphi technique, in which experts’ opinions are collected and processed in iterative cycles to reach a consensus decision.49 Another possibility is to use the repertory grid technique, which is a method for eliciting personal constructs. This method has been used for putting tacit knowledge into words and formulating assessment criteria in aesthetic learning,50 which is somewhat similar to formulating levels of quality in the ASDM instrument.
Conclusion
Taken together, the findings of this study suggest that the ASDM represents a valid measure of SDM with three major components: communication, closure, and perception. With the exception of item 11, reliability estimates were satisfactory and could probably be increased further by training of the raters.
Footnotes
Disclosure
The author reported no conflicts of interest.