| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Educational Methodologies |
Key words: educational research, preclinical dental training, 3D virtual reality, haptics-based dental simulator, faculty content validity study
Submitted for publication 05/18/07; accepted 09/25/07
 |
Abstract |
|---|
 Top Abstract Materials and MethodsResultsDiscussionConclusionsReferences |
|---|
Simulators have become a standard in aerospace and aviation1 and useful in a wide variety of medical fields for procedures such as laparoscopy,2 sigmoidoscopy,3 lung biopsy,4 neurosurgery,5 vascular access, and cardiovascular catheterizations.6 Besides providing detailed computerized, graphical renderings of anatomical sites, simulators recreate how injecting, cutting, inserting probes, and palpating tissues actually feel to the individual performing the procedure. A number of studies published in the area of medical simulators indicate that training on simulators significantly improved students’ skill and performance on patients when compared to those individuals not trained on these devices.2–5,7 Additionally, in a recent best evidence in medical education (BEME) systematic review on the effectiveness of learning by using medical simulators, it was noted that simulators facilitate trainee learning and complement clinical medical education.8
The majority of current dental simulators used in dental schools use realistic manikins along with dentiform models (KavoTM, Lake Zurich, IL; A-decTM, Newberg, OR; and NevinTM, Chicago, IL) incorporated into a simulated dental operatory.9 Dental schools are also employing more sophisticated simulators for training students,10,11 none of which use a 3D, VR, haptic-based system. A number of dental schools use the DentSimTM simulator (DenX Ltd., Jerusalem, Israel),12 a more sophisticated manikin simulator that incorporates computer-aided audiovisual simulations having a VR-based component. It uses a tracking system rather than haptics. Buchanan11 and Jasinevicius et al.13 performed controlled studies using the DentSim that showed the dental simulator aided in decreasing faculty time in instruction and facilitated student training of technical skills.
By adding a haptic component to dental simulators, greater sophistication in creating simulations of various procedures becomes possible. Tactile sensations are very important in all aspects of dental treatment, and incorporating these into a simulator should greatly enhance its usefulness. Haptic-based simulators require less initial financial investment, are more versatile, require less maintenance, do not require replacement of models or parts, and have a much wider range of applicability than do previously developed manikin-based simulators. Because they are computer-driven, it is feasible to monitor students’ progress and their frequency of use. Furthermore, the devices permit testing of student performance. These simulators are so new that well-controlled studies of their effectiveness are lacking. The dental simulators that have been developed are: the Iowa Dental Surgical Simulator (College of Dentistry, University of Iowa, Iowa City, IA),14 Novint Technologies (Novint, Albuquerque, NM),15 SimulifeTM Systems (Simulife SystemsTM, Paris, France),16 and PerioSim (Perio-Sim, University of Illinois at Chicago, College of Dentistry, Chicago, IL).17 Of this group, only a limited evaluation study of the Iowa Dental Surgical Simulator14 has been reported. This 2D system was found to provide good trainee responses.
The goal of our study was to use experienced clinical instructors to assess faculty perception of the realism of the first version of a haptic-based dental training simulator and provide future direction for its development. The simulator software was designed primarily to provide a self-instructional program in clinical periodontal probing for novice students in dentistry and dental hygiene. The research goal was to introduce a computerized training program having 3D, VR graphics, along with haptics ability to enhance instructional ability for specific periodontal procedures, such as periodontal probing. This content validity study is one in a series to establish "evidence-of-concept" that PerioSim can serve as a self-directed learning tool enabling students to master basic periodontal techniques determined by clinical instructors.
|
Materials and Methods |
|---|
|
TopAbstractMaterials and MethodsResultsDiscussionConclusionsReferences |
|---|
) used two computer monitors with the haptic device placed between them. The monitor on the left (the simulator screen) provided a view of 3D, VR, upper and lower, human dental arches. Crystal Eyes Stereo GlassesTM and a Crystal Eyes WorkstationTM (Stereo Graphics Corp.TM, San Rafael, CA) were used for 3D viewing. The primary component of the dental simulator is a PHANToMTM haptic device from SensAble TechnologiesTM (SensAble Technologies, MA) with 3-degrees of freedom. This force-feedback device permitted implementation of touch-based 3D representation through user movements of a small robot-like arm containing a handheld stylus. The PHANToM provides tactile feedback with respect to tooth surface and gingival tissue.
|
By operating the haptic device, an onscreen 3D, VR William’s periodontal probe (Hu-FriedyTM, Chicago, IL) or periodontal explorer (Hu-Friedy), was manipulated to sense contact with the crown or root surface of the tooth and its surrounding gingival tissues. The sensation of contact was obtained from the PHANToM stylus.
In this first version of the software, a control panel on the simulator monitor permitted limited control of several parameters (e.g., gingival and other model transparency, haptic fidelity, template creation, and instrument choice). Movements of the operator-held stylus positioned the instruments and model. The software that enabled the use of the haptic device was a modified version of GHOSTTM (SensAble Technologies, MA).
Evaluation Methods
The UIC Institutional Review Board reviewed and granted permission before beginning this study. Thirty male and female experienced dental and dental hygiene instructors were asked to volunteer to assess the simulator. All agreed to participate. The instructors were all experienced faculty of the COD from a variety of clinical areas recruited by the lead author (ADS) and the fourth author (SA). As previously described, a PowerPoint presentation on one monitor screen provided instructions for using and evaluating the simulator, while the second monitor contained the simulator program. Each subject used the simulator individually with only ADS or SA present. The subjects were individually guided through a series of onscreen tasks that taught them to use the simulator and adjust to 3D, VR stereoscopic viewing and haptic onscreen feel. When they felt competent to handle the simulator, they reported this to the investigator who was present and able to observe that this was so.
The next step in the process involved the use of a questionnaire prepared by PGB, who has experience in writing questions for and assessment of the educational validity of programs used in medical education. ADS worked with PGB to ensure that the questions were relevant to the dental procedures being assessed. These were then further evaluated and edited by the other authors.
Only the individual test subject and individual investigators (ADS or SA) were present during simulator training period and actual simulator assessment time. Both the individual test subject and individual investigators (ADS or SA) wore stereoscopic goggles to observe 3D effects during short initial simulator training and the actual simulator assessment period.
A total of thirty-six questions were used, all but five of which were assessed using a 1 through 7 value indicator. The five questions not using the 1–7 scale involved specific data on the subjects: start time, end time, dental specialty, years in dental practice, and number of hours each week playing computer video games. The subjects did not see the questionnaire until the time of their assessment period. Each subject recorded his or her value determination for each question in complete privacy. Even the test administrator was limited in his or her ability to witness the responses.
The entire questionnaire was divided into five segments, shown in Tables 1–5. Each of the required activities of the five segments was performed on the simulator. This was immediately followed by the test subject grading on the 1–7 scale his or her assessment of the realism, etc. of each activity. For example, subjects examined the graphics shown on the screen then evaluated their realism (Table 1).
|
|
|
|
|
Subjects used an arbitrary seven-point rating scale with 7 on the scale defined as "very realistic" and 1 as "not realistic." A mean rating of greater than 5 on a seven-point realism or usefulness scale was set as the threshold for judging whether the rated object or action was realistic or useful. Since the content validity was demonstrated by experienced instructors who have expertise in judging PerioSim content, the content and simulation processes used are assumed to be realistic enough for teaching and evaluation. While the scales and selecting a score greater than 5 as being significant are arbitrary, it assumes that "real" be at least 5/7 better than chance or a neutral rating. Thus, a rating greater than 5 mean rating was selected because it would require the trend of responses to be significantly greater than the neutral point. Additionally, Alpha stats were used to determine if the raters/judges were using a single underlying premise/description of "real" and "use in teaching."
Data Analysis
Using SPSS® (Chicago, IL) statistical modules, subject means, confidence intervals, and statistical comparisons for all ratings were determined. Significant differences in ratings were determined by a one sample two-tailed t-test, with expected value of >5.00 (Tables 1
, 2, and 3). The Spearman’s rho (
) statistic was used to correlate subject’s realism ratings and expected usefulness of the simulator with background and experiences with computers (Tables 4 and 5). Statistical significance was established at p<0.05 for both analyses. The internal consistency (reliability) of the ratings in each scale set was calculated using the alpha (
) statistic.18 The statistic had a range of 0.00 to 1.00, and a value over 0.80 is considered highly internally consistent. The rater is using a single underlying principle or concept in making judgments.
|
Results |
|---|
|
TopAbstractMaterials and MethodsResultsDiscussionConclusionsReferences |
|---|
Table 1 reports the realism ratings for anatomical models and instruments simulated by PerioSim. Ratings above 5.00 are considered realistic, and ratings below 5.00 are considered not realistic. The overall teeth image (p<0.001) was an equally weighted combined rating of the separate ratings for crowns (p<0.001), roots (p<0.001), and upper and lower dental arch images (p<0.001). All of the teeth images, individually and combined, were significantly greater than 5.00. The gingival image overall rating (not significant) was below the 5.00 realism threshold. It was comprised of ratings for gingival contours (p<0.001), which was significantly less than 5.00 (not realistic), and gingival colors (not significant). The William’s periodontal probe image was rated realistic (p<0.01) as were the probe’s mm markings (p<0.001). The periodontal explorer also was rated very realistic (p<0.001). All ratings of instruments were significantly greater than 5.00.
Table 2 shows the ratings when using the haptic device in PerioSim to manipulate the periodontal instruments to feel the teeth and gingival tissue. Consistent with the realism rating for teeth images, instructors rated as realistic the feel of the teeth overall (p<0.001), feel of the crowns (p<0.01), crown contours (p<0.001), and crown anatomy (p<0.001). Instructors rated use of the periodontal probe and explorer (neither statistically significant) slightly lower for probing the gingival crevice or pockets. In contrast to the high ratings for feel of teeth, the overall feel of the gingiva (p<0.05), the gingival touch (p<0.001), and anatomy (p<0.05) were statistically significantly lower than the 5.00 realism threshold. The ratings of gingival contours were not found to be statistically significantly lower than the 5.00 realism threshold.
Table 3 reports the instructors’ ratings concerning value of the simulator instructions and the instructors’ technical background. Instructors reported that the simulator instructions and practice with the haptic device were very useful (p<0.001). Also very useful was the instructional format that employed a PowerPoint slide format (p<0.001). Comfort using the simulator was above the expected 5.00 threshold as was confidence in using computers, but neither was significantly greater than 5.00. The instructors had little experience using simulators and very limited experience with haptic-based simulators. Both of these groups were found to be statistically significantly lower than the 5.00 realism threshold. Two instructors reported using computer games on a regular basis, suggesting they were experienced computer users.
Results in Table 4 provide data about practice years as a surrogate measure of age and how practice years relates to confidence in computer use, comfort in using PerioSim, and use of the simulator for teaching. Years in practice was inversely correlated with confidence using computers (p<0.01). The correlations were near zero between practice years and comfort using any simulator ( =–0.11, ns) or use of the simulator for teaching ( =–0.07, ns). These findings suggest age of instructors and computer expertise are not barriers for instructors to use PerioSim for teaching.
Table 5 addresses the questions with respect to comfort with the simulator related to how useful the simulator would be for students to learn dental skills, how to assess student’s dental skills, how useful the simulator is for teaching, and how to tailor the simulator instructions for training. Instructors’ comfort with the simulator was highly correlated with students’ learning dental skills (p<0.001) and usefulness in teaching, but less so for assessing dental skills (p<0.05) and tailoring instructions for training (p<0.05). These findings suggest that those who are more comfortable with the simulator perceive PerioSim as potentially more useful for its intended purposes.
The alpha statistic measures the internal consistency or reliability of ratings.18 A higher alpha (0.80 or greater) means there is a significant probability that instructors are using one factor or latent variable when they judge the realism or usefulness of the simulation. Separate alpha statistics were calculated for ratings that logically cluster together. The eight items for rating images of teeth, gingival, and periodontal instruments (Table 1) had an alpha of 0.85, and the ten items for feel of the VR simulator (Table 2) had an alpha of 0.89. Ratings about the instructions for using the simulator (items in Table 3) were highly reliable (alpha=0.90) as were the four items in Table 5 for judging the potential use of the simulator for teaching and assessment (alpha=0.91).
|
Discussion |
|---|
|
TopAbstractMaterials and MethodsResultsDiscussionConclusionsReferences |
|---|
This study used thirty clinical instructors in various dental specialty areas to evaluate the reality of the various components of this system. The mean time required for both training in the use of the simulator and evaluation of the program took approximately thirty-five minutes. Most instructors had moderate confidence in using computers and comfort using the simulator. Few had experience with simulators, with even fewer having experiences with haptic devices. The ratings observed in Table 3 indicate that the overall simulator instructions and instructional format were very good.
The realism of the 3-D visual images of the teeth overall, the crowns, roots, and dental arches (Table 1) were all found to be high (p<0.001), while the gingiva overall, the gingival colors, and contours were not realistic. The realism of the William’s periodontal probe image and the ability to read its mm markings within and outside of the periodontal pocket or crevice were found to be significant (p<0.01, p<0.001). A second instrument, the periodontal explorer, instructors found to be very similar in image to the clinically used instrument. All these judgments were highly reliable (internally consistent) based upon the -statistic, suggesting that the faculty were making judgments from a single basis as intended by the questionnaire for questions about "how real was the simulator?" or "how useful was the simulator for the intended purpose?"
The periodontal probe and/or periodontal explorer were used to evaluate the feel of the crowns of the teeth and gingiva, while the periodontal probe was used primarily to probe the gingival crevicular or pocket areas. Instructors reported a distinct difference in feel between the crowns and the gingiva (Table 2) and in some cases reported probe sticking in the crevice or pocket area (individual instructor observations). The crowns had a very realistic clinical feel (p<0.001), while the consistency of the gingival soft tissues was significantly less than 5 on a seven-point realism scale. Some of these differences, in part, may be due to graphical defects at the interface of different objects (crowns, gingiva, roots) in the original commercial model. These defects created minor spacing between objects (crowns/roots/gingiva), occasionally causing the probe to stick at the crevice or pocket/object interface. This problem possibly was consistent enough to produce the wide variance and contributed to the lack of realism in the gingival appearance and feel. However, the periodontal probe sticking problems did not prevent the instructors’ ability to read accurately the probe’s mm markings in the pockets or crevices.
The graphical defects may also contribute to the difference in stiffness or deformability of the two types of tissues. Tooth structure is stiff and not deformable, while gingiva is more deformable. Efficient collision detection algorithms for nondeformable models, such as teeth, exist and are being used. Collision detection algorithms between highly deformable models, such as soft tissues, have not yet been satisfactorily developed.20 Gingival tissues appear to fit between the nondeformable model and deformable models since gingiva is closely adapted to the underlying bone, providing more stiffness than just soft tissue. Thus, further research is required to achieve a functional solution to develop an algorithm for the gingival tissue problem.
In Table 4, correlating the years in practice, confidence using computers, and comfort using the simulator showed that while most instructors were not confident in using computers, everyone was comfortable using the simulator, no matter how many years they had been in dental practice. Furthermore, this study found that any age practitioner could use the simulator, and instructors indicated the simulator would be useful in training dental skills and assessing teaching. Especially important for future adoption in teaching is the instructors’ reporting they could tailor the simulator instructions to the way they teach.
This study provided an important service. It generated clinical instructor feedback on what worked well in the simulator and focused on some of the simulator flaws. An interesting overall observation of the study participants was that although the trainee’s hand was on the stylus off-screen, the brain appeared to react to the 3D model on screen. This may be due to the simulator’s tactile feedback to the stylus in which the user reacted as though movements and sensations involved when manipulating the on-screen instruments were real—as if he or she was performing the task on an actual mouth.
This study was limited by certain necessary design constraints. The questions used were relevant to the authors and were admittedly subjective, but nevertheless revealed important information. Furthermore, the subjects used were all clinical instructors. In the future, students should be asked to assess the simulator and comparisons made with students receiving similar training in a traditional manner. This is a good beginning to elucidate a complex problem, but further work will be required to achieve a complete evaluation of the PerioSim.
One major limitation in the use of the haptic-base simulator program was the sticking or catching of the periodontal probe instrument tip in the sulcular area. Additionally, the current haptic instrument only has three degrees of freedom. This means that only the tip of an instrument has haptic feel. The lateral aspects of the instruments do not have haptic properties. This results in a lack of collision detection with the lateral aspect of the instrument, moving in and through whatever object it contacted (e.g., teeth, gingiva). This particular problem is now under investigation and may be resolved with the use of a six degrees of freedom haptic unit along with the creation of new algorithms. In spite of these drawbacks, the thirty instructors accommodated easily to the use of the program and believed it would be quite useful for student training along with unlimited practice.
While this study focuses on the haptics aspect of the simulator, it must be remembered that 3D visualization is a very important component of the simulation process. Each of these senses—visualization and feel—complement and, in fact, enhance each other. This most likely is one reason for the success thus far attained. Even though only the tip of the periodontal explorer and periodontal probe have a haptic component, having a 3-D VR graphical rendering with them results in successful simulation documentation. In the process of learning and especially training, sight, touch, and sound all appear to be preeminent.20 Thus, the presence of sight and touch in this study complemented each other. Sound is being added to future versions of this software to enhance the overall effect of the simulator’s use.
The haptic-based VR dental simulator being developed at UIC has the potential to lead to radical changes in the training of dental students, dental hygiene students, and practicing professionals. This study suggests that the use of high-fidelity simulation programs may help ameliorate some of the problems facing dental education. The haptic-based, VR, periodontal probing simulation program is in its early development and continues to evolve. Future systems being considered for using PerioSim for training and evaluation include evaluation and standardization of periodontal probing (which will include determining examiner probe pressure applied to gingival pocket area), the evaluation of the sub-gingival root surface, the therapeutic process of root planing and scaling, the diagnosis of active or inactive white spot lesions, the evaluation of restorative procedures, and the diagnosis of lymph node involvement in a variety of oral pathologies. In summary, as we explore how PerioSim can be used, it is hoped that this simulator system may prove to provide training and evaluation of a variety of dental clinical therapeutic skills.
|
Conclusions |
|---|
|
TopAbstractMaterials and MethodsResultsDiscussionConclusionsReferences |
|---|
|
Acknowledgments |
|---|
|
Footnotes |
|---|
This project was supported by a grant from the University of Illinois at Chicago College of Dentistry.
|
REFERENCES |
|---|
|
TopAbstractMaterials and MethodsResultsDiscussionConclusionsReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  J. Phillips and Z. L. Berge Second Life for Dental Education J Dent Educ., November 1, 2009; 73(11): 1260 - 1264. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
