Die Etablierung und Evaluation einer zahnärztlich-chirurgischen OSCE-Prüfung mit sechs Stationen in der ZMK-Klinik Göttingen - die Bewertung ärztlicher und studentischer Rater im Vergleich
The establishment and evaluation of an OSCE with six stations in dental surgery at the dental clinic Göttingen - comparing the evaluation of the teaching doctor raters and the student raters
von Sophie-Kristin Schwarzer
Datum der mündl. Prüfung:2017-12-11
Erschienen:2017-11-27
Betreuer:PD Dr. Sabine Sennhenn-Kirchner
Gutachter:PD Dr. Sabine Sennhenn-Kirchner
Gutachter:PD Dr. Anne Simmenroth-Nayda
Dateien
Name:Dissertation_Schwarzer_Sophie-Kristin.pdf
Size:1.22Mb
Format:PDF
Zusammenfassung
Englisch
Context: During the winter semester 2014/2015 the first formative OSCE in dental surgery took place at the Universitätsmedizin Göttingen in order to examine clinical skills as part of the dental surgery assessment. We constructed six stations, two of them were video-stations without primary raters. All stations were rated by both teaching doctor and dental student raters. The basic aim of this study is to evaluate the stations and to improve them to a summative level and further to assess the reliability of dental student rater compared to the teaching doctor raters. Methods: In July 2014 we started with a pilot-OSCE with five stations. The first formative OSCE half a year later, in October 2014, included those five stations: three practical dental stations (local anesthetic to extract a tooth, the tooth extraction itself and the intraoral seam) and two video-stations (explanatory meeting before tooth extraction and giving instructions about postoperative behaviour after mouth-antrum-connection). They were enlarged by a basic life support station. The former students that participated in the pilot-OSCE took part as dental student raters in this OSCE. To standardise the two rater groups they were trained in advanced by using sample videos and getting detailled instructions on how to use the tablet software and rating system. The students were rated by a combination of checklists and global rating. All data was captured with the help of SPSS no. 22. The interrater-reliability was analysed with intra-class-correlation and the checklists with an item-analysis. Results: The average difficulty of the OSCE was 0,7 and the selectivity ranged from 0,099 to 0,469, average value 0,256 with Cronbach´s alpha 0,575. The interrater agreement was over all stations high (ICC=0,888, average 0,678). The average difference between teaching doctor and dental student rating was -0,02 with standard deviation of 0,04. They also did not differ significantly in the other items at the other stations. The difference in checklist rating was 0,00 by standard deviation of 0,04 whereas in global rating it was 0,02 by standard deviation of 0,27. Both external ratings showed hardly any differences in average difficulty, but high and significant correlation. The OSCE was correlated the highest with a coefficient of 0,9, which is significant with p < 0,001. Discussion: In order to evaluate practical clinical skills we were able to show that dental students are just as reliable raters as teaching doctors. Using standardised and structured training before rating is a valuable tool to achieve high inter-rater reliability between the two groups of raters, which is therefore independent of the level of education. With this study we verify existing studies for medical examinations. Student raters are not allowed in summative examinations but can reliably rate dental students to pilot more stations on the way to a high stakes 12-station-OSCE without requiring high staff. From the results of the item-analysis we can improve the existing stations that are not all reliable yet and aim to have a summative OSCE in the near future.
Keywords: OSCE; dental surgery; student raters; teaching doctors; interrater-reliability; stations