Multi-Rater Assessment of Student Communication Skills Using the Gap-Kalamazoo in a Simulated Encounter
Purpose
Effective communication is a pre-requisite for successful patient encounters however physical therapy (PT) students may be unaware of “where they are at” with regards to their own performance prior to full-time clinical experiences. The Gap-Kalamazoo Communication Skills Assessment Form (GKCSAF) is used by other health professions to assess student ability to demonstrate attributes of patient-centered communication. The GKCSAF was implemented in comprehensive standardized patient (SP) examinations with 1st and 2nd year DPT students to promote and assess readiness for upcoming full-time clinical experiences. The purposes of this study were to analyze and compare responses by type of rater, and to explore student and CI perceptions of the use of the GKCSAF.
Methods/Description
Following the simulated encounters, CIs and SPs rated students, and students self-rated using the GKCSAF. For each class, mean scores of the 9-items in the communication checklist and an overall average were calculated for each of the 3 types of raters. One-way ANOVAs were used to determine differences in means for the 9 individual communication skills and the overall average score among the 3 rater types. Where ANOVA omnibus tests were significant, paired t-tests were used to determine differences by rater type. Alpha was set at .005 to correct for multiple comparisons. Students and CIs completed brief surveys to share their perceptions of the tool.
Results/Outcomes
Mean ratings by all 3 types of raters for first-year students ranged between 3.4 and 4.4 on a Likert scale of 1(poor) to 5 (excellent). Ratings by SPs were significantly higher than ratings by students and/or CIs for 6 of the 9 communication skill items and for overall average scores. Mean ratings by all 3 types of raters for second-year students ranged between 3.2 and 4.2. Ratings by SPs were significantly higher than ratings by students and/or CIs for 4 individual communication skills and for overall average scores. There were no significant differences between mean student and CI ratings within the first- or second-year classes.All respondents agreed the skills were important to practice and students valued the quantitative and the narrative feedback.
Conclusions/Relevance to the conference theme: The Pursuit of Excellence in Physical Therapy Education
The GKCSAF was successfully implemented in a PT standardized patient examination to provide multi-rater feedback on student communication skills. Mean student self-ratings were not significantly different than those of the CIs demonstrating accurate self-assessments overall. SP ratings tended to be higher across all areas. It is unknown whether this discrepancy was due to differences in SP expectations or to training in use of the tool. This presentation will provide educators with information on one PT program’s novel experience with use of the GKCSAF tool to assess student outcomes in the development of essential communication skills to promote readiness for upcoming full-time clinical experiences. Implications for use in the classroom and clinic will be discussed.
References
Peterson EB, Calhoun AW, Rider E (2014) The reliability of a Gap-Kalamazoo consensus statement checklist for assessing the communication skills of multidisciplinary clinicians in the simulated environment. Patient Educ. Couns, http://dx.doi.org/
Calhoun AW, Rider EA, Meyer EC, Lamiani G, Truog RD (2009). Assessment of communication skills and self-appraisal in the simulated environment:feasibility of multi-rater feedback with gap analysis.Simulation in Healthcare,4 (1), 22-9.