The Learn PT Rubric: Student and ICE Preceptor Pilot Data
Purpose:
Opportunities for educational research in physical therapy abound. Recent increases in resources and collaborations are fostering the growth of effective educational research for the profession1. This stems from a need to ensure that a changing educational environment continues to effectively prepare students to become expert practitioners2.
Many tools, including the Clinical Performance Instrument3, are used to assess DPT student readiness for clinical practice, but assessment of student readiness and/or progress in clinically important domains prior to full-time clinical experiences is less common. The XXXX university has developed and implemented the simple and easy-to-use LEARN PT Rubric with the goal of assessing student progress from program entry to completion. This rubric includes four domains important for effective clinical practice: professionalism, communication, clinical reasoning4, and adaptability5, and can be used by students, faculty and clinicians to assess student progress in an efficient manner.
The purposes of this project include 1) to understand the feasibility of using the LEARN PT Rubric and 2) to collect pilot data. The goal is to continue to refine the LEARN PT Rubric and ultimately assess its psychometric properties.
Methods/Description:
This study was conducted at XXXX university. Student participants provided institutionally approved consent for the data to be used for research purposes. Data was collected as a part of the students’ integrated clinical experiences (ICE) for each cohort of students. Participants included students enrolled in the DPT program and their ICE preceptors since August 2018. The 2021 cohort data was collected at two time points, upon entrance to the DPT program (time point 1) and at the end of ICE I (semester 2 (fall) or time point 2). The 2020 cohort data was collected at one time point at the end of ICE III (semester 5 (fall) or timepoint 5). The LEARN PT rubric uses a visual analogue scale, on which, participants mark their self-assessment (student) or student assessment (ICE preceptor) in each of the 4 domains. Each domain is associated with a score measured from 0 to 125 mm (1 mm = 1 point). A score of 0 indicates that the scale was marked at the leftmost part of the scale (in the “beginning” category); a score of 125 indicates that the scale was marked at the rightmost part of the scale (in the “exemplary”) category. Descriptive analyses and bivariate (paired and unpaired) comparisons were used to analyze the pilot data.
Results/Outcomes:
Study participants included 114 students and their associated ICE preceptors. At time point 1 (entrance to the DPT program, n=58), students in the 2021 cohort rated themselves on average at 77.6 for the Professionalism domain, 64.3 for Communication, 37.9 for Critical Reasoning and 59.4 for Adaptability. Compared to these scores, this same cohort rated themselves at time point 2 (ICE I), on average, lower across the Professionalism (72.3 ± 20.7) and Communication (62.7 ± 20.2) domains and higher across the Critical Reasoning (43.8 ± 19.7) and Adaptability (61.3 ± 23.3) domains. The paired-samples t-test indicated that these differences were significant across the two time points (p< 0.001). Differences between student and ICE preceptor average scores in each of the four domains for the 2021 cohort at time point 2 (ICE I) were not significant (p > 0.05).
At time point 5 (ICE III), students in the 2020 cohort (n=56) rated themselves on average at 89.8 ± 17.8 for the Professionalism domain, 80.3 ± 17.9 for the Communication domain, 66.6 ± 18.1 for the Critical Reasoning domain, and 77.7 ± 19.1 for the Adaptability domain. Differences between student and ICE preceptor average scores in each of the four domains for this cohort at time point 5 (ICE III) were not significant (p > 0.05). However, an independent samples t-test indicated that students in the 2020 cohort, on average, rated themselves higher than the students in the 2021 cohort during the same semester (fall, ICE I vs. ICE III), and this difference was statistically significant (p< 0.001) for every domain with the greatest average increase in the Critical Reasoning domain (22.8-point difference). It was also noticed that the 2020 cohort data had smaller standard deviations for each domain compared to the 2021 cohort data.
Conclusions/Relevance to the conference theme:
This pilot study has demonstrated that both students and ICE preceptors were able to use the LEARN PT Rubric to assess the students’ performance in the Professionalism, Communication, Critical Reasoning and Adaptability domains. The pilot study also demonstrated that students in two, successive cohorts rated themselves significantly higher in each of the Professionalism, Communication, Critical Reasoning and Adaptability domains further along in the program. In addition, the ICE preceptors’ scores of the students do not differ significantly from the students’ self-assessment. The LEARN PT Rubric has been developed as a foundational element of an educational research agenda designed specifically to longitudinally assess student progress in four clinically important domains. Data collection will continue for these and incoming students at 7 different time points throughout the DPT program. Understanding student performance in these areas throughout the curriculum is useful for partially understanding student readiness for full-time clinical affiliations, and informing curricular modifications that can improve outcomes and student learning success.