Delayed Video Skills Assessment: Pilot Study in a Doctor of Physical Therapy Course
Background and Purpose: Currently there is no consensus on the most appropriate method to evaluate student acquisition of psychomotor skills prior to clinical experiences or comparisons of the impact of assessment on student performance. This pilot study describes impact of delayed video assessment of psychomotor skills on DPT student perception and lab grades.
Case Description: 45 students enrolled in a 2nd year biophysical agents course consented to complete an anonymous web based 8-item survey describing their preference for delayed video and in-person psychomotor skills assessment. After the first clinical experience a 3-item follow up survey was administered to determine any change in perception. Prior to assessment, faculty demonstrated the skills face to face in class, students practiced and were provided feedback. The delayed video assessment involved application of moist hot pack, ultrasound and traction while the in-person assessment allowed observation of electrical stimulation application. A performance-based checklist rubric was utilized to assess the student's performance for both assessment methods. Four faculty members graded the delayed video and in-person performance of psychomotor skills and discussed any grading concerns coming to a mutual agreement to maintain consistency.
Outcomes: The mean score for the delayed video assessments (99.6 ± 0.71) were higher than the mean scores for the in-person (98.6 ± 3.4) skills assessment (p<0.001). 54% of the students had previously completed a video assignment in a college course but this was the first psychomotor skills assessment submitted via video in the DPT curriculum. 45% of the students felt that in-person assessments better prepared them for the clinic however 49% felt that delayed video assessment was the most appropriate way to assess their skills in the course. 71% of the students reported more anxiety during in-person compared to delayed video skills assessment. 85% of students recommended delayed video or a combination of assessment methods for future cohorts. The follow-up survey indicated no significant change in student preference for assessment method (p< 0.228, p<0.046, p<0.787). Faculty reported increased lab time allocated for student practice however there was increase faculty time required outside of class to grade the delayed video skills.
Discussion: The pilot study appears to have been successful with the students having a positive view of the delayed video skills assessment. Delayed video and in-person skills assessments appeared to be valuable from a student perspective even with student report of increased anxiety with in-person assessments. There was no negative impact on the students’ lab grades as the scores were consistent with previous cohorts. Future research should explore how assessment method impacts student performance of psychomotor skills while in the DPT program and during clinical experiences.