Reliability and Validity of the Apta Physical Therapist Student Evaluation: Clinical Experience and Clinical Instruction
Purpose:
One approach to enhancing the role of the clinical educator as a mentor to PT students is to measure clinical instructor effectiveness in exhibiting the characteristics and performing the duties of a clinical mentor. The most widely-distributed tool used by the physical therapy profession to assess clinical instructor effectiveness, the American Physical Therapy Association Physical Therapist Student Evaluation: Clinical Experience and Clinical Instruction (PTSE), has been used since 2003 without an assessment of its construct validity and with the only study of its reliability having been conducted on a narrow sample of students from one physical therapist education program.
The physical therapy profession is without a valid and reliable instrument to assess clinical instructor effectiveness. The purpose of this research was to assess the construct validity and reliability (internal consistency) of the PTSE.
Methods/Description:
A non-probability convenience sample of 5,128 students from 29 physical therapist education programs from eight of nine geographic regions in the United States completed 6,916 PTSEs in 2017 using a particular third-party clinical education database management system. This sample yielded one subset of data completed at the midpoint of a clinical education experience, n = 1788, and two subsets of PTSE data completed at the endpoint of a clinical education experience, n = 1788 and n = 3340.
To assess the construct validity of the PTSE, the researcher conducted exploratory factor analysis with principle axis factor extraction and promax oblique rotation on each of the three PTSE data subsets. To assess the reliability (internal consistency) of the PTSE, the researcher conducted Cronbach’s Alpha analyses on each data subset and on each factor identified in the factor analyses.
Results/Outcomes: For each PTSE subset, factor analysis identified one factor of 17 items, named Clinical Instructor Effectiveness. These factor analyses did not identify a factor solution that included all 21 items of Section Two, Question 22, of the PTSE. Resultant Cronbach’s alpha values met minimum levels of acceptable internal consistency while exceeding the 0.90 level that raises the concern of item redundancy.
Conclusions/Relevance to the conference theme:
Factor analyses and Cronbach’s alpha analyses of three subsets of PTSEs completed by students from a non-probability convenience sample of physical therapist education programs in the United States did not establish the American Physical Therapy Association’s Physical Therapist Student Assessment: Clinical Experience and Clinical Instruction as a valid instrument while identifying its reliability as suggestive of a redundancy of items.
Demonstrating the impact of innovations in developing clinical mentors is difficult without valid and reliable tools to measure behaviors consistent with mentorship. This results of this research provide a bridge to the future of clinical educator assessment in three ways. First, it lays the foundation for further development of a valid and reliable tool to measure clinical educator effectiveness. Second, it provides an example of use of a common database that approaches the concept of a national clinical education research database. Finally, it highlights the importance of evaluating innovations in developing clinical educators as mentors with evidence-based outcome measures.