The NPTE Research Agenda and How It Can Help Your Program
Purpose
This session is to inform educators of the research performed by the Federation of State Boards of Physical Therapy (FSBPT) in support of the National Physical Therapy Examination (NPTE) and in response to questions from educators. The presentation will cover questions commonly asked about the NPTE by educators, including the rationale for the eligibility changes, how to use school reports for program development, the rationale behind test security measures, and the usefulness of the Practice Exam and Assessment Tool (PEAT) for predicting success on the NPTE. The presenters will illustrate how NPTE research is organized through a research framework, from generating research questions to implementing and disseminating the findings, and how this results in better physical therapy education and public protection.
Methods and/or Description of Project
The NPTE Research Agenda is a plan used to conduct research to support the validity of the NPTE and related public protection outcomes. The Research Agenda supports the “NPTE Validity Framework,” which explicitly describes the relationship between the processes used to develop the NPTE and FSBPT's public protection goals.
The Research Agenda is a living document; topics are added in response to current and relevant public protection issues through various means. Often, topics are added because of questions asked about the NPTE by educators, licensing boards, and other stakeholders. For example, educators often ask if the scores from PEAT scores predict performance on the NPTE, and whether there was a "passing score" for PEAT. Over three years, FSBPT conducted the necessary work to link PEAT performance to NPTE performance in a reliable and valid way. Another way topics are added to the research agenda is through FSBPT's external advisors, the NPTE Technical Advisory Panel (TAP) and the Buros Center for Testing, which accredits the NPTE. The TAP consists of educators and testing professionals who advise FSBPT staff on ways to improve the NPTE and how to better demonstrate its validity as a public protection tool. Buros reviews FSBPT's testing processes and gives feedback for better demonstrating validity and fairness through meeting The Standards for Education and Psychological Testing (AERA, APA, & NCME, 2014).
The vast majority of NPTE research is conducted internally using extant data, typically linking information obtained during registration with examination data. FSBPT internal staff, including licensed physical therapists, psychometricians, and industrial psychologists, typically has sufficient expertise to plan, conduct, and interpret the research. In other cases, FSBPT relies on volunteers to provide additional expert judgments, or recruits external experts for an independent perspective. For large studies, FSBPT may conduct surveys of experts in the field or NPTE candidates.
Once the research is complete, internal staff reviews the findings and presents them to the TAP and to the FSBPT Board of Directors. Once the Board of Directors has approved the research findings, FSBPT disseminates them through the Educator’s Newsletter, FSBPT Forum, and various public meetings as appropriate.
A very common question from outside researchers is whether FSBPT will provide data about the NPTE. To assure the security of the examination and candidate confidentiality while minimizing legal complications, the NPTE data is almost never shared with outside collaborators.
Results/Outcomes
The NPTE Research Agenda has both periodic research projects and ad hoc projects. The two most important elements of the Research Agenda are the Practice Analysis and the Standard Setting. Both occur every five years. The Practice Analysis informs the educational and regulatory communities of tasks performed by entry level PTs and PTAs. Notably, the practice analysis identifies the knowledge and skills required to perform those tasks. FSBPT is currently conducting a practice analysis in 2016, which will be followed by a standard setting meeting to determine if the passing standard for the NPTE should be adjusted. Changes to the passing standard are always framed in the results of the Practice Analysis: what tasks, knowledge areas, and skills are required now that might not have been required previously.
FSBPT also schedules ad hoc elements in the NPTE Research Agenda. In 2015, FSBPT examined the skills required for PTs to perform dry needling activities in relation to the skills conferred by the NPTE. This report was well received by both educators looking to add dry needling to their curriculums and by regulators charged with determining if additional requirements should be implemented prior to performing this intervention.
In response to direct feedback from educators, FSBPT revised PEAT and introduced software changes to provide a more authentic assessment experience and robust performance report compared to the NPTE standard. On the basis of years worth of research, FSBPT was able to confidently predict how an examinee would have done on the NPTE after taking PEAT. FSBPT also examined the impact of fixed-date testing, finding that candidates were, on average, taking the NPTE about the same time after graduation as they did under the continuous testing model (about 80-90 days). Moreover, the study showed that candidates who had to wait a few more days to take the NPTE did not have lower passing rates than candidates who took the exam closer to graduation.
Over the last three years, FSBPT has also conducted a substantial amount of research on passing rates to address concerns that unqualified candidates were sitting for the NPTE. As a result, FSBPT imposed a six-time attempt limit for taking the NPTE, a ban for candidates who have consecutive non-legitimate attempts, an English proficiency requirement, and a requirement for non-CAPTE-educated candidates to have a standardized coursework evaluation.
Conclusions/Relevance to the conference theme: The Pursuit of Excellence in Physical Therapy Education
The goal of the NPTE Research Agenda is to provide additional evidence of the validity of the NPTE, which along with a relevant education, is the best way to ensure competence among licensed PTs and PTAs. Much, if not most, of the research conducted as part of the agenda is highly relevant to educators and other stakeholders. First and foremost, the Practice Analyses and Standard Setting provide excellent resources for schools trying to examine strengths and weaknesses of their programs. FSBPT also provides pass rate reports free of charge and purchasable reports that can provide additional insight into how this information can be linked to a school’s curriculum. Providing better NPTE preparation guidance through PEAT can help educators identify those students most in danger of failing to meet the competency standard and provide them with targeted remediation. That helps more qualified PTs and PTAs to enter the healthcare field quickly. Restricting examination eligibility and testing dates enhances test security, preventing incompetent and unscrupulous candidates from obtaining licenses.
Current elements of the Research Agenda include identifying characteristics that predict success on the NPTE and making revisions to the content outline for the NPTE based on the Practice Analysis. In 2017, FSBPT is planning to conduct a “gap analysis” to determine what areas of competency are not sufficiently measured by the NPTE and what could be done to address this deficiency. This might lead to such changes as introducing new question types to the NPTE or a different assessment altogether.
References
American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). The standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Mueller, L., & Layton, S. (2015). Changes are coming to NPTE Eligibility. https://www.fsbpt.org/Portals/0/documents/free-resources/2015FallForum_NPTEEligibility.pdf
Mueller, L., Wang, A., & Zhang, Y. (2015). How well does the PEAT predict success on the NPTE?
https://www.fsbpt.org/Portals/0/Content%20Manager/PDFs/Forum/Forum_Fall2014_PEAT.pdf
Kernan, J., Mueller, L., & Wollack, J. (2015). Crafty cheaters and determined detectives: What you need to know about NPTE security. https://www.fsbpt.org/Portals/0/Content%20Manager/PDFs/Forum/Forum_Fall2014_CraftyCheaters.pdf
Course Objectives
Attendees will be able to identify resources on the FSBPT website that can inform program development at their institutions.
Attendees will be able to compare their curriculum with the elements in the NPTE Practice Analysis.
Attendees will be able to provide feedback and guidance on the NPTE passing standard to their students.
Instructional Methods
The presentation will consist of PowerPoint slides, primarily showing screenshots of resources and the results of research. The presenters will provide a handout with links to key resources and references. There will be time for questions and research suggestions at the end of the presentation.
Tentative Outline/Schedule
5 minutes – Introductions and agenda
15 minutes – Purpose the NPTE Research Agenda
15 minutes – How topics are chosen for the Research Agenda and how research is done
25 minutes – Examples of NPTE research and outcomes
15 minutes – How to relate research outcomes to public protection and future research
15 minutes – Questions and suggestions