COMPARISON OF A NOVEL IMMEDIATE FEEDBACK TOOL VS. SCANTRON ON STUDENT RETENTION OF KNOWLEDGE: A PILOT STUDY
Purpose
Performance feedback in the academic setting is typically done in delayed format, despite research suggesting immediate performance feedback (IPF) is superior to delayed performance feedback (DPF), when assessing a student’s retention of knowledge. The Immediate Feedback Assessment Tool (IF-AT) can be used to provide a student with IPF, and the effectiveness of this tool has been studied with undergraduate psychology students. To date, no research exists on the effect of using the IF-AT for IPF with graduate students, more specifically, Doctor of Physical Therapy (DPT) students. The purpose of this study was to investigate whether using the IF-AT to provide IPF was more effective than using the Scantron, the DPF tool, for increasing retention of knowledge in DPT students.
Methods/Description
A pilot study was done at XXX and included 36 first year DPT students enrolled in the Basic Patient Care Skills course in fall semester 2016. Students were randomly assigned to one of two groups and a randomized crossover study design was utilized. A total of 8 quizzes were given in the course, four prior to the midterm examination and four prior to the final examination. Group one used the IF-AT for odd numbered quizzes and the Scantron for even numbered quizzes; group two did the opposite. Three questions were repeated from each quiz, for a total of 12 repeat questions, on the midterm examination. The same format was followed for the final examination. Retention of knowledge was considered to have occurred if a student incorrectly answered a question on a quiz, but correctly answered that same question on the midterm or final examination. Statistical analysis was performed to determine retention of knowledge on the repeated quiz questions.
Results/Outcomes
Thirty-five subjects completed the study. One subject was dropped due to voluntary withdrawal from the DPT program. No statistically significant difference in performance was found between the two groups on the midterm examination. A statistically significant (p<0.05) difference between the two groups was found for two questions (Q6 (p=0.004) and Q10 (p=0.039)), from quiz 5, on the final examination. Statistical significance was found for 16.67% of repeated questions on the final examination, indicating students who received IPF had better retention of knowledge compared to those who received DPF.
Conclusions/Relevance to the conference theme: Through the Looking Glass: Transforming Physical Therapy Education
This pilot study detected a small, but significant trend in improved retention of knowledge for students receiving IPF as compared to those receiving DPF on a quiz. The results also revealed that those students who received DPF never showed a significant improvement in knowledge from a quiz to the midterm or final examination. This suggests that, similar to undergraduate students, DPT students who receive IPF have improved retention of knowledge from a quiz to examination in a single course. Further research is needed to determine if implementing IPF throughout a DPT curriculum results in better knowledge retention and performance on the NPTE.
References
1. Epstein ML, Epstein BB, Brosvic GM. Immediate feedback during academic testing. Psychol Rep. 2001;; 88(3 Pt 1):889--894.
2. Epstein, MH, Lazarus, AD, Calvano, TB, et al. Immediate feedback assessment technique promotes learning and corrects inaccurate first responses. The Psychological Record. 2002;; 52:187–201.
3. Dihoff RE, Brosvic GM, Epstein ML. The role of feedback during academic testing: The delayed retention effect revisited. The Psychological Record. 2003;; 53:533--548.
4. Brosvic GM, Epstein ML. Enhancing learning in the introductory course. The Psychological Record. 2007;; 57: 391
5. Brosvic GM, Epstein ML, Cook, MJ. Efficacy of error for the correction of initially incorrect Assumptions and of feedback for the affirmation of correct responding: Learning in the classroom.The Psychological Record. 2005;; 55:401--418.
6. Robin AL. The timing of feedback in personalized instruction. Journal of Personalized Instruction.1978;; 3: 81--87.
7. Dihoff RE, Brosvic GM, Epstein ML. Provision of feedback during preparation for academic testing: Learning is enhanced by immediate but not delayed feedback. The Psychological Record. 2004;; 54:207--231.
8. Bruning R, Schraw G, Ronning R. Cognitive Psychology And Instruction. Third Edition [e-- book]. 1999. Available from: ERIC, Ipswich, MA. Accessed December 15, 2015.
9. Guthrie ER. The psychology of learning. New York. Book: Harper;; 1935.
10. Kulhavy RW, Stock WA. Feedback in written instruction: The place of response certitude. Educational Psychology Review. 1989;; 1: 279--308.
11. Smith JA. . Immediate feedback and assessment technique (IF--AT) testing forms: An overview of the tool and uses. Development in Business Simulation and Experimental Learning. 2013;; 40:109
12. APTA. APTA. Available at: https://www.apta.org/