Grading policy, email nudges, and grade outcomes
Publication id: shakya-2024-grading-policy-email-nudges
Status: verified
Citation: Shakya, S., & Levinstein, M. (2024). Grading policy, email nudges, and grade outcomes. The Journal of Economic Education. Download PDF
Facts¶
Policy hook¶
Professor-sent email reminders to students scoring below 30 points on weekly assignments.
Main finding¶
Email nudges improved low-performing student grades by 22-28 points per 50 points (44-56% improvement). Effect remained robust across 12 weeks despite some evidence of nudge fatigue with repeated treatments.
Data and setting¶
80 consenting students across three sections of Principles of Macroeconomics, hybrid format, spring 2021 at Shippensburg University. Students received part-I and part-II assignments weekly, each worth 50 points. Email sent if part-I score below 30.
Research design (plain language)¶
Difference-in-differences approach using observational data where treatment assignment was conditional on prior performance (part-I score). Exploited university power outages as natural experiment for robustness (triple difference-in-differences). Conducted 1,000 permutation tests to verify results were not due to chance.
One caveat¶
Results based on observational, not randomized, data. Treatment effects specific to low-performing students and cannot be interpreted as average treatment effects for the population. Repeated nudges showed diminishing returns (22-28 points for one nudge vs. 24 points after four nudges), suggesting saturation effects.
PDF or DOI¶
Why it matters¶
Prior studies on email nudges in higher education show mixed results. This paper demonstrates substantial effects when nudges target low performers specifically. The 22-28 point improvement is large compared to other interventions, suggesting cost-effective reminders could improve retention and academic performance, particularly for struggling students.