Notes on “ai” tutoring
You recently mentioned the Alpha School and their claims about AI tutoring. I share the skepticism expressed in your comments section regarding selection bias and the lack of validated academic benchmarks.
I wanted to highlight a more rigorously evaluated project called Tutor CoPilot, conducted jointly by Stanford’s NSSA and the online tutoring firm FEVTutor (sadly they’ve since gone bankrupt). To my knowledge, it’s the first and only RCT examining AI-assisted tutoring in real K-12 school districts.
Here’s the study: https://nssa.stanford.edu/studies/tutor-copilot-human-ai-approach-scaling-real-time-expertise
Key findings:
- Immediate session-level learning outcomes improved by 4-9%.
- Remarkably, the tool impacted tutors even more than students. After six weeks, inexperienced tutors reached performance parity with seasoned tutors, and previously low-performing tutors achieved average-level results.
