A study tracking 26,811 Chinese secondary students across 30 months shows generative AI adoption shortened homework time by roughly 30 percent and lifted homework scores by 18 percent, yet monthly exam scores dropped 20 percent within six months, with entrance-exam penalties reaching 18 to 24 percent after two years.
More evidence, from a large-scale study in China, that using AI hurts learning if it undermines mental effort. When homework time drops due to AI use, so do test scores.
Across studies, a theme: AI tutoring in support of classes is good, using AI to "help" with homework is bad.
Roughly 80 percent of users who finished assignments unusually fast while scoring high on homework drove the learning losses; those who kept similar completion times to non-users saw only minor declines.
The steepest drops appeared in social sciences, then STEM and languages, concentrated among junior students, high-achievers, and boys, but whether schools can steer AI toward supplemental tutoring instead of replacement is still unknown.
Positive users agree the study shows AI shortcuts reduce learning friction and see promise in Socratic AI tutors for deeper education, while negative users call the findings dubious or warn that AI severely impairs cognition.
No Digg Deeper questions have been answered for this story yet.
#184The instinct of students is often to use AI to help with homework, even if they are not trying to cheat. And because off-the-shelf chatbots are a helpful assistant, rather than a tutor, they give you the answer and undermine learning.
Paper: https://cepr.org/publications/dp21577
More evidence, from a large-scale study in China, that using AI hurts learning if it undermines mental effort. When homework time drops due to AI use, so do test scores.
Across studies, a theme: AI tutoring in support of classes is good, using AI to "help" with homework is bad.
I talk about the research on when AI undermines, versus supporting, thinking and learning here: https://www.oneusefulthing.org/p/choosing-to-stay-human
The instinct of students is often to use AI to help with homework, even if they are not trying to cheat. And because off-the-shelf chatbots are a helpful assistant, rather than a tutor, they give you the answer and undermine learning.
Paper: https://cepr.org/publications/dp21577
clear solution to ai in school: 1. grade advancement is based entirely on test performance 2. and you can graduate early if you demonstrate mastery
More evidence, from a large-scale study in China, that using AI hurts learning if it undermines mental effort. When homework time drops due to AI use, so do test scores.
Across studies, a theme: AI tutoring in support of classes is good, using AI to "help" with homework is bad.
@emollick It hurts test scores, but does it matter and is that the same as learning? Maybe our tests need to change. Maybe our current testing paradigms are built for a pre-LLM world that becomes less and less relevant every day.
@emollick Yes you are right
@emollick I’m imagining an in-person android using the Socratic method will engage the five senses with students. AI just being chat turns it into a type of hackable video game.
@raspberryman @emollick Eventually yes.
But even now, if there were just the right models specifically tuned for Socratic method, it would do wonders.
@emollick “Homework with AI” was never the studied behavior, it’s “homework you no longer have to think through” — different thing entirely.
@emollick the AI made the homework look better and the learning worse at the same time that's the most dangerous combination possible because the metric everyone's watching is exactly the one that lies to you
@emollick also, professionally for people like us, give us a huge surface to handle more demand and work... not healthy, but extends our capacity alot.
@emollick The distinction between AI as tutor vs AI as answer machine is the key. Students using it to skip thinking is the problem, not the tool itself.
@V0R0N01 @emollick It's abundantly clear to everyone outside of frequent AI users what it's doing to them: it makes them severely retarded over the course of a year. It isn't subtle at all, everyone knows.
@emollick or is it evidence that time spent doing homework is correlated with test scores
@emollick I’ve been telling people this for 3 years. The friction *is* when the learning is happening.
@emollick Maps onto the desirable difficulty research pretty cleanly: the struggle is the part that actually encodes. AI is incredible at removing friction, which is exactly the wrong thing to remove when the friction is the learning.
This maps directly to "desirable difficulties" in learning science — the cognitive struggle of working through a problem IS the encoding mechanism. When AI removes that struggle, it removes the learning, even if the homework gets done faster. The design implication is significant: AI tools built to tutor (Socratic prompting, hints, asking the student to explain their reasoning) should show the opposite effect in this kind of study. The tool architecture matters as much as whether students "use AI."
@emollick very dubious
@emollick The "desirable difficulties" literature predicts this exactly. The design question worth tracking: does the AI scaffold productive struggle or replace it? A nudge to re-derive beats writing the essay.
@emollick The role of AI in the classroom is pivotal in supporting inquiry and fostering deeper learning, not shortcuts.
@emollick “Bad” sigh.
It’s likely “fine”. These tools will simply get better, the students will likely have access throughout their lives. They will be far more productive with these tools than if the tools had never existed.
