Volckano and Huison announce strategic cooperation
VOLCKANO
Talk to us

Can Using AI Leave You Less Practiced? A Skills Model, Checked

A new study asks: can using AI leave you less practiced? Its example is an expert who reviews code. Alone, she finds the bug 80% of the time; her AI assistant finds it 77% of the time. She is slightly better, but checking takes effort, and once effort is priced in, accepting the AI's answer pays more on every task. Results hold at 77% while her skill without the AI fades, unseen. These numbers are made up, from the paper's own example: a model, not a measurement of real workers.

What the study found

Today's Work, Tomorrow's Workforce: Incentivizing Learning in AI-Assisted Work, by Diener, Hendriks, Spitzer, Satzger, Holstein (KIT) and Sun (IBM Research), was posted on arXiv (2611.06346) on 4 November 2026. It is a preprint; we could not verify its peer-review status. It is a mathematical model of one worker and a line of similar tasks, drawing on two earlier studies. Solving a task alone costs effort but can raise skill a step; accepting the AI saves effort but skill can slip. In the model:

How strong is the evidence?

That practice builds skill is well supported (a review of 86 studies), though it measured memory over minutes to a week, not skill at work over months. Skill fades without practice, but no source we found measures a set loss per accepted AI answer; that is an assumption. The real-world evidence is two studies: a one-hour trial where 52 programmers using AI scored 4.15 points lower on a 27-point quiz than those without (small, a preprint), and an observational study at four clinics in Poland where detection on exams without AI fell from 28.4% to 22.4%; the paper says the AI was withdrawn, but it wasn't, so this root is stretched. The paper also leaves out an earlier experiment by some of the same authors: with 180 people, a fixed bonus for correct independent answers was gamed and accuracy fell slightly (63.2% against 64.3%). Verdict: plausible idea, thin evidence.

The experiment we ran

We rebuilt the model as a simulation with the paper's made-up values: a novice at 40%, a mid-level worker at 60% and an expert at 80%, 300 tasks each, AI at 77%. With the paper's rule, all three accepted every task and results stayed at 77%, but the expert's expected skill without AI fell to about 53% and 95% of simulated experts lost at least one step. With the person answering first on every second task, results stayed at 77% and the novice rose to about 59%, but the expert still ended at about 74%. One small test is noisy: if skill falls from 80% to 70%, a 20-case test shows no drop about 29% of the time. This checks the idea; it does not confirm the paper.

One practical takeaway

Use this as a design principle, not a measured fact. On some tasks you now hand to AI, do the work yourself first, then compare. Once a month, run a small blind set of past cases without the AI and watch the trend over months, not one score.

Watch the episode

First published by Volckano on volckano.com. © Verve Group Inc. If you reuse this material, credit Volckano and link to this page. https://volckano.com/media/ai-use-skill-practice-model

Start with one small pilot.

Tell us the task. We will say whether it is worth doing.

Book a consultation