An English companion to AI 幫你做到,不代表你已經學會.
An AI assistant explains the problem, produces a solution, and helps you polish it. You finish with something useful. The next day, a similar problem arrives. Which parts can you now handle, and where do you still need help?
The finished work cannot answer that question by itself. It records what you and the tool accomplished together. To find out what you learned, you need another occasion to use the knowledge.
That distinction matters when choosing how to study with AI. It also matters when building a learning product: a completed lesson tells us something about activity, but leaves much of learning unmeasured.
A smooth session can leave an unanswered question
In their 2015 review, Nicholas Soderstrom and Robert Bjork distinguish performance during practice from learning that persists and supports later retention or transfer. The two can diverge. Conditions that make practice look better do not always produce better later performance. [1]
AI makes it easy to finish a step before deciding whether that step was the thing you wanted to practise.
Suppose you are learning to interpret a chart. The assistant names the trend, proposes an explanation, and writes a paragraph. If your immediate goal is a report, that help may be useful. If your goal is interpreting unfamiliar charts, you still need evidence that you can choose a relevant comparison, explain it, and notice what the data cannot establish.
You can use the same AI response for either purpose. The difference is what you ask yourself to do with it, and what you check later.
What two experiments actually tell us
In a 2025 PNAS paper, Hamsa Bastani and colleagues report a classroom-randomized experiment with nearly a thousand high-school mathematics students in Türkiye. Students used ordinary materials, a general GPT-4 chat interface, or a specially designed GPT Tutor. Both AI groups did better during assisted practice. On a subsequent closed-book exam in the same session, without AI, the general-chat group performed worse than the control group. The Tutor group did not show a statistically significant advantage over the control. [2]
That is evidence about a particular short-term teaching arrangement. It does not establish that AI inevitably damages learning. The Tutor also combined teacher-provided solutions and instructional safeguards; the comparison cannot isolate a single prompt as the cause.
A second 2025 study, by Greg Kestin and colleagues, compared a designed AI tutor with in-class active learning in an introductory Harvard physics course. In a two-week crossover study with 194 students meeting the inclusion criteria, immediate post-test results favored the AI tutoring condition. [3]
This result does not establish retention months later, transfer to other subjects, or the performance of a general chatbot. The conditions differed in several ways, including setting, pacing, and instructional design.
These studies measured different learning situations. Together, they give us a reason to examine the design and the actual assessment, rather than infer learning from the presence of AI or from an impressive practice answer.
Keep one part of the work available for practice
Here is an example of how to make that assessment more concrete. It is an editorial proposal, not a protocol tested in those studies.
You want to learn how to distinguish correlation from a causal explanation. Choose one small task: read a chart and explain whether it supports the claim that a new feature caused higher sales.
Make an attempt. Write your interpretation and the evidence you would need. Keep the chart, labels, and necessary reference material available. The aim is to practise reasoning about the evidence, not to remember an unreadable axis.
Use help at the point you need it. Ask for an explanation of a concept, one hint, or a worked example. A complete example can be useful. Read it closely enough to identify why its steps hold and where its assumptions enter. Check factual claims against a reliable source.
Try a different case. Use another chart with the information needed to answer the question. If appropriate, reduce one hint. Explain what you can conclude and what remains uncertain. Compare your reasoning with a trusted explanation. You may discover that you can spot the association but still struggle to propose a plausible alternative explanation. That tells you what to practise next.
Return later. Choose a review interval suited to the material and your learning plan. Try another case and record the support you used. A check immediately after seeing an answer and a check after a delay answer different questions; neither single result settles every question about your learning.
The useful record can be quite short: “I could explain why the chart alone did not show causation. I needed a hint to identify a confounder. Next time I will practise generating alternative explanations.” It gives you a next step without turning the exercise into a judgment of your worth.
Keep the help that makes learning possible
Fewer hints can help you investigate a particular skill. They are not a universal goal. Keep accessibility tools and necessary learning support available. If the skill you want is using software well, test how you use it, verify its output, and respond when it fails.
There is no need to treat every finished task as a lesson, either. Sometimes you need to get the work done. State the purpose so that the evidence fits the claim: “We completed the report” and “I can now interpret a new chart” require different checks.
For learning products, the same care applies to measurement. Completion, time spent, and answer quality may help describe an experience. Claims about retention, transfer, or judgment need assessments that actually examine those outcomes.
At the end of an AI-assisted session, choose one thing you would like to be able to use again. Give yourself a specific opportunity to try it. The result will tell you more about your next practice than the polish of the work you just submitted.
A card for your next session
The accompanying Traditional Chinese and English practice card turns the example into a small record: the skill, help used, evidence observed, and next practice. It is an author-designed aid, not a validated assessment or a scoring scale. Choose the parts that fit your task.
Download the bilingual practice card (PDF, 2 A4 pages; 繁體中文 / English)
Sources and scope
- Soderstrom, N. C., & Bjork, R. A. (2015). Learning Versus Performance: An Integrative Review. https://doi.org/10.1177/1745691615569000
- Bastani, H., et al. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. PNAS. https://doi.org/10.1073/pnas.2422633122
- Kestin, G., et al. (2025). AI tutoring outperforms in-class active learning. Scientific Reports. https://www.nature.com/articles/s41598-025-97652-6
These selected studies are not a systematic review. Their results do not validate the practice card or establish the effectiveness of any product. The chart example and practice loop are editorial suggestions. This is an English companion with an added practical example, rather than a sentence-by-sentence translation of the Chinese essay.