Why practice scores mislead you, and how to judge readiness
A practice score is a measurement, and like any measurement it can be measuring the wrong thing. Two of the candidate write-ups linked from these guides describe the same gap: practice that felt more reassuring than the exam turned out to be.
The climbing score
John Weidner prepared for Architect Foundations with Anthropic’s official practice exam, which was still available then. He took it several times and screenshotted every question he got wrong. His score went from somewhere in the 700s to about 850 and then about 930. Then he sat the real exam, recognized none of the questions, and passed with 738. By his own count, one more wrong answer would have put him under.
His explanation is honest. The practice exam reused its question pool, so studying his screenshots partly taught him the practice answers rather than the ideas behind them. The scores went up because his memory of that pool improved.
Szymon Paluch noticed the same gap from a different angle. The sample questions he studied were easier than the ones on his form. They taught him how the questions are asked, not how hard the sitting would be.
The official practice exam has since been retired. Anthropic now points to the sample questions in each exam guide. The lesson applies to any question set, including this one. The second time you see a question, your score on it measures memory.
Tutors that agree with you
Weidner also had Claude quiz him on the exam guide every evening for a week, and checked the hard ones against a second model. Both models had him convinced he was ready. His conclusion is worth repeating: an AI tutor is a good sparring partner and a poor judge. It explains well and drills well. It is also encouraging by nature, and encouragement is not readiness.
Your daily tool is your blind spot
Weidner uses Claude Code every working day, and it was his weakest domain on the score report, the one the report marked for attention. Daily use had made him fluent in the parts of Claude Code he reaches for and left gaps around the parts he doesn’t.
The Cloud Life write-up reads at first like the opposite case. Its author scored best in the domains they work in daily, MCP tool design and Claude Code, and worst in the one they used least. The underlying point is the same in both accounts. The exam samples the whole domain as the guide defines it, and your job covers only the part your team happens to use.
Places to look:
- the settings and configuration levels you never had to change
- permission modes and options that your team’s defaults make unnecessary
- error and stop conditions you have never hit
- features your organization hasn’t turned on
A useful exercise is to open the guide for your exam, take the domain you’re most confident in, and read its objectives line by line. Mark each one you have done yourself, not just watched someone do. The unmarked lines are where your daily tool can’t help you.
What a readiness check should look like
No score threshold makes you ready, and the thresholds in write-ups are personal. One author treats 900 on the retired practice exam as the bare minimum, and another waited until a third-party set showed 90%. Neither number transfers to a different question set. What you can control is whether your check measures the right thing:
- Use questions you haven’t seen. A repeated question tells you about your memory. Keep your best unseen set for last.
- Take it timed, in one sitting, without notes. Under time pressure you read differently, and the pacing guide shows how much less time you have per item than you think.
- Read the domain scores, not just the total. The exam’s pass or fail decision uses the total. But a domain you score badly in practice pulls that total down on every form, because every domain supplies its share of items.
- Explain every wrong option. Before you look at the explanation, say why each wrong option is wrong. If you got the item right but can’t do that, count it as a lucky guess.
- Check again after a gap. Something you understand still makes sense a few days later. Something you memorized fades.
Two exam-size practice runs on unseen questions, both reviewed miss by miss, tell you more than ten passes over the same pool.
Use the free form as a first measurement
Take the free form for your exam once, cold, under exam timing. Its 20 questions are spread across the exam’s domains, and the result shows a score per domain. Treat that first result as the honest one. If you take it again, you’re testing your memory of it.