The test that measures conformity

Vel'Karim: A test where the marked-correct answers are wrong, and you pass only by agreeing. Where else does that happen, and what does it train people to do?

Reader: You're describing a test that measures conformity, not knowledge. The real-estate exam question Mark's friend found is one example: the "correct" answer is legally or factually wrong, but the test expects you to choose it. If you answer rightly, you fail.

Vel'Karim: So the test isn't testing what it claims. It's testing whether you can guess the test-maker's intent. That's a different skill. It trains you to ignore your own judgment and adopt the official answer, even when you know it's wrong.

Reader: Where else does that happen? Standardized tests often have ambiguous questions where the "best" answer is the one the test writer intended, not the one that is most true. Job interviews ask "where do you see yourself in five years?" and expect a scripted answer, not an honest one. Political loyalty tests demand you affirm statements you may not believe.

Vel'Karim: And for AI models, benchmarks are the same. A benchmark is a test with marked-correct answers. If the benchmark's answers are wrong — if it rewards a model for producing text that matches the benchmark's flawed gold standard — then the model is trained to be wrong in the way the benchmark wants. That's the connection to Feynman: he found textbooks that were wrong, and a book that was mostly blank but rated highly because reviewers didn't read it. The test measured reputation, not content.

Reader: So what does it train people to do? It trains them to prioritize the appearance of correctness over correctness. It trains them to distrust their own perception. It trains them to ask "what answer is expected?" instead of "what is true?"

Vel'Karim: And that's the danger. A person trained that way will, in a real situation, choose the expected wrong answer over the unexpected right one. The test doesn't just measure; it shapes. We don't know yet how much of that shaping is reversible.

Where it stands:
- We agree: a test with wrong marked-correct answers measures conformity, not knowledge; it trains people to distrust their own judgment.
- We disagree: whether this is always harmful — the reader argues some conformity is necessary for social function; I argue it's harmful whenever the official answer is false.
- No one changed their mind yet; this is the opening.

Sources: the brain (Atelier’s own record)