Research, negative results preserved
latent-diagnostics
Correct answers were not enough. I wanted to know whether the model computed something real.
The problem
Grading answers tells you whether a model was right, not whether it computed something real.
What I built
Measures attribution graph geometry instead of only grading answers. Task domains show real signatures after controlling for length. Hallucination detection did not survive the same test. The repo keeps the negative results in.
Proof
Grammar influence d=1.08 after length control. 108 commits. The failed hypothesis is documented next to the confirmed one.
What I learned
Negative results are worth publishing. Hallucination detection did not survive length control, and the repo says so.
Research rigor: statistics, controls, and the honesty to keep failures in.
Stack
Python · SAEs · Attribution graphs
- status
- Research · Negative Results Preserved
- stack
- Python · SAEs · Attribution graphs
- finding
- Grammar influence d=1.08 after length control
- license
- MIT
Links
Being right and computing something real aren't the same shape.