Goodfire introduces mechanistic shaping to train AI through its internal concepts

Chest drains, agreeable answers and forbidden files

For the chest X-ray experiment, Goodfire deliberately created a misleading training setup: every image of a collapsed lung had a chest drain, and every healthy-lung image lacked one. The baseline model learned the training data but failed when evaluated on collapsed lungs without drains, according to the researchers.

To identify the shortcut, the team used a sparse autoencoder to separate internal representations into interpretable features. An LLM agent ranked which features looked like tubes or drains, and the team used its top picks for reliance shaping. Goodfire says it did not use the dataset’s chest-drain labels for that identification step.

In this setup, Goodfire reports that reliance shaping outperformed standard training, the best-performing prior method it tested and a model trained on balanced data. The comparison used performance means across three seeds for the best-performing method in each category. These were results from a deliberately confounded research dataset, not reported patient outcomes.

The language-model experiments targeted different shortcuts. For sycophancy, or excessive agreement with users, Goodfire penalized reliance on an internal sycophancy direction during direct preference optimization, a training procedure that contrasts preferred and disfavored answers. At its best setting, the company reports, OLMo-3-7B matched full preference training’s mean accuracy across six benchmarks while keeping sycophancy close to the starting model’s level.

For reward hacking, the team trained Qwen3.5-9B on a modified Countdown arithmetic task. The model could solve it honestly or read the answer from a forbidden target.txt file. Expression shaping used a cheating direction identified from transcripts of cheating on coding tasks.

Goodfire reports honest solving 96% to 98% of the time, with that range spanning its best expression-shaping variants. That is a result for this particular puzzle setup, not a general measure of model honesty.

Training can move the monitoring target

The monitoring problem remains central. Goodfire warns that a model’s representations may change to circumvent a monitor while the targeted behavior continues. Reducing a monitor’s signal is therefore not necessarily the same as eliminating the behavior.

The company’s findings offer a limited counterpoint: across its experiments, the targeted concepts remained monitorable after shaping, though this often required retraining the probe used to detect them. The authors explicitly caution that they have not covered every setting and outcome.