Changing the code around the same AI model can reportedly create up to a 6× performance gap

A post describing a Stanford and MIT paper says an AI model’s “harness”—the code controlling storage, retrieval, what the model sees and how workflows run—can substantially affect performance. It describes gaps of up to 6× with the same model on the same benchmark when the harness changes. The post says Meta-Harness automatically improves that code by giving an optimizing agent access to previous code, logs and execution traces. It reports a 7.7-point improvement on online text classification over a strong state-of-the-art context-management approach while using 4× fewer context tokens, plus an average 4.7-point gain on retrieval-augmented math reasoning across five held-out models on 200 International Mathematical Olympiad–level problems.