Commentary on X
@CurtTigges @tenobrus I agree with this for small amounts of RL, not sure it holds for large amounts of RL. At some point both weights and behavior have changed dramatically; seems quite possible that the stuff discussed in the screenshot scales far enough to get qualitatively different models?
@tenobrus Agreed! “Oracle” models/agents will probably play a greater role in mech interp (as we’ve already seen) as well as RL + post training agents who manage and customize agents front to end.
@zephyr_z9 Why people forget it is the country that perfected sweatshops? We are going to look back at this as the stupidest accusation of the history.