Ask a question below.
Published answers will appear here.
I would claim that GPT-5.5 and maybe models like GPT-5.2 could already do that in Aidans case this task will probably look something like this: - they want to improve something like sycophancy or honesty or intent recognition - Aidan has an idea - Aidan tells model that idea - the model implements some LLM-as-a-Judge grader or the model implements some multi-agent game where good communication is rewarded or what would be a bit more impressive is the model gathers and trains a reward model on existing user data and then uses this for RL but in the end it will always be executed on existing OpenAI RL infrastructure, and there's no need to write any training or inferencing code I don't think that's really end-to-end Aidan is still providing the ideas, and existing OpenAI infrastructure makes RL training probably as simple as writing a config like in primerl but what is probably very nice is having the model babysit the RL runs and analyze them afterwards and come up with new ideas for better reward shaping or whatever
we’ve been tracking this capability for quite a while (https://posttrainbench.com/) in a more open-ended setting where the agent doesn’t have the best possible post-training tools. it appears that if the agent has SOTA infra and data access, just running post-training autonomously is super straightforward! so AI *research* automation is still an open problem, but the AI *engineering* automation part is largely already there. very interesting!
What do I mean with open-ended prompts for research? Essentially “here’s some data, find what’s interesting and showcase it to me”. Sol, similar to old models, latches on pre-existing ideas and doesn’t look at the data the way a good human would. Thus, its results are bland
this post is not to doubt that GPT-5.6 is useful or not accelerating work internally but I want to distance these statements from literal autonomous end-to-end post-training or research models can't do that right now
Minh Nhat Nguyen adds that end-to-end RL runs are highly feasible using established setups
I would claim that GPT-5.5 and maybe models like GPT-5.2 could already do that in Aidans case this task will probably look something like this: - they want to improve something like sycophancy or honesty or intent recognition - Aidan has an idea - Aidan tells model that idea - the model implements some LLM-as-a-Judge grader or the model implements some multi-agent game where good communication is rewarded or what would be a bit more impressive is the model gathers and trains a reward model on existing user data and then uses this for RL but in the end it will always be executed on existing OpenAI RL infrastructure, and there's no need to write any training or inferencing code I don't think that's really end-to-end Aidan is still providing the ideas, and existing OpenAI infrastructure makes RL training probably as simple as writing a config like in primerl but what is probably very nice is having the model babysit the RL runs and analyze them afterwards and come up with new ideas for better reward shaping or whatever
okay probably only possible since GPT-5.5 https://x.com/scaling01/status/2075357355168915744/photo/1
like i don't think that statement alone has any weight without knowing what previous models would look like
@AndrewCurran_ in an existing setup, w new data/some tweaks yeah this is entirely possible
also this https://x.com/scaling01/status/2075366238977454203/photo/1
Ask a question below.
Published answers will appear here.
like i don't think that statement alone has any weight without knowing what previous models would look like