/TECH

Ryan Greenblatt proposes single forward pass to isolate model pretraining quality

The method bypasses later reinforcement learning to evaluate core-measure core capabilities.

Combined views

4K

5 posts, first seen 11h ago

51 likes

Ryan Greenblatt proposes single forward pass to isolate model pretraining quality

The method bypasses later reinforcement learning to evaluate core-measure core capabilities.

Commentary on X

RN Ramez Naam@ramez

11:02 PM · Jul 16, 2026

1995TECH

Thanks. This is a very interesting metric, which I'd observed before in the context of the ability of AI models to act covertly without human observation. Can you share anything of your results with K3? I realize this is day of release which is unreasonably fast, so no worries if it will be some time before you publish your findings.

RN Ramez Naam@ramez

8:30 PM · Jul 16, 2026

1995TECH

@RyanGreenblatt Tell us more about these tests, please. What sort of test can determine what fraction of a model's capabilities are pretrain vs RL or other later stages? Might we be able to replicate these ourselves, on this model and others?

RG Ryan Greenblatt@RyanGreenblatt

11:00 PM · Jul 16, 2026

1105TECH

Oops, I linked the wrong thing. This is a better canonical paper, but I just did quick and dirty experiments using this dataset: https://www.lesswrong.com/posts/Ty5Bmg7P6Tciy2uj2/measuring-no-cot-math-time-horizon-single-forward-pass

Combined views

4K

5 posts, first seen 11h ago

51 likes4 comments

4 comments

5 saves

Sentiment

Positive100%0%Negative

Positive Read

Users appreciate the metric in AI pretrain capability tests because it reveals models acting covertly without human observation.

Based on 1 visible X reactions from 3 accounts; directional sample.

Useful Links

Measuring no CoT math time horizon (single forward pass) — LessWrong(lesswrong.com)viaRyan Greenblatt

Digg Deeper

Ask a question below.

Published answers will appear here.

Commentary on X

RN Ramez Naam@ramez

11:02 PM · Jul 16, 2026

1995TECH

Thanks. This is a very interesting metric, which I'd observed before in the context of the ability of AI models to act covertly without human observation. Can you share anything of your results with K3? I realize this is day of release which is unreasonably fast, so no worries if it will be some time before you publish your findings.

RN Ramez Naam@ramez

8:30 PM · Jul 16, 2026

1995TECH

@RyanGreenblatt Tell us more about these tests, please. What sort of test can determine what fraction of a model's capabilities are pretrain vs RL or other later stages? Might we be able to replicate these ourselves, on this model and others?

RG Ryan Greenblatt@RyanGreenblatt

11:00 PM · Jul 16, 2026

1105TECH

Oops, I linked the wrong thing. This is a better canonical paper, but I just did quick and dirty experiments using this dataset: https://www.lesswrong.com/posts/Ty5Bmg7P6Tciy2uj2/measuring-no-cot-math-time-horizon-single-forward-pass

RG Ryan Greenblatt@RyanGreenblatt

10:52 PM · Jul 16, 2026

1105TECH

@ramez As other people have noted, it's single forward pass math:

View all

RN Ramez Naam@ramez

10:56 PM · Jul 16, 2026

1995TECH

@RyanGreenblatt Thanks. Will read up and educate myself.

RG Ryan Greenblatt@RyanGreenblatt

10:52 PM · Jul 16, 2026

1105TECH

@ramez As other people have noted, it's single forward pass math:

View all

RN Ramez Naam@ramez

10:56 PM · Jul 16, 2026

1995TECH

@RyanGreenblatt Thanks. Will read up and educate myself.

The Conversation on X

5 posts

RN Ramez Naam@ramez

8:30 PM · Jul 16, 2026

1995TECH

@RyanGreenblatt Tell us more about these tests, please. What sort of test can determine what fraction of a model's capabilities are pretrain vs RL or other later stages? Might we be able to replicate these ourselves, on this model and others?

RN Ramez Naam@ramez

11:02 PM · Jul 16, 2026

1995TECH

Thanks. This is a very interesting metric, which I'd observed before in the context of the ability of AI models to act covertly without human observation. Can you share anything of your results with K3? I realize this is day of release which is unreasonably fast, so no worries if it will be some time before you publish your findings.

RN Ramez Naam@ramez

10:56 PM · Jul 16, 2026

1995TECH

@RyanGreenblatt Thanks. Will read up and educate myself.

RG Ryan Greenblatt@RyanGreenblatt

11:00 PM · Jul 16, 2026

1105TECH

Oops, I linked the wrong thing. This is a better canonical paper, but I just did quick and dirty experiments using this dataset: https://www.lesswrong.com/posts/Ty5Bmg7P6Tciy2uj2/measuring-no-cot-math-time-horizon-single-forward-pass

RG Ryan Greenblatt@RyanGreenblatt

10:52 PM · Jul 16, 2026

1105TECH

@ramez As other people have noted, it's single forward pass math:

Ryan Greenblatt@RyanGreenblatt

This is based on single forward pass perf (on math). This is imperfect, but corresponds to pretrain quality. (Minimally, it isn't improved by post-training.) Loss would be better, but we don't have the base and can't measure loss on closed weight models. https://www.lesswrong.com/posts/SieLowPgNgRSPGhFw/estimating-no-cot-task-completion-time-horizons-of-frontier

5 saves

Sentiment

Positive100%0%Negative

Positive Read

Users appreciate the metric in AI pretrain capability tests because it reveals models acting covertly without human observation.

Based on 1 visible X reactions from 3 accounts; directional sample.

Useful Links

Measuring no CoT math time horizon (single forward pass) — LessWrong(lesswrong.com)viaRyan Greenblatt

Digg Deeper

Ask a question below.

Published answers will appear here.

Ryan Greenblatt proposes single forward pass to isolate model pretraining quality · Digg