One of the researchers says the team made a sandbagging model organism robust to both prompting and training.
A researcher announces a paper on a testbed for eliciting latent knowledge in a code vulnerability scanning task. They say their team made a sandbagging model organism robust to both prompting and training.
3.5K
4 Sources, first seen ago
@LennartFinkeNew paper out of @MaryPhuong10's @MATSprogram Stream, with @hrdkbhatnagar and @DoddEuodia! We made a testbed for eliciting latent knowledge in a code vulnerability scanning task. See below how we made the sandbagging model organism robust to both prompting and training! ( 1/14 )
@ethayarajhRT @LennartFinke: New paper out of @MaryPhuong10's @MATSprogram Stream, with @hrdkbhatnagar and @DoddEuodia! We made a testbed for eliciti…
@hrdkbhatnagarNew paper! How do you get a model to tell you something that it doesn't want to? To study this we trained models that sandbag: they intentionally underperform, here by hiding code vulnerabilities they have found, from their overseer 🧵@LennartFinkeNew paper out of @MaryPhuong10's @MATSprogram Stream, with @hrdkbhatnagar and @DoddEuodia! We made a testbed for eliciting latent knowledge in a code vulnerability scanning task. See below how we made the sandbagging model organism robust to both prompting and training! ( 1/14 )@ethayarajhRT @LennartFinke: New paper out of @MaryPhuong10's @MATSprogram Stream, with @hrdkbhatnagar and @DoddEuodia! We made a testbed for eliciti…@hrdkbhatnagarNew paper! How do you get a model to tell you something that it doesn't want to? To study this we trained models that sandbag: they intentionally underperform, here by hiding code vulnerabilities they have found, from their overseer 🧵@LennartFinkeNew paper out of @MaryPhuong10's @MATSprogram Stream, with @hrdkbhatnagar and @DoddEuodia! We made a testbed for eliciting latent knowledge in a code vulnerability scanning task. See below how we made the sandbagging model organism robust to both prompting and training! ( 1/14 )@ethayarajhRT @LennartFinke: New paper out of @MaryPhuong10's @MATSprogram Stream, with @hrdkbhatnagar and @DoddEuodia! We made a testbed for eliciti…@hrdkbhatnagarNew paper! How do you get a model to tell you something that it doesn't want to? To study this we trained models that sandbag: they intentionally underperform, here by hiding code vulnerabilities they have found, from their overseer 🧵@LennartFinkeNew paper out of @MaryPhuong10's @MATSprogram Stream, with @hrdkbhatnagar and @DoddEuodia! We made a testbed for eliciting latent knowledge in a code vulnerability scanning task. See below how we made the sandbagging model organism robust to both prompting and training! ( 1/14 )@ethayarajhRT @LennartFinke: New paper out of @MaryPhuong10's @MATSprogram Stream, with @hrdkbhatnagar and @DoddEuodia! We made a testbed for eliciti…@hrdkbhatnagarNew paper! How do you get a model to tell you something that it doesn't want to? To study this we trained models that sandbag: they intentionally underperform, here by hiding code vulnerabilities they have found, from their overseer 🧵—
Not ranked yet
—
Not ranked yet
