We can perform this conformalization process with limited data as predicting whether improvement is *possible* is often easier than knowing the *precise* language sequence that will elicit the improvement. [5/n]
However, this LFP can generalize poorly OOD. In response, we use techniques from conformal prediction to figure out when to "fall back" to the default user prompt, making sure steering is "mostly harmless." [4/n]
Users praise the conformal prediction research for safer language model steering as an incredibly impressive first PhD paper showcasing strong work ethic and depth.
No Digg Deeper questions have been answered for this story yet.
However, this LFP can generalize poorly OOD. In response, we use techniques from conformal prediction to figure out when to "fall back" to the default user prompt, making sure steering is "mostly harmless." [4/n]
We propose a method for doing a tractable "local search" (similar to what we explored in https://gokul.dev/sailor/) in the space of natural language, before using expert iteration to train our "language feedback policy" (LFP). [3/n]
Anyhow, I think this is an *incredibly* impressive first paper of the PhD for @hyunjoej, who continually impressed me with his work ethic and depth of thought. I can't recommend working with him enough :). Check out our website for more: https://hyunjoe.xyz/LanguagePolicy/.