Geoffrey Irving, previously Chief Scientist at the UK AI Security Institute, has started Sequent Research as a nonprofit pulling in alignment researchers from the former AISI Alignment Team, Timaeus, and other groups. The new outfit targets higher-confidence methods for keeping superintelligent systems aligned, mixing theoretical work with empirical experiments that the team plans to accelerate through heavy automation.
We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵
Sequent will recruit for in-person Berkeley roles plus remote spots in London, Melbourne, and elsewhere, with early strengths noted in scalable oversight, complexity theory, and learning theory. Interest also extends to agent foundations and game theory, though exact team size and launch timeline remain unspecified in the announcement.
No details have surfaced yet on initial funding, total raised, or runway, leaving open how the nonprofit will sustain its combined theory-plus-automation approach over the long term.
Positive users expressed excitement about the Sequent Research nonprofit launch and its ex-AISI team for superintelligence alignment, while negative users called the effort incoherent, impossible, or actively harmful.
Considering the language of the announcement alone, taken entirely at face value: This seems an enormous advance in attitude (and scientific integrity) over previous big projects. They claim non-optimistic results will be considered allowable, valuable, and publishable!
We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵
We don't know how to align superintelligence and I'm quite confident we won't learn in time.
We can try and make the coin more likely to land heads, but it's still a coin toss.
Tails: we all die.
We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵
Timaeus is joining forces with @geoffreyirving and researchers from UK AISI to found Sequent Research. 1/9
We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵
@geoffreyirving Overall, a surprisingly strong open that doesn't show many catastrophic attitudes of earlier groups.
> If AGI is possible then automated alignment research is possible, by definition
This however is false. Eg RLVR could give you AGI but not an aligned alignment researcher.
We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵
Timaeus was a beautiful dream, of using the Rising Sea of mathematics to turn the wheel of progress on the alignment problem. But we need the sea to rise faster. Time for some new axioms! 🧵
We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵
Looking forward to what comes out of Geoffrey's new alignment org. His 2018 paper on AI safety via debate is one of my all-time favorites: an elegant framing of the scalable oversight problem, way ahead of its time.
We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵
Sequent’s goal is to clear a higher bar:
1. We are aiming at higher confidence via a portfolio of theory and empirics bets (which could all fail!) 2. We’ll invest heavily in automation for fast progress 3. Theory boosts automation, via better filters for good research directions
Artificial superintelligence (ASI) may be developed in the next few years, and alignment is not on track! At a minimum, empirical research at AI labs is unlikely to deliver confidence, before training ASI, that alignment will go well.
this is my superbowl
We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵
Please reach out if you’re interested in working with us! Sequent will have a large in-person presence in Berkeley, as well as researchers remote from London, Melbourne, and elsewhere. 🇺🇸🇬🇧🇦🇺
1. Full post: http://sequent.org/launch 2. Express interest: http://sequent.org/apply
Artificial superintelligence (ASI) may be developed in the next few years, and alignment is not on track! At a minimum, empirical research at AI labs is unlikely to deliver confidence, before training ASI, that alignment will go well.
Our full announcement post has more details. Please express interest if you’d like to join us!
1. Full post: http://sequent.org/launch 2. Express interest here: http://sequent.org/apply
We believe Sequent will have reputation + funding to recruit world-class teams in many areas. Our initial team knows scalable oversight, complexity + learning theory, and personas. Areas we love include agent foundations, game theory, and heuristic arguments. Please pitch more!
Theory makes automation more likely to work: the models are great at prose math and Lean, which means significant acceleration even while most research taste comes from humans. But good automation is still hard: a single org will let us amortize the challenge across many areas.
New player on the scene, expect it to be big:
We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵
Excited to see people taking new, different bets on how to solve alignment!
We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵
I'm excited to work with you, @danielmurfet!
Timaeus was a beautiful dream, of using the Rising Sea of mathematics to turn the wheel of progress on the alignment problem. But we need the sea to rise faster. Time for some new axioms! 🧵
@geoffreyirving What an exciting combination of people! My mind is kind of blown by you and Daniel working together (with your colleagues). Looking forward to seeing what you accomplish!
We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵
Different research bets can help each other! Partial successes from one area will fill the gaps in others, increasing the value of bringing them together in one organization, and will focus on fast publication for sharing and engagement with the broader community. ❤️
We believe Sequent will have reputation + funding to recruit world-class teams in many areas. Our initial team knows scalable oversight, complexity + learning theory, and personas. Areas we love include agent foundations, game theory, and heuristic arguments. Please pitch more!
AI labs underinvest in theory and other principled approaches to alignment, and we will aim to fill this gap. Theory won’t be strong enough for guarantees: our goal is to combine evidence from theoretical models and empirics to increase overall confidence or find hard obstacles.
@danielmurfet I'm excited to work with you, @jesse_hoogland!
Timaeus is joining forces with @geoffreyirving and researchers from UK AISI to found Sequent Research. 1/9
Theory makes automation more likely to work: the models are great at prose math and Lean, which means significant acceleration even while most research taste comes from humans. But good automation is still hard: a single org will let us amortize the challenge across many areas.
But I just published “Automated alignment is harder than you think” (https://arxiv.org/abs/2605.06390)! Automated alignment is not the best plan! A better plan is to not build ASI yet, and the world should try hard to realise that plan. Alas, the speed of progress calls for backups.
Geoffrey Irving, previously Chief Scientist at the UK AI Security Institute, has started Sequent Research as a nonprofit pulling in alignment researchers from the former AISI Alignment Team, Timaeus, and other groups. The new outfit targets higher-confidence methods for keeping superintelligent systems aligned, mixing theoretical work with empirical experiments that the team plans to accelerate through heavy automation.
We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵
