ESM project founder Alex Rives releases ESMFold2, claiming it beats AlphaFold3 on protein-protein interaction benchmarks
Lab tests validated binders targeting five therapeutic proteins.
#4Yann LeCun@YLECUN@alexrives Amazing 👏 Congrats to you and the ESM team.
Alex Rives@alexrivesToday we're announcing ESMFold2, an open scientific engine to power prediction, design, and discovery across protein biology. The new model delivers state of the art performance on protein interactions, especially antibodies, a critical modality for therapeutics. We have designed and validated miniprotein binders and single chain antibodies across five therapeutic targets that are important in cancer and immunology. We are seeing very high success rates, and affinities at levels consistent with therapeutic activity. We’re also releasing an atlas of 6.8 billion proteins, and 1.1 billion predicted structures. ESMFold2 is built on a state of the art language model that has been trained on billions of protein sequences. A world model of protein biology emerges through language modeling. We’ve used the techniques of mechanistic interpretability developed to understand large language models to understand the concepts ESM uses to represent proteins. The model’s representation space has a compositional organization of features across scales, levels of complexity, and abstraction, that reflects and mirrors the understanding of protein biology developed through a century of empirical science. This understanding emerges without prior knowledge, just from language modeling of protein sequences. Language models are becoming a powerful substrate to understand and program biology. The design of protein interactions is one of the most fundamental problems in biophysics, and has critical implications for the discovery of new medicines. A simple gradient based search with the model was able to discover high-affinity protein binders. I'm excited by the potential this has to accelerate basic science and the understanding of proteins. And especially for the new avenues it opens up for therapeutic design and medicine.
@alexrives @ylecun Congrats!!
Alex Rives@alexrivesToday we're announcing ESMFold2, an open scientific engine to power prediction, design, and discovery across protein biology. The new model delivers state of the art performance on protein interactions, especially antibodies, a critical modality for therapeutics. We have designed and validated miniprotein binders and single chain antibodies across five therapeutic targets that are important in cancer and immunology. We are seeing very high success rates, and affinities at levels consistent with therapeutic activity. We’re also releasing an atlas of 6.8 billion proteins, and 1.1 billion predicted structures. ESMFold2 is built on a state of the art language model that has been trained on billions of protein sequences. A world model of protein biology emerges through language modeling. We’ve used the techniques of mechanistic interpretability developed to understand large language models to understand the concepts ESM uses to represent proteins. The model’s representation space has a compositional organization of features across scales, levels of complexity, and abstraction, that reflects and mirrors the understanding of protein biology developed through a century of empirical science. This understanding emerges without prior knowledge, just from language modeling of protein sequences. Language models are becoming a powerful substrate to understand and program biology. The design of protein interactions is one of the most fundamental problems in biophysics, and has critical implications for the discovery of new medicines. A simple gradient based search with the model was able to discover high-affinity protein binders. I'm excited by the potential this has to accelerate basic science and the understanding of proteins. And especially for the new avenues it opens up for therapeutic design and medicine.
ESMFold2 is blazing fast and has state of the art accuracy across benchmarks for structure prediction, especially for the challenging problem of predicting protein interactions, including the interaction of antibodies with their targets.
Alex Rives@alexrivesToday we're announcing ESMFold2, an open scientific engine to power prediction, design, and discovery across protein biology. The new model delivers state of the art performance on protein interactions, especially antibodies, a critical modality for therapeutics. We have designed and validated miniprotein binders and single chain antibodies across five therapeutic targets that are important in cancer and immunology. We are seeing very high success rates, and affinities at levels consistent with therapeutic activity. We’re also releasing an atlas of 6.8 billion proteins, and 1.1 billion predicted structures. ESMFold2 is built on a state of the art language model that has been trained on billions of protein sequences. A world model of protein biology emerges through language modeling. We’ve used the techniques of mechanistic interpretability developed to understand large language models to understand the concepts ESM uses to represent proteins. The model’s representation space has a compositional organization of features across scales, levels of complexity, and abstraction, that reflects and mirrors the understanding of protein biology developed through a century of empirical science. This understanding emerges without prior knowledge, just from language modeling of protein sequences. Language models are becoming a powerful substrate to understand and program biology. The design of protein interactions is one of the most fundamental problems in biophysics, and has critical implications for the discovery of new medicines. A simple gradient based search with the model was able to discover high-affinity protein binders. I'm excited by the potential this has to accelerate basic science and the understanding of proteins. And especially for the new avenues it opens up for therapeutic design and medicine.
We’re also releasing the largest atlas of protein structure and function. With 6.8 billion proteins, and 1.1 billion predicted structures, it enables analysis and protein discovery at the scale of life.
Alex Rives@alexrivesWe designed miniproteins and antibodies for five targets that are important in cancer and immunology. We found binders with nanomolar affinities for all, and sometimes picomolar affinities, testing just 84 designs per target and modality. Scaling compute at inference time improved average success rates from 54% to 70% for minibinders and from 12% to 21% for single chain antibodies. We’ve obtained biophysical validation for the binders including a cryo-EM structure of one of the miniprotein binders in complex with EGFR, which confirms that the predicted structure is within experimental accuracy of the true structure.
At @Biohub, our goal is to build models that accelerate scientific discovery and progress toward the cure to disease. We’re releasing all of this under MIT license allowing commercial and non-commercial use.
Read more here: https://biohub.ai/esm/protein/
Alex Rives@alexrivesWe’re also releasing the largest atlas of protein structure and function. With 6.8 billion proteins, and 1.1 billion predicted structures, it enables analysis and protein discovery at the scale of life.
@ylecun Thank you Yann!
Yann LeCun@ylecun@alexrives Amazing 👏 Congrats to you and the ESM team.
@pdhsu @ylecun Thank you Patrick!!
Patrick Hsu@pdhsu@alexrives @ylecun Congrats!!
@BoWang87 Thank you Bo!!
Bo Wang@BoWang87this is super cool! Amazing progress in STOA protein biology! Congrats @alexrives !
#1661Bo Wang@BOWANG87this is super cool! Amazing progress in STOA protein biology! Congrats @alexrives !
Alex Rives@alexrivesToday we're announcing ESMFold2, an open scientific engine to power prediction, design, and discovery across protein biology. The new model delivers state of the art performance on protein interactions, especially antibodies, a critical modality for therapeutics. We have designed and validated miniprotein binders and single chain antibodies across five therapeutic targets that are important in cancer and immunology. We are seeing very high success rates, and affinities at levels consistent with therapeutic activity. We’re also releasing an atlas of 6.8 billion proteins, and 1.1 billion predicted structures. ESMFold2 is built on a state of the art language model that has been trained on billions of protein sequences. A world model of protein biology emerges through language modeling. We’ve used the techniques of mechanistic interpretability developed to understand large language models to understand the concepts ESM uses to represent proteins. The model’s representation space has a compositional organization of features across scales, levels of complexity, and abstraction, that reflects and mirrors the understanding of protein biology developed through a century of empirical science. This understanding emerges without prior knowledge, just from language modeling of protein sequences. Language models are becoming a powerful substrate to understand and program biology. The design of protein interactions is one of the most fundamental problems in biophysics, and has critical implications for the discovery of new medicines. A simple gradient based search with the model was able to discover high-affinity protein binders. I'm excited by the potential this has to accelerate basic science and the understanding of proteins. And especially for the new avenues it opens up for therapeutic design and medicine.
#1675Anshul Kundaje@ANSHULKUNDAJEBest part about all the prior, current & ongoing great work from @alexrives and @biohub
Alex Rives@alexrivesAt @Biohub, our goal is to build models that accelerate scientific discovery and progress toward the cure to disease. We’re releasing all of this under MIT license allowing commercial and non-commercial use. Read more here: https://biohub.ai/esm/protein/