Bradley points to existing zero-data-retention APIs on Microsoft Azure.
coherent in-the-weights continual learning is the biggest open problem left for LLMs, and anthropic is hoping it never gets solved.
deploying it broadly requires open weights. the only ZDR option is on-prem. if you are a researcher who wants open models to thrive, work on it.
Users in the replies voiced strong approval for the claim that continual learning poses the biggest open challenge for LLMs.
@willccbb more on this within a ~week
coherent in-the-weights continual learning is the biggest open problem left for LLMs, and anthropic is hoping it never gets solved.
deploying it broadly requires open weights. the only ZDR option is on-prem. if you are a researcher who wants open models to thrive, work on it.
@willccbb why does it require open weights? obviously open weights would make it easier, but couldn't anthropic just provide a managed solution for continual learning where the weights don't leave their premises?
@willccbb @henrytdowling i'd guess this goes the other way and most enterprises slowly and painfully accept + adopt non-ZDR
@henrytdowling enterprises broadly need ZDR for sensitive info. already going to be a bottleneck for short term fable adoption, they’ll allow ZDR for it eventually, but they can’t for anything that stays in the weights.
@willccbb the labs have had fine-tuning APIs via Azure with ZDR for a while though (SFT)? so why not just do that, but continual learning?
coherent in-the-weights continual learning is the biggest open problem left for LLMs, and anthropic is hoping it never gets solved.
deploying it broadly requires open weights. the only ZDR option is on-prem. if you are a researcher who wants open models to thrive, work on it.
@tenobrus @willccbb @henrytdowling would bet against enterprises are just not going to be AGI pilled enough for that
@tenobrus @henrytdowling they will if they don’t have any other viable options. but many enterprises are definitely willing to accept being N months behind the absolute frontier for small enough N. it’s a competitive lever that openai and google can reach for if they lose enough deals to ant.
@willccbb BASED
Bradley points to existing zero-data-retention APIs on Microsoft Azure.
coherent in-the-weights continual learning is the biggest open problem left for LLMs, and anthropic is hoping it never gets solved.
deploying it broadly requires open weights. the only ZDR option is on-prem. if you are a researcher who wants open models to thrive, work on it.