It was trained using expert-labeled data and on-policy distillation
So, we are back to 2016 after all
Bridgewater, one of the worlds largest hedge funds, a Tinker customer talks through how they've carefully fine-tuned a model focused on what makes interesting financial news. Their fine-tuned model is more effective and cheaper than any frontier model.
Many users praised Bridgewater's fine-tuned LLM for financial document filtering as clever with strong ROI from domain expertise, while a few dismissed the work as managerial overreach.
No Digg Deeper questions have been answered for this story yet.
@tinkerapi that is a clever way to handle data noise
People sometimes ask why fine-tune when general-purpose models keep getting better. Bridgewater's work is a good reminder that with the right data -- here, expert judgements -- you can beat prompting-only approaches by a lot. @ddkang and the Bridgewater AIA Labs team are great -- glad to see them sharing this.
Sorting which financial docs are worth an analyst's time is surprisingly hard for frontier LLMs. With an expert-labeled dataset and on-policy distillation, Bridgewater fine-tuned a model to do it reliably and cheaply. https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment-in-financial-tasks/
Bridgewater, one of the worlds largest hedge funds, a Tinker customer talks through how they've carefully fine-tuned a model focused on what makes interesting financial news. Their fine-tuned model is more effective and cheaper than any frontier model.
Sorting which financial docs are worth an analyst's time is surprisingly hard for frontier LLMs. With an expert-labeled dataset and on-policy distillation, Bridgewater fine-tuned a model to do it reliably and cheaply. https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment-in-financial-tasks/
Bridgewater and Thinking Machines just published a blog on training a custom model to replicate expert investor judgment. The task is filtering financial documents and news for relevance. Sounds trivial. Turns out it's not.
Sorting which financial docs are worth an analyst's time is surprisingly hard for frontier LLMs. With an expert-labeled dataset and on-policy distillation, Bridgewater fine-tuned a model to do it reliably and cheaply. https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment-in-financial-tasks/
@soumithchintala You basically created an overfitted filter ?
@soumithchintala @MartinShkreli Did not expect a mega HF to be leading the charge on token cost optimization.
@soumithchintala How does cost per task account for the added upfront cost to fine tune? At what task volume does training actually break even?
@tinkerapi Is it open source?
@johnschulman2 @ddkang very cool, a lot of potential for finetuning!
@iAmHenryMascot @tinkerapi No strong reason. This claim that frontier models fail this task 50% is not something I believe.
@tinkerapi very tuff @SarahhhSuuu
@soumithchintala cheaper because they don't have to pay 95% GM?
@tinkerapi 🧠🧠 @SarahhhSuuu
@soumithchintala Feels like a VP trying too hard for scopemaxxing!
@tinkerapi @47fucb4r8c69323 Mira’s been cooking
@cherry_mx_reds @tinkerapi Why? Passed my smell test
Frontier models were bad at this. Claude, Gemini, GPT variants averaged around 50% accuracy with a naive prompt.
Expert prompting pushed that into the mid-to-high 70s. Still short of the 80% threshold, investors need to trust a system.
Bridgewater and Thinking Machines just published a blog on training a custom model to replicate expert investor judgment. The task is filtering financial documents and news for relevance. Sounds trivial. Turns out it's not.
@fabsbz @tinkerapi This is brilliant (albeit not really groundbreaking) work. Not surprised Bridgewater hired the best.
Newer model generations barely helped. GPT 5.4 cost 43% more than 5.2 and was only marginally more accurate.
Frontier models were bad at this. Claude, Gemini, GPT variants averaged around 50% accuracy with a naive prompt.
Expert prompting pushed that into the mid-to-high 70s. Still short of the 80% threshold, investors need to trust a system.
