Google DeepMind researchers Susan Zhang and Alexander Doria called out selective numbers in Mistral's OCR 4 launch materials, where reported Chandra 2 scores fell below what public code and repositories actually deliver while Infinity Parser's stronger 87.6 percent result never appeared in the comparison charts at all.
it's fine, it's fine, the eu simply gets what the eu deserves
In their OCR 4 launch this week, Mistral shared a significantly lower score for Chandra 2 than you get from our repo or by running our public code.
They also omitted Infinity Parser, which reports 87.6%, from their olmocr comparison.
Enterprises evaluating document pipelines now face an extra step of cross-checking public leaderboards before trusting launch-day aggregates.
Replies on X highlight the risk that directional benchmarks become treated as definitive when competing results stay off the slide deck.
Positive users praise Chandra's recognition in PDF OCR while negative users criticize Mistral for inaccurate or self-serving OCR 4 benchmark scores.
No Digg Deeper questions have been answered for this story yet.
Vik has been doing fantastic work on PDFs for years and Datalab's suite of tools (including Marker and Chandra) carried us through multiple DARPA evaluations. Weird to see them misrepresented, but I cannot recommend Chandra highly enough if you are dealing with complex PDFs.
In their OCR 4 launch this week, Mistral shared a significantly lower score for Chandra 2 than you get from our repo or by running our public code.
They also omitted Infinity Parser, which reports 87.6%, from their olmocr comparison.
@suchenzang fine enough to compete with (*check notes*) accenture
it's fine, it's fine, the eu simply gets what the eu deserves
@suchenzang They need to have a discussion on this, clearly! It’s urgent! 🚨
@suchenzang First heatwave and then this
@suchenzang Why Mistral do this :( They have good people like Umar Jamil
@suchenzang well there are sota models coming from the eu : https://huggingface.co/lightonai/LightOnOCR-2-1B
@suchenzang Mistral benchmarks itself, finds itself excellent, declares victory. That's what the EU deserves, apparently.
@deliprao chandra is seriously underrated in the pdf space. always happy to see the team get proper recognition.
@deliprao 用过都说好这波必须无脑安利