1d ago

Nishant Balepur presented findings at ACL 2026 showing that reasoning large language models solve multiple-choice questions without the stem by exploiting option patterns

Models from OpenAI, Grok, and Anthropic all exhibit the behavior.

β€”β€”0β€”β€”
Original post

🚨 New Paper! 🚨 One of my first Ph.D. papers found that LLMs can answer multiple-choice questions without seeing the question πŸ€” At #ACL2026, I'm presenting a follow-up showing that current reasoning LLMs can still do this! And quite similarly to a clever test-taker πŸ§‘β€πŸŽ“πŸ§΅

6:02 AM Β· May 18, 2026 View on X
Reposted by