Many users praised Perplexity for open-sourcing the WANDR benchmark as a smart, transparent move that gives the community a concrete standard for evaluating research agents.
Based on 22 visible X reactions from 52 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@perplexity_ai The dual utility of WANDR as both an evaluation harness and a training data factory is brilliant.
via X@AravSrinivas Great move by Perplexity to open source WANDR for the benefit of the wider research community
via X@AravSrinivas Perplexity is 100% not the best on cost. You're literally reselling models. What are you saying?
via X@AravSrinivas this is a game changer for open source research, nice work.
via X@AravSrinivas I love perplexity! Great work always!
via X@AravSrinivas You lost the battle btw
via Xtoday we're open-sourcing an eval/RL environment for measuring agentic search performance. importantly, these environments were synthesized from production traces, offering a real-world distribution, with weak human supervision. internally, we've been using these RL environments to train capable models (more on that soon), and we're scaling this paradigm to other domains and tasks, drawing on use cases from our products to cover the entire knowledge work distribution. more details here: https://github.com/perplexityai/wandr/blob/main/reference/wandr_paper.pdf
We’re open sourcing WANDR. WANDR is an internal benchmark we built and used for building deep and wide research capabilities inside Perplexity Computer. https://research.perplexity.ai/articles/wandr-benchmark-evaluating-research-agents-that-must-search-wide-and-deep
The benchmark was synthesized from real-world user search traces.
@AravSrinivas I love perplexity! Great work always!
via X@AravSrinivas You lost the battle btw
via Xtoday we're open-sourcing an eval/RL environment for measuring agentic search performance. importantly, these environments were synthesized from production traces, offering a real-world distribution, with weak human supervision. internally, we've been using these RL environments to train capable models (more on that soon), and we're scaling this paradigm to other domains and tasks, drawing on use cases from our products to cover the entire knowledge work distribution. more details here: https://github.com/perplexityai/wandr/blob/main/reference/wandr_paper.pdf
We’re open sourcing WANDR. WANDR is an internal benchmark we built and used for building deep and wide research capabilities inside Perplexity Computer. https://research.perplexity.ai/articles/wandr-benchmark-evaluating-research-agents-that-must-search-wide-and-deep
Perplexity has the best (both on cost and performance) deep and wide research harness in Computer. One of the contributing factors is strong internal evals and benchmarks. Today, we're open-sourcing WANDR, the benchmark we use internally for measuring research capabilities.
We’re open sourcing WANDR. WANDR is an internal benchmark we built and used for building deep and wide research capabilities inside Perplexity Computer. https://research.perplexity.ai/articles/wandr-benchmark-evaluating-research-agents-that-must-search-wide-and-deep
the companies that figure out how to capture their value propositions into evals and envs will win. the ability to create high quality evaluations is a new type of strategic advantage, and it is more important than access to capital, economies of scale, network effects, or any other traditional competitive advantage
We’re open sourcing WANDR. WANDR is an internal benchmark we built and used for building deep and wide research capabilities inside Perplexity Computer. https://research.perplexity.ai/articles/wandr-benchmark-evaluating-research-agents-that-must-search-wide-and-deep
Wide Research is now available in Perplexity Agent API
Narrow search is essentially solved. That's why simple browse/search benchmarks are so saturated. The frontier is wide research: find every qualifying result and back each one with evidence. Today we launched WANDR, a 500-task benchmark built on real knowledge work. It's difficult even for today's most powerful models. Perplexity Agent API's Search as Code architecture excels on WANDR because it allows the model to design the research once and then deterministically execute it at scale without overwhelming the model’s context. Using wide research in your existing agent research workflows? Try out our wide-research preset in Agent API: https://docs.perplexity.ai/docs/agent-api/wide-research WANDR Research Article: https://research.perplexity.ai/articles/wandr-benchmark-evaluating-research-agents-that-must-search-wide-and-deep Github Repo: https://github.com/perplexityai/wandr/
Many users praised Perplexity for open-sourcing the WANDR benchmark as a smart, transparent move that gives the community a concrete standard for evaluating research agents.
Based on 22 visible X reactions from 52 accounts; directional sample.
Ask a question below.
Published answers will appear here.
Perplexity has the best (both on cost and performance) deep and wide research harness in Computer. One of the contributing factors is strong internal evals and benchmarks. Today, we're open-sourcing WANDR, the benchmark we use internally for measuring research capabilities.
We’re open sourcing WANDR. WANDR is an internal benchmark we built and used for building deep and wide research capabilities inside Perplexity Computer. https://research.perplexity.ai/articles/wandr-benchmark-evaluating-research-agents-that-must-search-wide-and-deep
the companies that figure out how to capture their value propositions into evals and envs will win. the ability to create high quality evaluations is a new type of strategic advantage, and it is more important than access to capital, economies of scale, network effects, or any other traditional competitive advantage
We’re open sourcing WANDR. WANDR is an internal benchmark we built and used for building deep and wide research capabilities inside Perplexity Computer. https://research.perplexity.ai/articles/wandr-benchmark-evaluating-research-agents-that-must-search-wide-and-deep
Wide Research is now available in Perplexity Agent API
Narrow search is essentially solved. That's why simple browse/search benchmarks are so saturated. The frontier is wide research: find every qualifying result and back each one with evidence. Today we launched WANDR, a 500-task benchmark built on real knowledge work. It's difficult even for today's most powerful models. Perplexity Agent API's Search as Code architecture excels on WANDR because it allows the model to design the research once and then deterministically execute it at scale without overwhelming the model’s context. Using wide research in your existing agent research workflows? Try out our wide-research preset in Agent API: https://docs.perplexity.ai/docs/agent-api/wide-research WANDR Research Article: https://research.perplexity.ai/articles/wandr-benchmark-evaluating-research-agents-that-must-search-wide-and-deep Github Repo: https://github.com/perplexityai/wandr/