Company Knowledge Bench introduced to test retrieval on messy company knowledge

The team behind Company Knowledge Bench says it built the benchmark from 1,000 real-world queries by people and agents. In its tests of seven retrievers, grep worked surprisingly well but was five times slower and filled the context with 40,000 tokens per query. Kapa Deep scored highest at 0.65 in about five seconds, using one-eighth as many tokens.