How do you support full-text search JSON filtering over agent traces that span up to hundreds of MBs, while keeping a median (P50) latency of 400ms?
Here’s an inside look at how we built a custom inverted index from scratch for SmithDB.
https://www.langchain.com/blog/full-text-search-in-smithdb-designing-an-inverted-index-for-object-storage
Positive users praise LangChain's SmithDB performance for agent trace search, while negative users dismiss the work as just database infrastructure with better marketing.
first in a series of technical blogs of how we build llm infra
How do you support full-text search JSON filtering over agent traces that span up to hundreds of MBs, while keeping a median (P50) latency of 400ms?
Here’s an inside look at how we built a custom inverted index from scratch for SmithDB.
https://www.langchain.com/blog/full-text-search-in-smithdb-designing-an-inverted-index-for-object-storage
We’re building SmithDB to solve the systems problems that come with agent observability.
If that kind of infrastructure work sounds interesting, we’re hiring.
Take a look at our open roles ⤵️
http://langchain.com/careers
@hwchase17 building a custom inverted index instead of reaching for postgres full-text or elasticsearch tells you exactly where the off-the-shelf tools broke down at that scale and latency target
@AlbertAnaBoss @hwchase17 Exactly right
@AlbertAnaBoss @hwchase17 If that how you solved it?
@LangChain 400ms P50 over hundreds-of-MB agent traces is the part to watch. Agent observability stops being useful fast if searching traces feels like digging through logs after the incident.
@LangChain It's just a simple inverted index" – famous last words before the repo hits 10k stars.
it’s actually so cool to work at LangChain the…database company (??)
yup, the cracked team that built SmithDB is doing a cool blog series on “How to build the internals of a database” —> for agent scale
A lot of thought goes into design decisions so that we can scale to agentic workloads that will far exceed the quantity of data anyone has needed to process in history
Understanding Trace data at scale is going to be crucial to Continual Learning and broadly figuring out what our agents have been doing
and it’s fun to read how db engineers solve this after prompting and looking at loss/reward graphs all day :)
How do you support full-text search JSON filtering over agent traces that span up to hundreds of MBs, while keeping a median (P50) latency of 400ms?
Here’s an inside look at how we built a custom inverted index from scratch for SmithDB.
https://www.langchain.com/blog/full-text-search-in-smithdb-designing-an-inverted-index-for-object-storage
@hwchase17 This is what I am working on. Thanks
@hwchase17 the 400ms p50 over hundreds of MB traces is the real flex here
wait this is internal or client-facing?
@hwchase17 building a custom inverted index instead of reaching for elasticsearch on hundred-MB agent traces is the kind of decision that only makes sense after you've been burned by the alternative
@AlbertAnaBoss @hwchase17 I am experimenting as we speak
@hwchase17 Curious if this covers inference stack or training pipeline — both are bottleneck layers right now.
@Vtrivedy10 Agent infrastructure is just database work with better marketing. Traces, filters, latency, retention: the boring internals decide whether the magic scales or becomes a support ticket.
@hwchase17 read the reverse chronology part first. curious if yall tried any doc-store tricks before settling on the inverted index
@hwchase17 damn that P50 constraint at those trace sizes is nasty. built from scratch custom inverted index is the kind of flex most teams just skip
@hwchase17 its fascinating how these decisions ripple through stack.
How do you support full-text search JSON filtering over agent traces that span up to hundreds of MBs, while keeping a median (P50) latency of 400ms?
Here’s an inside look at how we built a custom inverted index from scratch for SmithDB.
https://www.langchain.com/blog/full-text-search-in-smithdb-designing-an-inverted-index-for-object-storage