This quote from our friends at @cogent_security says a lot:
“At Cogent, our background agents can produce a huge volume of traces all at once. We need live observability into those systems, and SmithDB has been able to deliver that experience: seeing traces in seconds instead of minutes, which is what we experienced when testing other providers.”
When building SmithDB, we focused heavily not only on query performance but also ingestion performance.
We spent a lot of time meticulously optimizing our ingestion path (batching, compressing, indexing, uploading, etc) to achieve sub-second median write latency.
This is a time it takes for data to be durably stored and become available to queries from the time the data is received.
Waiting tens of second or even minutes for your traces to show up is completely unacceptable.
