Atlassian moved a 4.8-billion-point metrics pipeline toward OpenTelemetry

The migration kept the existing StatsD interface while replacing the collection, routing, aggregation and forwarding layers behind it.

1 Source, 2h ago, first seen 2h ago

Architecture diagram from Atlassian’s migration to an OpenTelemetry metrics platform.

TLDR

Atlassian engineers describe migrating a metrics platform serving about 100,000 hosts across 14 regions from gostatsd toward OpenTelemetry without forcing every service team to rewrite instrumentation first. The new collectors accept both StatsD and OTLP, allowing the platform group to replace infrastructure behind the existing interface. Atlassian says combining sidecars cut average CPU use by 3.9% per service, stream-based routing eliminated hot shards and a custom aggregation processor reduces roughly 4.8 billion datapoints per minute to about 220 million. The rollout remains incremental, with application instrumentation migration still ahead.

Atlassian is replacing the internals of a large StatsD-based metrics platform with OpenTelemetry while keeping the interface used by application teams largely unchanged. The approach turns a company-wide instrumentation rewrite into a platform migration that can move one layer at a time.

In a CNCF engineering post, Atlassian says the existing gostatsd pipeline handled metrics from roughly 100,000 hosts across 14 regions with a 99.95% service-level objective. Its limitations were architectural: UDP-only input, no common path for traces or logs, and growing duplication of features already available in the OpenTelemetry Collector ecosystem.

Keep the contract, replace the engine

Applications previously sent StatsD packets to a familiar address. Atlassian preserved that contract while replacing the collection sidecar with a custom OpenTelemetry Collector distribution that accepts both StatsD and OTLP.

The migration is divided into collection, ingest, aggregation and forwarding. Each stage gets a purpose-built collector distribution so it can be changed without forcing all services to move at once.

Combining metrics with an existing tracing sidecar reduced average CPU use by 3.9% per service across Atlassian’s most expensive Micros services, which the company describes as roughly a 30% reduction in sidecar cost at fleet scale.

Better routing flattened hot shards

The old ingest layer routed traffic by service and environment. Because a small number of large services produced disproportionate volumes, some aggregation shards became overloaded while others sat idle.

OpenTelemetry’s load-balancing exporter let the team hash by individual time-series identity instead. That kept each series on one shard while spreading a large service across the pool, producing a more even CPU distribution and better off-peak scaling.

At the aggregation stage, Atlassian says the pipeline receives about 4.8 billion datapoints per minute and writes about 220 million after reducing redundant delta metrics. The company built and open-sourced a custom delta-aggregation processor. The new aggregation tier uses about half the CPU, according to the engineering report.

The migration is still in progress. Atlassian plans to move application instrumentation itself toward OpenTelemetry SDKs after the platform can accept both old and new inputs. Its broader lesson is operational: preserve the user-facing contract, roll through lower-risk environments first and profile components under real production load rather than trusting small benchmarks.

Today's Rank

—

Not ranked yet

Combined views

368

1 Source, first seen 2h ago

Atlassian moved a 4.8-billion-point metrics pipeline toward OpenTelemetry

The migration kept the existing StatsD interface while replacing the collection, routing, aggregation and forwarding layers behind it.

1 Source, 2h ago, first seen 2h ago

TLDR

Keep the contract, replace the engine

The migration is divided into collection, ingest, aggregation and forwarding. Each stage gets a purpose-built collector distribution so it can be changed without forcing all services to move at once.

Atlassian moved a 4.8-billion-point metrics pipeline toward OpenTelemetry

TLDR

Keep the contract, replace the engine

Better routing flattened hot shards

Combined views

Atlassian moved a 4.8-billion-point metrics pipeline toward OpenTelemetry

TLDR

Keep the contract, replace the engine

Better routing flattened hot shards

1 Source

Combined views

Keep the contract, replace the engine

Better routing flattened hot shards

Keep the contract, replace the engine

Better routing flattened hot shards

Sentiment

Summary

Sentiment

Summary

Sentiment

Summary

1 Source

1 Source

Sentiment

Summary

1 Source