5 trillion tokens of the finest code data just landed on the Hub 👀
For over two years, the largest open code dataset was The Stack v2… until today. 🥞 The Stack v3 is out: the largest open code dataset ever released: 114 TB, 770 languages, 224M repositories, ~5T tokens of deduplicated and filtered source code. Fully open, no restrictively licensed code included. The upgrade: - v2 (2024): 68 TB raw -> 2 TB / ~550B tokens, 618 languages filtered - v3 (2026): 114 TB raw -> 15.9 TB / ~5T tokens, 713 languages filtered C++ x15, TypeScript x7.5, Rust x7, Python x4.8. Even the COBOL corner of GitHub got a bigger slice. Part of that is two fresh years of open source. Part of it is a bug we found in v2's deduplication - story below. 👀 Things that make v3 different: 1. Contents inline. The #1 complaint about v2 was "cool dataset, where's the actual code?" v2 shipped file IDs from the Software Heritage graph, and fetching contents was a DIY treasure hunt. v3 is self-contained: sources embedded directly, one row = one repository. Download finishes -> you start training. 2. Fresh crawl. v2 was a 2023 snapshot of even older crawls. v3 is a direct re-crawl of GitHub at the latest commit, completed by August 2025: 224M repositories, 44B files. Forks were only included if they had 5+ stars. Two ways in: 🥞 stack-v3-train - near-deduplicated, quality-filtered, PII-redacted, contents inline. Point load_dataset at it and go. https://huggingface.co/datasets/HuggingFaceCode/stack-v3-train 🏗 stack-v3-full - the entire 114 TB corpus as an HF Storage Bucket: every duplicate kept with cluster IDs, stubs for excluded files. Roll your own dedup, filters, and mixes. https://huggingface.co/buckets/HuggingFaceCode/stack-v3-full The record isn't really ours to claim - it's millions of everyone's repositories, neatly stacked. Thanks to every developer who keeps their code public, the BigCode community, and friends who built the infrastructure that made this possible!
Story time: while rebuilding dedup for v3, we found a bug in v2's original pipeline. 😬 MinHash shingles were tokenized with \W instead of \W+. A 5-gram shingle from ordinary C++ was mostly empty strings - ['', 'include', '', 'vector', ''] - and any run of 5+ non-word characters (indentation, operators) produced the all-empty shingle, shared by virtually every high-level language. With that many hash collisions, MinHash gets trigger-happy. v2 over-deduplicated by a lot. v3 fixes the regex, runs MinHash-LSH across all languages at once, and verifies every candidate pair against its Jaccard estimate. Zero false positives.