Denis Peskoff, Joe Barrow and collaborators at UC Berkeley have turned 7 million pages of municipal PDFs into a single structured corpus of 2.2 million local ordinances covering 9,239 cities and counties, now freely available on Hugging Face under a non-commercial license.
A massive legal datasets just dropped on Huggingface.
For the first time, researchers used AI to gather, run optical character recognition, process, and build a database of every law in America.
That is 2.2M laws.
LocalLaws/LOCUS-v1 - Datasets on Hugging Face.
Browser scripts pulled the documents, AI OCR converted them to Markdown, and ModernBERT models tagged each chunk for substance, function, topic, and four normative scores like opacity and enforcement discretion.
The release does not model state preemption, controlling authority, or full jurisdictional hierarchy, and another 7,000 documents will require signed researcher agreements.
Users are excited about the release of a 2.2 million US laws dataset as amazing work with massive value for machine-readable analysis and new AI tools, while some object to perceived political bias or pricing.
No Digg Deeper questions have been answered for this story yet.
Once we had processed the laws, the question was: how to analyze them.
One idea I really like is @TheStalwart's http://Havelock.ai, where he distills pairwise comparisons from an LLM + TrueSkill into a BERT model to score orality.
Paper Link: https://arxiv.org/abs/2606.19334 Dataset: https://huggingface.co/datasets/LocalLaws/LOCUS-v1 Models: https://huggingface.co/LocalLaws
And obviously a huge thank you to @modal for enabling this work.
First, the released dataset has pretty good geographic coverage, and accounts for a majority of the US population!
Everything including zoning, noise, housing codes, and whether or not you can ride an ATV without a license.
New paper: every law in America is technically public. But not really, until now!
With @DenisPeskoff at UC Berkeley, we built a corpus of ~every publicly accessibly city and county law, and released a huge chunk of it!
2.2 million laws, you're (probably) covered in it!
🧵
How do you build such a dataset when all the laws come in heterogeneous formats? To me the obvious answer is: OCR it all!
In this case, using the smallest/best OCR model we could, @LightOnIO's LightOnOCR-2-1b.
That gives us a high-quality, consistent format we can parse.
California and Florida, you need to get your shit together so people can actually understand your laws! And Ohio and West Virginia, wtf is going on with how you run people's lives??
In our case, we do the same for the laws across 4 axes: paternalism, opacity, enforcement discretion (how much leeway does the law provide), and salience.
We analyzed the 2.2 million laws and some interesting patterns emerged.
Thankfully @modal (and @charles_irl) very kindly provided a compute grant to make this work happen!
You wanna *really* see GPU's go brrr? (And CPUs, for rendering)
We were able to process all 7M pages at ~$0.30/1k, much cheaper than options like AWS Textract ($1.50/1k pages).
Our released dataset scores all 2.2 million laws along these axes, and we've released the distilled models for scoring/understanding the laws!
All distilled into @benjamin_warner/@bclavie/@antoine_chaffin/@orionweller/@answerdotai's ModernBERT.
First approach was to try and make my local GPUs go brrrr.
That worked quite well for the first million pages, but we had millions more to go, and needed to scale.
N.B. I wonder if we're the first academic citation for @TheStalwart's @Havelock_AI.
If there's a better way to cite, let me know and I'll update the preprint!
@barrowjoseph @DenisPeskoff this is incredible! i've been trying to build a similar dataset for zoning code for months. i don't think your ocr-it-all method will work for my use case, but just putting this together is awesome. i'm excited to dig into the data
as @chiefofscouts mentioned, there’s an open API that covers almost all of this, no OCR required.
a few months back I indexed the 4,281 municipalities that use municode specifically, pulled their entire code down, and then laid the ground work to start tracking changes over time with a weekly diff.
on top of that, we can pull meeting minutes from the meetings that created them and track changes in the code back to the legislators that voted for the changes in the first place. you can start to picture the possibilities with this kind of data, especially tracked live and over time.
I’d be more than happy to chat with you and the team on this because I agree with all of your foundational notions about how important a project like this is.
@trallala93 @DenisPeskoff Really hoping to release something soon for this!
@willdjthrill @DenisPeskoff Whole thing is already available for download and perusing on @huggingface !
https://huggingface.co/datasets/LocalLaws/LOCUS-v1
@iammattduff @DenisPeskoff If you give me more info about the shape of the data happy to offer advice!
@matospiso @DenisPeskoff Thanks! Was 7M pages total, I and I think the released set comes from around 2.5M pages.
@barrowjoseph @DenisPeskoff What do you think it would take to get these laws into a queryable/analyzable form? Like suppose I’m a researcher and I want to understand if a certain ordinance affects something I measure. How does one approach this?
Hi Joe, I really appreciate the LOCUS work. My question/hypothesis: Have you (or do you think the corpus/models could) look for highly similar or near-identical phrasing of specific policy topics across different jurisdictions? I’m curious whether national lobbying/model legislation efforts show up as detectable “boilerplate” or semantic clusters in the text — e.g., tenant-rights/housing-code sections that use very similar wording in many places, potentially pushed by the same industry groups under different local labels. The idea is that coordinated national pushes (real estate umbrella orgs, etc.) might produce recognizable textual patterns or low-variance language on issues like landlord obligations, eviction processes, or business regulations, even if the political framing differs by state/city. Similar to how model bills get copy-pasted at the state level, but now at scale with modern embeddings/clustering instead of just string matching. Would love your thoughts on feasibility with the LOCUS pipeline or the distilled scorers — or if you’ve already seen any signals of that kind of cross-jurisdictional textual similarity in the data.
@soaring3agl3 @DenisPeskoff Ah, 2 notes:
1. This first public release and analysis contains about a third of the collected data. 2. Wasn’t possible to get *every* municipality online. :/
