What’s new here

Federal law (USC, eCFR) has had bulk XML downloads via govinfo. State codes have sat behind commercial APIs or geo-restricted government sites. This repo scrapes state legislatures (with some states still in progress), normalises the output to a single 27-column Parquet schema, and publishes quarterly snapshots on Hugging Face under CC BY-NC 4.0. The scrapers themselves are Apache-2.0, so you can reproduce or audit every row.

What it does

The published snapshot (v2026.09.1) contains 5,081,709 sections across 308 Parquet files: state and federal statutes, the CFR (current and historical editions), state administrative regulations, court rules, state constitutions, AG opinions, agency guidance, executive orders, treaties, and session laws.

Each section carries a stable hierarchical ID, a human citation (e.g. C.R.S. § 3-1-101), the full title/chapter hierarchy, act_status (in_force, repealed, superseded, …), cross-references into the USC and CFR, and the source URL of the official government publisher. Every section appears once.

The repo also ships the ingestion pipeline: per-state HTML scrapers, bulk XML ingesters for states that publish official zips, and streaming parsers for the federal USLM XML and eCFR API. Most states use plain requests/BeautifulSoup; a handful need Selenium for JavaScript-rendered pages.

Who it’s for

Researchers building legal NLP datasets, developers building RAG pipelines over statute text, legal aid organisations that need offline access to the full corpus, and journalists or policy analysts who want bulk law without a Westlaw subscription. The data download requires no account. The scrapers are for anyone who wants to rebuild or extend coverage.

Try it

You need Python and the datasets library. No API key is required for non-commercial use.

from datasets import load_dataset
 
ds = load_dataset("vaquill/open-us-law", data_files="us_ca_statutes.parquet")
rules = load_dataset("vaquill/open-us-law", data_files="us_*_court_rules.parquet")

For a direct download, browse oss-data-us.vaquill.ai, grab the combined tarball (6.91 GB), or read the manifest. index.json and latest.json always describe the current snapshot.

To run a scraper locally:

pip install -r requirements.txt
 
# Federal: US Code (USLM XML) and the eCFR
python scripts/federal/download_usc_zips.py --help
python scripts/federal/parse_ecfr_streaming.py --help
 
# A state statutory code (Kansas, HTTP - no browser needed):
cd scripts/state_scrapers
OUT_DIR=./data python -m src.scrapers.us.states.ks.statutes.scrapeKS

Swap ks / scrapeKS for any state in the coverage table. A few state scrapers require Chrome/Chromium and chromedriver on your PATH. Some state sites only serve US traffic, so run those from a US host.

How mature is it

79 stars, 24 forks, 1 contributor, 45 commits in the last 90 days, 7 open issues, no open PRs. Created July 2026, last pushed October 12, 2026. No formal GitHub releases; versioned snapshots are published directly to Hugging Face and the Cloudflare R2 mirror on a quarterly schedule. Scrapers are Apache-2.0; the compiled dataset is CC BY-NC 4.0.