Dataset
us-patents
US patent full text assembled by the Allen Institute from the USPTO, millions of records under ODC-BY: claims, abstracts, descriptions and metadata, in Parquet. A broad base for measuring IP intensity, technology landscapes and the direction of R&D spending.
Data preview
Dataset card
The Allen AI us-patents dataset is an ODC-BY licensed corpus of roughly 7.9 million US patent grants and applications from 1976 through 2025, cleaned and formatted for language-model pre-training. It is distributed in Parquet through the Hugging Face datasets hub and has been downloaded more than one thousand times.
What this dataset contains
The corpus aggregates approximately 7.9 million US patent documents published between 1976 and 2025, presented as one record per grant-application pair and including applications that are still pending or abandoned. Allen AI applied a sequence of preprocessing steps aimed at preserving technical content, including deduplication at the document level using 5-gram 90% Jaccard similarity, removal of boilerplate legal language and copyright assertions, and filtering of paragraphs with low English-prose probability. Design patents are excluded.
Structure and fields
The dataset is partitioned into train and validation shards totaling 7,899,341 rows and is published in Parquet format with loaders available for the datasets, Dask, Polars, and MLCroissant libraries. Each row carries four string-typed columns: corpus_id, filing_date, patent_type, and text. Patent type values observed in the sample include utility filings, while design patents have been removed upstream.
How analysts use it
Because the corpus spans nearly fifty years of US patent activity with structured identifiers and dates, it can support quantitative work such as mapping filing trends by technology area, tracking the evolution of patent types over time, or building NLP features that proxy for firm-level innovation activity. The combination of corpus_id, filing_date, and patent_type makes it straightforward to join patent records with public company identifiers for investable signals around R&D intensity or emerging technology adoption.
Documentation & references 1
Related datasets
Browse moreParsed eligibility criteria from 13,229 ClinicalTrials.gov studies representing the candidate pool gathered by two first-stage retrievers during TREC Clinical Trials 2021–2023, formatted as typed entity-relation graphs for reranking models.
The clinical-trials-patient-graphs dataset is a small tabular collection of patient-level clinical records distributed in Parquet format by the 2001jdev publisher. It organizes entities and relations extracted from trial data into structured rows spanning 2021 through 2023.
This dataset is a parsed snapshot of ClinicalTrials.gov records, distributed in Parquet format on the Hugging Face Hub under an unspecified license. It contains roughly 52,000 trial entries drawn from public registry filings.
clinical-trials-trec-qrels is a tabular relevance-judgment file distributed by the 2001jdev user on the Hugging Face Hub. It maps clinical-trial topic identifiers to NCT registry IDs with graded relevance scores.
The clinical-trials-trec-topics dataset is a small tabular collection of clinical-trial topic records distributed in Parquet format on the Hugging Face Hub. It appears to be derived from TREC clinical-trial retrieval topics, with rows partitioned by year.
SEC EDGAR 10-K & 10-Q financial statements for 2,600+ NASDAQ companies.
Discussion
Questions or observations about this dataset? Join the conversation on X.
No comments yet. Start the conversation on X.