Dataset
clinicaltrials-scraper
ClinicalTrials.gov scraping tool that extracts study records based on criteria like condition, sponsor, phase, status, or geography, yielding one row per NCT identifier containing fields such as sponsors, phases, enrollment, outcomes, and locations, with no API key needed and approximately 11,191 rows and 63 fields collected over 65 runs.
Data preview
Dataset card
The clinicaltrials-scraper dataset, published by reapxdev, aggregates records scraped from the public ClinicalTrials.gov registry, with one row per NCT trial including sponsors, phases, enrolment, outcomes and locations. The corpus sits in the 10K–100K size band, is marked CC BY 4.0 for the published metadata, and tags every row with the collector run that produced it.
What this dataset contains
Each row corresponds to a single NCT trial collected from ClinicalTrials.gov by a public collector run, rather than generated from a keyword template. Coverage spans studies filterable by condition, intervention, sponsor, phase, status or country, with rows deduplicated on content across multiple runs. A browsable subset of 8,386 trials that carry an addressable nctId is published alongside the full dataset, which also includes rows whose payload cannot address an entity page.
Structure and fields
Concrete column names are not documented in the provided metadata, so the schema cannot be enumerated honestly. Rows are known to carry the trial's nctId, sponsors, phases, enrolment, outcomes and locations, plus provenance fields _run_id and _dataset_id that name the collector run which observed each record. Every row exists because a real run observed it, and 0 duplicate observations were collapsed during deduplication.
How analysts use it
Analysts can query the dataset by condition, intervention, sponsor, phase, status or country to map the competitive landscape of clinical development, tracking sponsor activity, trial geography and enrolment counts. Because each trial links to a public NCT record and carries run-level provenance, the data is suited for longitudinal tracking of pipeline additions, geographic site distribution and phase progression over time. Public-records sourcing and CC BY 4.0 licensing on the metadata support reproducible use in pipeline intelligence and event-driven healthcare research.
Documentation & references 1
Related datasets
Browse moreParsed eligibility criteria from 13,229 ClinicalTrials.gov studies representing the candidate pool gathered by two first-stage retrievers during TREC Clinical Trials 2021–2023, formatted as typed entity-relation graphs for reranking models.
The clinical-trials-patient-graphs dataset is a small tabular collection of patient-level clinical records distributed in Parquet format by the 2001jdev publisher. It organizes entities and relations extracted from trial data into structured rows spanning 2021 through 2023.
This dataset is a parsed snapshot of ClinicalTrials.gov records, distributed in Parquet format on the Hugging Face Hub under an unspecified license. It contains roughly 52,000 trial entries drawn from public registry filings.
clinical-trials-trec-qrels is a tabular relevance-judgment file distributed by the 2001jdev user on the Hugging Face Hub. It maps clinical-trial topic identifiers to NCT registry IDs with graded relevance scores.
The clinical-trials-trec-topics dataset is a small tabular collection of clinical-trial topic records distributed in Parquet format on the Hugging Face Hub. It appears to be derived from TREC clinical-trial retrieval topics, with rows partitioned by year.
SEC EDGAR 10-K & 10-Q financial statements for 2,600+ NASDAQ companies.
Discussion
Questions or observations about this dataset? Join the conversation on X.
No comments yet. Start the conversation on X.