Dataset
ClinicalTrials.gov
A snapshot of processed ClinicalTrials.gov records filtered to completed-recruitment Phase 3 and Phase 4 studies, plus trials of medical devices or behavioral interventions, totaling 156,887 trials as of March 2025, used for an agentic medical retrieval and evidence-grounding system.
Data preview
Dataset card
This repository from vkola-lab accompanies the published manuscript on an agentic retrieval-augmented generation system for medical reasoning and references ClinicalTrials.gov as an underlying data source. The download package contains the paper, supplementary material, and source code rather than a redistributable ClinicalTrials.gov extract.
What this dataset contains
The repository itself does not ship a ClinicalTrials.gov dump. It hosts the manuscript, supplementary material, and Python source code for an agentic medical RAG system, with installation notes that reference NVIDIA L40S and RTX PRO 6000 Blackwell GPUs, CUDA 13.2, and vLLM compiled from source. ClinicalTrials.gov is cited as a retrieval corpus used in the published experiments, not as an attached tabular file.
Structure and fields
Because the resource is a code and paper release, no explicit ClinicalTrials.gov column schema is published with the repo. The included scripts reference Hugging Face Transformers and Triton caches via TRANSFORMERS_CACHE, HF_HOME, HF_DATASETS_CACHE, and TRITON_CACHE_DIR, but these are environment variables for model and dataset caching rather than field definitions. Field-level documentation would need to be sourced from ClinicalTrials.gov itself or the paper's supplementary tables.
How analysts use it
An investment analyst would primarily use this repo to reproduce the authors' RAG pipeline and to benchmark its retrieval over ClinicalTrials.gov records, rather than to ingest a ready-made alternative-data file. Practically it serves as a research artefact on retrieval-augmented clinical question answering. As a direct dataset for trading signals the available facts are too thin to support specific use cases.
Documentation & references 1
Related datasets
Browse moreParsed eligibility criteria from 13,229 ClinicalTrials.gov studies representing the candidate pool gathered by two first-stage retrievers during TREC Clinical Trials 2021–2023, formatted as typed entity-relation graphs for reranking models.
The clinical-trials-patient-graphs dataset is a small tabular collection of patient-level clinical records distributed in Parquet format by the 2001jdev publisher. It organizes entities and relations extracted from trial data into structured rows spanning 2021 through 2023.
This dataset is a parsed snapshot of ClinicalTrials.gov records, distributed in Parquet format on the Hugging Face Hub under an unspecified license. It contains roughly 52,000 trial entries drawn from public registry filings.
clinical-trials-trec-qrels is a tabular relevance-judgment file distributed by the 2001jdev user on the Hugging Face Hub. It maps clinical-trial topic identifiers to NCT registry IDs with graded relevance scores.
The clinical-trials-trec-topics dataset is a small tabular collection of clinical-trial topic records distributed in Parquet format on the Hugging Face Hub. It appears to be derived from TREC clinical-trial retrieval topics, with rows partitioned by year.
SEC EDGAR 10-K & 10-Q financial statements for 2,600+ NASDAQ companies.
Discussion
Questions or observations about this dataset? Join the conversation on X.
No comments yet. Start the conversation on X.