Dataset
OnchainPlanBench-Seed
OnchainPlanBench Seed OnchainPlanBench Seed is an early CodePit dataset for evaluating whether small open-weight models can critique, repair, reject, or approve Web3 AI-agent action plans before wallet execution. This is not a production safety benchmark yet. It is a public seed release for the first CodePit official model track: CodePit PlanGuard. What It Tests Each row gives a model: user intent
Data preview
Dataset card
OnchainPlanBench-Seed is a small public seed dataset published by CodePit for evaluating whether small open-weight models can critique, repair, reject, or approve Web3 AI-agent action plans before wallet execution. It is explicitly described as an early, not-yet-production benchmark for the PlanGuard model track.
What this dataset contains
The dataset covers a narrow launch domain of Web3 AI-agent action plans that are reviewed prior to on-chain execution. Each row pairs an input plan with an expected strict-JSON decision output. The release is tagged for text-generation and text-classification tasks and is sized at fewer than one thousand examples.
Structure and fields
The publisher does not document a formal column schema. From the readme, each row is expected to be a single JSON object containing the model input plan and a decision payload, evaluated by the bundled script `evaluation/evaluate_outputs.py` for structural JSON validity and simple label-based expectations. No sample rows or train/validation/test splits are provided in the source facts.
How analysts use it
The dataset is positioned as a research and benchmarking artifact rather than an operational alternative-data feed, and its tags (web3, agents, tool-use, function-calling) point to AI-evaluation workflows rather than investment signals. Any investment-fund application, such as screening AI-agent risk in crypto trading flows, would be premature given the seed-level size and the publisher's own caveat that this is not a production safety benchmark.
Documentation & references 1
Related datasets
Browse moreHumanoid On-Chain Interaction Events This dataset captures interactions between humanoid agents and blockchain systems. Designed to bridge humanoid intelligence with on-chain logic and decentralized coordination. Contents Transaction intent Smart contract interaction On-chain result Use Cases Autonomous agents On-chain AI behavior Web3 coordination Part of Humanoid Network (HAN) License MIT
Datamule, Teraflop AI, and Eventual jointly contributed a 590 gigabyte SEC EDGAR collection encompassing about 8 million filings and 43 billion tokens, assembled using the datamule-python library and the official datamule API.
Most recent raw data and token balance snapshots from the Arbitrum network.
This entry provides unprocessed base data.
A collection of unprocessed Ethereum blockchain records.
This entry contains unprocessed Optimism blockchain data.
Discussion
Questions or observations about this dataset? Join the conversation on X.
No comments yet. Start the conversation on X.