Web & Pricing Data
From AltData.wiki, The Alternative Data Encyclopedia
Web & Pricing Data is one of the categories of alternative data covered by The Alternative Data Encyclopedia: Structured collection of product-level web data: prices, availability, catalogs, reviews and promotions captured from retailer and marketplace sites on a fixed schedule. The signal answers questions official statistics cannot, such as what a specific SKU costs at a specific merchant today and how online assortment is shifting week by week.
Price scraping industrialized alongside the e-commerce boom of the 2010s, initially serving retailers and brands with competitive-pricing dashboards. The category became macro-relevant during the 2021-22 inflation wave, when academic and central-bank research demonstrated that high-frequency web prices could nowcast CPI components. Today the market spans pricing-intelligence platforms for commerce teams and data feeds sold into quantitative funds.
The signal
Structured collection of product-level web data: prices, availability, catalogs, reviews and promotions captured from retailer and marketplace sites on a fixed schedule. The signal answers questions official statistics cannot, such as what a specific SKU costs at a specific merchant today and how online assortment is shifting week by week.
Raw output consists of SKU-level snapshots recording price, list versus selling price, availability, ratings and seller identity, keyed to product identifiers and timestamps. Derived layers add price-change events, discount depth, out-of-stock rates and category price indexes constructed from matched products. Vendor platforms extend this with share-of-search, content quality scores and assortment-gap comparisons across competitors.
Why investors pay for it
Prices and availability lead reported revenue and margin: promotional intensity flags demand weakness before sales are published, and stock-out waves anticipate supply constraints. During the 2021-22 inflation surge, several central banks and research groups built web-scraped price nowcasts that moved ahead of official CPI releases. Equity analysts use the same streams to benchmark competitive pricing power and track market-share shifts among retailers and brands.
Vendors use crawlers or licensed feeds with source-specific parsers and recrawl schedules that vary by retailer and product. SKU matching can combine exact identifiers, normalized titles, and similarity models; accuracy varies by source and should be validated against labeled samples rather than assumed from a vendor headline metric. Reproducible historical analysis requires timestamped snapshots and documented rematching rules when listings change.
Who uses it
Consumer-retail and e-commerce equity analysts track price gaps, promotion cycles and assortment share; inflation researchers and macro teams consume category indexes as nowcast inputs. CPG and retail pricing teams buy the same data commercially for competitive response.
Questions to ask vendors in this category
How broad and frequent is coverage per retailer and category? What is SKU-match accuracy across merchants, and how do you handle rematching when listings change? What is your robots.txt and terms-of-service compliance posture? How far back does clean history go, and is it point-in-time? How are out-of-stocks treated in index construction?
Complementary signals
This signal pairs naturally with adjacent categories of the encyclopedia:
Caveats and limitations
Coverage and crawl frequency create survivorship-like bias because delisted products drop out of panels. A single daily snapshot can miss intraday repricing on dynamically priced marketplaces. Seller churn and geographic price variation inject noise, and category mix drives index results as much as underlying prices.
Compliance and legal considerations
Scraping sits against site terms of service and database rights, and relevant case law remains jurisdiction-dependent after disputes such as hiQ versus LinkedIn. Reputable vendors document robots.txt policies, exclude personal data from review content, and treat product imagery and copy under copyright constraints.
Further reading
Providers in this category
The register lists 31 providers and sources for this signal family:
Primers on Web & Pricing Data
- Web scraping and compliance in alternative dataHow scraped datasets are actually collected, where terms of service and computer-muse laws draw lines, what hiQ v. LinkedIn settled and did not, and the diligence checklist for legally fragile data.
- Alternative data for investingHow funds turn alternative data into an investment edge: signal families, evaluation, backtesting, compliance and the build-vs-buy decision, mapped to the datasets and providers indexed on this encyclopedia.
Alternative data providers in Web & Pricing Data
Explore provider profiles for related datasets, delivery options and further reading.
Marketplaces and hubs for Web & Pricing Data
- Databricks Marketplace (108 datasets)
- Kaggle (30 datasets)
- Snowflake Marketplace (25 datasets)