Job market data: datasets and providers
From AltData.wiki, The Alternative Data Encyclopedia | Primer | 4 min read
Reviewed
Compare job-posting APIs, free hiring indexes and official labour statistics. Learn what each measures and how to test coverage, duplicates and historical timing.
Job postings, vacancies and hires are different measures
Job market data describes advertised roles, unfilled positions, employment or worker movements. Choose the measure that matches the question. Use individual postings to study a company's advertised skills and locations, aggregate indexes to track hiring trends, and employment statistics to study jobs actually held. An advertisement is evidence of recruiting activity, not proof that a person was hired.
BLS defines JOLTS openings as unfilled positions on the last business day of the month that meet specific conditions, including active external recruitment and the ability to start within 30 days. Hires are additions to payroll during the month. A website posting count follows neither definition automatically: one advertisement can cover several openings, and recruitment can happen without an online advertisement.[1]
Free sources for aggregate labour-market research
Indeed Hiring Lab publishes job-posting indexes as CSV files under CC BY 4.0. The documentation describes daily observations refreshed weekly, a February 2020 baseline, and country, sector and regional series where available. These files are useful for comparing hiring conditions across markets. They do not contain individual advertisements or a complete company-level jobs database.[2]
Pair that index with official labour statistics when the question concerns the wider economy. Align the reference periods and account for seasonal adjustment before comparing the series. Indeed documents changes to its adjustment method and historical revisions, so a CSV fetched today may differ from one available during an earlier investment decision. Preserve dated copies if that distinction affects the analysis.[2]
Company-level job-posting APIs and datasets
TheirStack offers job search through an API and bulk datasets. Its published plans distinguish historical delivery, ongoing updates and a combination of both; dataset fields include job attributes and hiring-company information. That makes the job-postings record a starting point for examining available samples before testing your own company universe.[3]
LinkUp describes daily collection directly from employer websites. Its published schema includes dates posted, last checked, updated and removed, alongside job and company identifiers. It also lists security identifiers in company records. Those are useful evaluation criteria, but a vendor's description is not independent evidence that its coverage or deduplication is best for every buyer.[4]
Use the jobs and workforce provider register to extend the shortlist. Compare the same geography, observation period and employer list across candidates. A headline record count mixes history, coverage and collection rules; it cannot by itself establish which feed will answer your question more accurately.
Which fields should a sample contain?
Request a sample that includes active and removed roles, multiple locations and employers of different sizes. The following fields make common collection errors visible:
- Identity: stable job identifier, source URL, employer identifier and company domain.
- Timing: stated posting date, first observed time, last observed time, removal detection time and delivery time, with definitions for each.
- Job content: original title, description, location and any normalised occupation, seniority or skill labels.
- Change handling: whether a repost receives a new identifier, how duplicates are linked and how corrected records are delivered.
These are requested evaluation fields, not a claim that every product supplies them. Check actual populated values, not just column names in a data dictionary. A posting date copied from a website and the first time a vendor observed that posting can differ materially.
A worked example: apparent growth caused by duplicate listings
Consider an illustrative employer with 20 distinct advertised roles in September. In October a collector finds 30 records: the same 20 roles plus 10 copies on a newly covered job board. The raw series shows 50% growth even though the underlying advertised roles have not changed. This is a collection change, not evidence of business expansion.
Inspect matching employer domains, titles, locations and descriptions before deduplicating. Keep enough source detail to audit ambiguous matches. Then compare a stable-source series with the full-coverage series and separately count genuinely new roles. Even a correctly deduplicated total is not a count of hires: roles may be cancelled, left open or filled by more than one person.
Test history and employer coverage before buying
Choose an employer list before examining the sample, including companies with few postings and companies that later disappeared. Measure coverage by employer and period, then inspect collection breaks around website migrations and changes in job-board access. A file that contains only today's employers or currently active jobs is inadequate for many historical studies.
For backtesting, ask when each record became available to a customer and whether a historical export includes later additions or corrected labels. Follow the point-in-time procedure, then compare performance using realistic availability dates. Hiring activity can help investigate expansion or cost control; its predictive value and timing must be tested for the particular question rather than assumed from the data category.
Choose the delivery model for the workload
Use aggregate downloads for regional or sector trends. For a narrow company list, test a searchable API with explicit filters and pagination. For repeated analysis of a broad history, compare a bulk snapshot plus updates against the cost and runtime of repeated API retrievals. TheirStack publishes separate API-credit and bulk-dataset options; LinkUp describes raw datasets and custom feeds.[3][4]
Include historical access, update delivery, deleted or closed records and permitted uses in the quote. The pricing guide explains how to compare billing units. The vendor-evaluation guide covers the wider purchasing decision. Start with the smallest extract that can demonstrate whether the data answers the question.
Frequently asked questions
- What is job market data in alternative data?
- It includes advertised roles, vacancies, employment and worker movements. These measure different things: a job advertisement shows recruiting activity, while a hire is an addition to payroll.
- Which provider has the cleanest job-posting data?
- There is no universal winner established here. Compare samples for the same employers, dates and countries, checking source coverage, duplicates, identifiers and detection times.
- Can I get job data for free?
- Indeed Hiring Lab provides aggregate posting indexes under CC BY 4.0, and BLS publishes official labour statistics. Neither is a free company-level export of all job advertisements.
- Does job data cover private companies?
- It can, when the collector observes their recruiting sources. Coverage depends on the employer, country and collection period; private-company status does not guarantee inclusion.
- Why do two job datasets disagree on the same company?
- They can cover different sources, observe pages at different times and apply different rules for duplicates, reposts and removed roles. Check these definitions before comparing counts.