News and sentiment analytics

← Satellite and geospatial signals · Building an alternative data strategy →

Turning text into tradable signal: NLP pipelines behind news sentiment scores, entity resolution, event extraction, latency economics, and why most sentiment alpha lives in the details.

What sentiment products deliver

News analytics products transform unstructured text into structured observations: per-entity sentiment scores, event tags such as management changes, lawsuits or product launches, novelty scores that distinguish fresh information from syndicated echoes, and volume metrics counting how much attention an entity receives. Output arrives through APIs or batch feeds at speeds ranging from end-of-day files to millisecond pushes after publication. The buyer base spans fundamental analysts scanning shifts in coverage tone, risk teams monitoring adverse media on portfolio companies, and sub-second systematic strategies that consume machine-readable news directly inside execution infrastructure.

Products differ less in concept than in construction. Two vendors observing the same article stream can produce materially different scores because they weight sources differently, resolve entities differently and calibrate their scales over different windows. The provider landscape illustrates the range: RavenPack focuses on quantitative event and sentiment feeds for institutional strategies; GDELT and Event Registry maintain broad event indexes drawn from worldwide news; Meltwater and TVEyes grew out of media monitoring; and specialists such as LikeFolio, The Tie and MarketPsych package consumer, social and text-derived measures for trading desks.

Origins and development

Text-based investing signals long predate modern natural language processing. Clipping services, transcript summaries and manual tone scoring existed for decades before automation, and early academic work on dictionary-based sentiment measurement in financial text helped formalize the idea that word choice in disclosures and coverage correlates with subsequent outcomes. What changed in the 2010s was infrastructure: newswires became machine-readable at scale, computing costs fell enough to make statistical language models practical, and a vendor market formed to sell scored text to funds rather than to public-relations departments.

Modeling then progressed through distinct generations: hand-built lexicons of finance-specific terms, supervised classifiers trained on labeled headlines, and finally large pretrained transformer models fine-tuned on financial corpora. Each generation reduced sensitivity to negation, sarcasm and domain jargon, though none eliminated these problems entirely. The current frontier combines generative and retrieval-based methods with classical scoring pipelines, while the durable competitive assets remain what they were at the beginning: source access, reliable entity mapping and long calibration history.

Inside the NLP pipeline

The stack begins with ingestion from newswires, press-release wires, regulatory feeds, earnings-call transcripts and web sources, each carrying distinct licensing terms and latency profiles. It proceeds through entity resolution — the genuinely hard step of binding 'JPMorgan', 'JPM' and 'Chase parent' to one identifier — continues with classification and scoring, now dominated by transformer language models fine-tuned on financial text, and ends with aggregation windows and decay functions that turn article streams into time series a quantitative model can consume.

Vendor differentiation concentrates in three places. Source breadth separates one-wire products from those covering thousands of licensed and scraped outlets, including regional and non-English press. Entity-resolution precision matters because misattribution corrupts everything downstream: a story about one subsidiary contaminates the parent company's score until the error is caught. Calibration discipline determines whether a 0.7 positive score means the same thing in 2026 as it did in 2024; model upgrades that silently rebase historical scores are the sentiment-world equivalent of restated fundamentals.

Social sources add a second ingestion problem. Platforms such as Stocktwits expose retail conversation that moves faster than press coverage but arrives mixed with spam, automated accounts and coordination attempts; vendors filtering these streams publish precision characteristics of their cleaning steps, and buyers should read them. Open datasets lower the experimentation barrier considerably, which is why most serious teams prototype on open indexes before committing budget to premium feeds — a pattern common across the wider news and sentiment category.

Latency economics

Sentiment value decays with time since publication, and the decay curve splits the market into distinct price tiers. Machine-readable news delivered at millisecond latency serves latency-sensitive strategies and prices accordingly; minutes-to-hours latency serves intraday and daily horizons at a fraction of the cost; daily aggregates serve fundamental workflows nearly as well as faster tiers. Buying more speed than the rebalance frequency can use is the category's classic form of waste, and diligence should match the latency tier to the holding period of the strategy it is meant to inform.

Speed also shapes sourcing. Low-latency products depend on direct licensed wire feeds, while slower products aggregate scraped web coverage, local-language outlets and social posts. The slower tiers often see stories — regional reporting, specialist blogs, forum discussion — that reach the wires late or never, so some buyers deliberately pair a fast wire feed with a slow broad-crawl feed rather than treating the two as substitutes. The combination trades a small amount of timeliness for materially wider observational reach.

Event extraction beyond polarity

Positive-negative scores are the shallowest layer of the stack. Event taxonomies — detecting that an executive departed, a trial reached verdict, a factory resumed operation, a dividend was raised — create discrete, auditable signals that combine well with structured data: sentiment around an earnings announcement conditioned on surprise magnitude, litigation events joined to case databases, product-launch mentions aligned with hiring spikes at the launching company. Because events are categorical, they survive model upgrades better than continuous scores do and can be audited line by line against source articles. Related structured signals are catalogued under estimates, events and flows.

Event detection quality depends heavily on taxonomy design. Coarse schemas with a few dozen types are easier to validate but blur economically distinct situations; fine-grained schemas capture more nuance at the cost of inconsistent labeling across model versions and annotators. Buyers evaluating an event feed should sample labeled examples against their own reading of the underlying articles, since published precision figures are rarely comparable between vendors and are almost never computed on a buyer's own universe of entities.

How practitioners use it

Fundamental workflows use news analytics primarily for monitoring rather than prediction: alerting on adverse-media spikes at portfolio companies, tracking tone shifts in analyst and press coverage ahead of earnings, and checking whether a management narrative receives independent corroboration from reporting the company does not control. In these settings a daily aggregation window is sufficient, and the analyst treats the score as a triage device that directs human attention rather than as a standalone trading input.

Systematic workflows consume the same products very differently. Quantitative teams ingest scored events as features in cross-sectional models, typically combined with volume, novelty and source-diversity measures to suppress noise. Accumulated research practice converges on a few lessons: raw polarity alone is weak at most horizons, combinations conditioned on context are stronger, and discrete-event signals tend to be more stable over time than continuous sentiment levels. Digital-asset teams apply analogous methods to social and news streams, with Santiment among the vendors packaging that coverage for crypto markets.

Failure modes

Sentiment models stumble on negation and sarcasm far less than earlier generations did, but domain drift persists. Language describing the same fact shifts with cycles — 'headcount reductions' versus 'layoffs' — and models trained on older corpora score newer phrasing inconsistently until retrained. Source bias matters because outlet slant colors identical events differently, and echo amplification makes one story look like thirty when syndication spreads it across outlets within hours of publication.

Operational failures tend to be quieter but more damaging than linguistic ones. A silent model upgrade rebases history and invalidates back-tests; a dropped source removes coverage without notice; an entity-mapping error routes stories to the wrong ticker for weeks before anyone complains. Buyers should therefore request source lists, model-version history and recalibration notices as contractual items rather than courtesies, and should continuously monitor agreement between vendor scores and their own reading of major stories affecting covered names.

Limitations and criticism

Critics observe that widely distributed sentiment scores cannot retain much edge by construction: anything many subscribers act on simultaneously is priced rapidly, leaving durable value mainly in novel combinations, niche source coverage and disciplined execution rather than in the headline score itself. Text-based measures also inherit the biases of their corpus — English-language and large-cap tilt in many products — and sentiment expressed in media is not identical to the information content of the underlying event. These criticisms do not render the category useless, but they argue for modest priors, careful incremental attribution and continuous live validation rather than permanent confidence in any single vendor's number.