What does it mean for a prediction market to be well calibrated?

That its prices, read as probabilities, match how often things happen. If contracts priced near 30 cents resolve yes about 30% of the time, and so on across the range, the market is calibrated. The CFTC notes prediction markets 'can sometimes forecast event outcomes better than polling or other forms of forecasting.'

Are prediction markets calibrated?

In AltStreet's Kalshi study, 'none of nine series covering 195 settled contracts showed statistically significant calibration error' after restricting to quotes within three hours of settlement and spreads of 10 cents or less. The study stresses this is a small, liquid subsample with low power, and that larger academic studies report a favorite-longshot bias.

Is a calibrated forecast a good forecast?

Not necessarily. A forecaster who always predicts the long-run base rate can be perfectly calibrated and still say nothing useful about individual events. Calibration is one part of forecast quality; the Brier score combines it with sharpness, the ability to separate events that happen from events that do not.

How many events do I need to test calibration?

More than many claims use. If 50 contracts priced at 30 cents produce 18 yes outcomes (36%), the standard error is about 6.5 points, so the gap is within one standard error and proves nothing. The same 36% from 500 contracts is about 2.9 standard errors from 30% and would be a real signal.

Why might market prices be biased at the extremes?

Wolfers and Zitzewitz (NBER Working Paper 12200, 2006) show that prices 'typically provide useful (albeit sometimes biased) estimates of average beliefs,' and that wealth constraints and risk attitudes can bias prices toward 50%, implying 'a favorite-longshot bias, in which longshots win less often than might be expected given the prediction market price.'

Why does the time of the price matter?

Because a quote taken long before the event may not be a real price. AltStreet's Kalshi study found one series whose reference quotes were captured 4,199 minutes before settlement, during a dormant window with placeholder quotes; a calibration test on those quotes would measure the sampling schedule, not the market.

Forecast Calibration

Prediction Markets & Event Contracts

Definition

Calibration measures whether stated probabilities match observed frequencies. A forecaster, or a market, is well calibrated if, among all events given about a 70% probability, about 70% happen, and likewise at every probability level. Calibration is tested by grouping forecasts into probability bins and comparing each bin's average forecast with the share of events that occurred.

Why it matters

Calibration is the test behind claims that market prices are probabilities, and it is easy to report a flattering version. Ask how a calibration claim was measured: which markets, how many events in each bin, at what point before resolution, and with what spread filter. AltStreet's own study shows how large the gap can be between the headline sample and the testable one: 15,272 settled outcomes became 195 usable contracts. Headline accuracy claims often rest on liquid markets close to resolution, which say little about prices weeks earlier. And a calibrated market is a fair starting point for a trader, not a source of profit.

Common misconceptions

•Calibrated is not the same as accurate on each event, because a forecaster who always states the base rate can be calibrated while never distinguishing one event from another.
•Small samples cannot detect modest biases, because with 50 events per bin the standard error near 30% is about 6.5 points, larger than many reported biases.
•Calibration measured near resolution says little about prices weeks earlier, because quotes far from the event can be stale placeholders rather than traded prices.
•A null result is not proof of efficiency, because AltStreet's 195-contract test had too little power to rule out the favorite-longshot bias larger studies report.
•A calibrated market does not make trading profitable, because a trader who matches a calibrated price still pays fees and spreads on every trade.

Technical details

Testing calibration (illustrative)

Group forecasts into bins: 0 to 10%, 10 to 20%, and so on.

For each bin, compare the average forecast with the share that happened.

A calibrated forecaster's points lie near the diagonal.

Report the number of events per bin; bins with few events are noisy.

Worked example: when a gap is a signal (illustrative)

Bin at 30%: 50 contracts, 18 resolve yes, 36%.

Standard error: square root of (0.30 x 0.70 / 50) = about 0.065, or 6.5 points.

Gap: 6 points, about 0.9 standard errors: not evidence of miscalibration.

Same 36% from 500 contracts: standard error about 2.0 points, gap about 2.9 standard errors: a real signal.

What AltStreet's reviews and filings show

AltStreet's Kalshi study (July 23, 2026) archived 15,685 markets and 15,272 settled outcomes; 1,060 had a price captured while the contract was live, 218 had a quote within three hours of close, and 195, spread across nine series, also had a spread of 10 cents or less.

Kalshi study: none of the nine series showed statistically significant calibration error in that 195-contract sample.

Kalshi study: one series looked miscalibrated at z = -3.91 across 109 contracts; with collection extended to 170 contracts and filters applied it decayed to z = +0.28, a sampling instability, not an edge.

Kalshi review (August 2026): prices 'consistent with reasonable calibration,' while independent work on 300,000+ contracts reports low-price contracts underperform after fees.

How investors apply the term

Investors usually apply this term by tying the definition back to the controlling documents and then asking which cash flows, rights, restrictions, or valuation assumptions it changes. A useful review looks at the term alongside fees, leverage, liquidity, reporting frequency, and manager discretion rather than treating it as a standalone concept.

The practical diligence test is simple: identify who benefits if the assumption is wrong, who has control when conditions change, and what evidence the investor receives after closing. Terms that cannot be reconciled to documents, cash movements, and monitoring data should be treated as marketing shorthand until proven otherwise.

Frequently asked questions

What does it mean for a prediction market to be well calibrated?: That its prices, read as probabilities, match how often things happen. If contracts priced near 30 cents resolve yes about 30% of the time, and so on across the range, the market is calibrated. The CFTC notes prediction markets 'can sometimes forecast event outcomes better than polling or other forms of forecasting.'
Are prediction markets calibrated?: In AltStreet's Kalshi study, 'none of nine series covering 195 settled contracts showed statistically significant calibration error' after restricting to quotes within three hours of settlement and spreads of 10 cents or less. The study stresses this is a small, liquid subsample with low power, and that larger academic studies report a favorite-longshot bias.
Is a calibrated forecast a good forecast?: Not necessarily. A forecaster who always predicts the long-run base rate can be perfectly calibrated and still say nothing useful about individual events. Calibration is one part of forecast quality; the Brier score combines it with sharpness, the ability to separate events that happen from events that do not.
How many events do I need to test calibration?: More than many claims use. If 50 contracts priced at 30 cents produce 18 yes outcomes (36%), the standard error is about 6.5 points, so the gap is within one standard error and proves nothing. The same 36% from 500 contracts is about 2.9 standard errors from 30% and would be a real signal.
Why might market prices be biased at the extremes?: Wolfers and Zitzewitz (NBER Working Paper 12200, 2006) show that prices 'typically provide useful (albeit sometimes biased) estimates of average beliefs,' and that wealth constraints and risk attitudes can bias prices toward 50%, implying 'a favorite-longshot bias, in which longshots win less often than might be expected given the prediction market price.'
Why does the time of the price matter?: Because a quote taken long before the event may not be a real price. AltStreet's Kalshi study found one series whose reference quotes were captured 4,199 minutes before settlement, during a dormant window with placeholder quotes; a calibration test on those quotes would measure the sampling schedule, not the market.