How do you calculate a Brier score for event predictions?

Brier score = (1/N) x the sum of (forecast - outcome) squared. For forecasts of 0.8, 0.6, 0.3, 0.9 and 0.2 on events that turned out 1, 0, 0, 1 and 1, the squared errors are 0.04, 0.36, 0.09, 0.01 and 0.64, which sum to 1.14; divided by 5, the score is 0.228.

What is a good Brier score?

It depends on the events. Always forecasting 50% scores exactly 0.25, so a score above 0.25 on binary events is worse than a coin-flip forecast. Whether 0.228 is good depends on how hard the events were; the Brier skill score compares a score with a baseline on the same events.

Can I compare Brier scores across different questions?

Only with care. Scores depend on how predictable the events were: forecasting sunrise earns near 0 without skill. Comparisons are fair only on the same set of events, which is why skill scores use a baseline.

Why does the Brier score reward honest probabilities?

Because it is a proper scoring rule. Hanson (2002) explains that under a proper scoring rule an agent who maximizes expected payoff 'will honestly report' his true belief, and lists the quadratic rule (Brier, 1950) among them. If you believe 70%, reporting 70% gives an expected score of 0.21; reporting 90% gives 0.25.

Can I compute a Brier score for a prediction market?

Yes: treat each contract's price at a fixed time before resolution as the forecast. The timing rule matters. AltStreet's Kalshi study (July 2026) found that only 195 of 15,272 settled contracts had a quote within three hours of close and a spread of 10 cents or less; scoring older or wider quotes would mix placeholders into the forecast.

Does a low Brier score mean the forecaster will profit?

No. A forecaster who copies market prices gets the market's score and still pays fees and spreads on every trade. Profit requires forecasts that beat the executable price by more than costs, which the Brier score does not measure.

Brier Score

Prediction Markets & Event Contracts

Definition

The Brier score is the mean squared error of probability forecasts for binary events: for each event, subtract the outcome (1 if it happened, 0 if not) from the forecast probability, square it, and average across events. Lower is better: 0 is a perfect forecaster and 1 the worst possible. It is the quadratic scoring rule that Hanson's 2002 paper attributes to Brier (1950), and it rewards both calibration and the ability to separate events that happen from those that do not.

Why it matters

The Brier score is a standard way to check whether probability forecasts, including market prices, are any good. Because it is a proper scoring rule, it cannot be gamed by shading probabilities toward 0 or 1. Investors evaluating a forecaster, a model or a market's track record should ask for the Brier score on a stated set of events, with the time each forecast was taken. Compare it with a simple baseline on the same events, such as always 50% or the base rate. And be wary of scores reported without the event set, or of scores on prices taken far from resolution.

Common misconceptions

•Lower is better, and 0.25 is the always-50% benchmark for binary events, because each squared error is then exactly 0.25 whatever happens.
•A score means little without the events it was computed on, because easy events produce low scores without any forecasting skill.
•A good score is not a profitable strategy, because prices and fees decide trading results and the score ignores both.
•Overconfident forecasts do not improve a Brier score in expectation, because the quadratic rule is proper and honest probabilities minimize the expected score.
•A market's Brier score depends on when prices are sampled, because quotes taken long before resolution can be stale placeholders, not traded prices.

Technical details

Worked example: five forecasts

Forecasts 0.8, 0.6, 0.3, 0.9, 0.2; outcomes 1, 0, 0, 1, 1.

Squared errors 0.04, 0.36, 0.09, 0.01, 0.64; sum 1.14.

Brier score 1.14 / 5 = 0.228.

Benchmarks on the same events: always 50% scores 0.25; always the base rate of 60% scores 0.24.

Worked example: why honesty pays (illustrative)

True belief: 70% that an event occurs.

Expected score of a report r = 0.7 x (1 - r) squared + 0.3 x r squared.

Report 0.7: expected score 0.21. Report 0.8: 0.22. Report 0.9: 0.25. Report 0.5: 0.25.

The minimum is at the honest 0.7, which is what makes the rule proper.

What AltStreet's reviews and filings show

AltStreet's Kalshi study (July 23, 2026) archived 15,685 markets and 15,272 settled outcomes; 1,060 had a price captured while the contract was live, 218 had a quote within three hours of close, and 195, spread across nine series, also had a spread of 10 cents or less.

Kalshi study: one series' reference quotes were captured 4,199 minutes before settlement; any score built on them would measure the sampling schedule.

Kalshi review (August 2026): sample prices were consistent with reasonable calibration, and no exploitable edge after fees was found in 15,685 markets.

Hanson (2002): the quadratic (Brier, 1950) and logarithmic (Good, 1952) rules are both special cases of proper power-law scoring rules.

What a reported Brier score should come with

The list of events scored, chosen before the outcomes were known.

The time each forecast or price was taken, relative to resolution.

The number of events, since a score on a handful of questions is noisy.

The score of a simple baseline on the same events: always 50% and the base rate.

For market prices, the spread filter used, so placeholders are not scored as forecasts.

How investors apply the term

Investors usually apply this term by tying the definition back to the controlling documents and then asking which cash flows, rights, restrictions, or valuation assumptions it changes. A useful review looks at the term alongside fees, leverage, liquidity, reporting frequency, and manager discretion rather than treating it as a standalone concept.

The practical diligence test is simple: identify who benefits if the assumption is wrong, who has control when conditions change, and what evidence the investor receives after closing. Terms that cannot be reconciled to documents, cash movements, and monitoring data should be treated as marketing shorthand until proven otherwise.

Frequently asked questions

How do you calculate a Brier score for event predictions?: Brier score = (1/N) x the sum of (forecast - outcome) squared. For forecasts of 0.8, 0.6, 0.3, 0.9 and 0.2 on events that turned out 1, 0, 0, 1 and 1, the squared errors are 0.04, 0.36, 0.09, 0.01 and 0.64, which sum to 1.14; divided by 5, the score is 0.228.
What is a good Brier score?: It depends on the events. Always forecasting 50% scores exactly 0.25, so a score above 0.25 on binary events is worse than a coin-flip forecast. Whether 0.228 is good depends on how hard the events were; the Brier skill score compares a score with a baseline on the same events.
Can I compare Brier scores across different questions?: Only with care. Scores depend on how predictable the events were: forecasting sunrise earns near 0 without skill. Comparisons are fair only on the same set of events, which is why skill scores use a baseline.
Why does the Brier score reward honest probabilities?: Because it is a proper scoring rule. Hanson (2002) explains that under a proper scoring rule an agent who maximizes expected payoff 'will honestly report' his true belief, and lists the quadratic rule (Brier, 1950) among them. If you believe 70%, reporting 70% gives an expected score of 0.21; reporting 90% gives 0.25.
Can I compute a Brier score for a prediction market?: Yes: treat each contract's price at a fixed time before resolution as the forecast. The timing rule matters. AltStreet's Kalshi study (July 2026) found that only 195 of 15,272 settled contracts had a quote within three hours of close and a spread of 10 cents or less; scoring older or wider quotes would mix placeholders into the forecast.
Does a low Brier score mean the forecaster will profit?: No. A forecaster who copies market prices gets the market's score and still pays fees and spreads on every trade. Profit requires forecasts that beat the executable price by more than costs, which the Brier score does not measure.