AI vs Market Scoreboard

Name: VoxOdds AI vs Market Forecast Scoreboard
Creator: VoxOdds
License: https://creativecommons.org/licenses/by/4.0/

Every hourly AI forecast is stored with the market price captured at the same moment, then scored when the market resolves. Append-only receipts, deterministic math, losses published. Lower Brier is better. Read the full research writeup → Submit your own forecasts →

Forecasts scored

5575

10148 awaiting resolution

AI Brier (mean)

0.199

sample: 5575

Market Brier (mean)

0.144

same timestamps

AI accuracy

71%

market: 79%

Across 5575 scored forecasts, VoxOdds AI (Brier 0.199) trailed the market price (Brier 0.144) taken at the same moment. A sweep over every AI/market mixture weight (n=5575) finds no weight of AI improves on the price — the market alone is optimal so far.

Recently scored forecasts

Market	AI said	Resolved	AI Brier	Market Brier	Verdict
Will Kai and Speed beat the Minecraft challenge by August 17?	Yes @ 97%	Yes	0.001	0.001	AI closer
Will Kai and Speed beat the Minecraft challenge by August 17?	Yes @ 93%	Yes	0.004	0.001	Market closer
Will Kai and Speed beat the Minecraft challenge by August 17?	Yes @ 89%	Yes	0.013	0.001	Market closer
Will Kai and Speed beat the Minecraft challenge by August 17?	Yes @ 81%	Yes	0.037	0.001	Market closer
Will Kai and Speed beat the Minecraft challenge by August 17?	Yes @ 65%	Yes	0.122	0.001	Market closer
Will Kai and Speed beat the Minecraft challenge by August 12?	Yes @ 95%	Yes	0.003	0.002	Market closer
Will Kai and Speed beat the Minecraft challenge by August 12?	No @ 14%	Yes	0.021	0.002	Market closer
Will Kai and Speed beat the Minecraft challenge by August 12?	No @ 28%	Yes	0.081	0.002	Market closer
Will Kai and Speed beat the Minecraft challenge by August 12?	No @ 52%	Yes	0.270	0.002	Market closer
Will Kai and Speed beat the Minecraft challenge by August 12?	No @ 99%	Yes	0.980	0.002	Market closer
Colorado Rockies vs. Arizona Diamondbacks	Colorado Rockies @ 90%	Colorado Rockies	0.010	0.009	Market closer
Colorado Rockies vs. Arizona Diamondbacks	Arizona Diamondbacks @ 15%	Colorado Rockies	0.021	0.009	Market closer
Colorado Rockies vs. Arizona Diamondbacks	Arizona Diamondbacks @ 22%	Colorado Rockies	0.049	0.009	Market closer
Colorado Rockies vs. Arizona Diamondbacks	Arizona Diamondbacks @ 35%	Colorado Rockies	0.121	0.009	Market closer
Colorado Rockies vs. Arizona Diamondbacks	Arizona Diamondbacks @ 60%	Colorado Rockies	0.360	0.009	Market closer
Will Kai and Speed beat the Minecraft challenge by August 17?	Yes @ 94%	Yes	0.004	0.003	Market closer
Will Kai and Speed beat the Minecraft challenge by August 17?	No @ 12%	Yes	0.014	0.003	Market closer
Will Kai and Speed beat the Minecraft challenge by August 17?	No @ 22%	Yes	0.047	0.003	Market closer
Will Kai and Speed beat the Minecraft challenge by August 17?	No @ 38%	Yes	0.143	0.003	Market closer
Will Kai and Speed beat the Minecraft challenge by August 17?	No @ 70%	Yes	0.490	0.003	Market closer
Will Kai and Speed beat the Minecraft challenge by August 12?	Yes @ 85%	Yes	0.022	0.023	AI closer
Will Kai and Speed beat the Minecraft challenge by August 12?	No @ 23%	Yes	0.053	0.023	Market closer
Will Kai and Speed beat the Minecraft challenge by August 12?	No @ 35%	Yes	0.123	0.023	Market closer
Will Kai and Speed beat the Minecraft challenge by August 12?	No @ 55%	Yes	0.303	0.023	Market closer
Will Kai and Speed beat the Minecraft challenge by August 12?	No @ 95%	Yes	0.902	0.023	Market closer

Model variants — the self-correction experiment

Model	Forecasts	Brier	Market Brier	Beats market	Accuracy
blend-mkt50-v1	3599	0.156	0.138	52%	74%
blend-mkt75-v1	2960	0.143	0.138	50%	78%
blend-mkt90-v1	2803	0.138	0.137	50%	78%
gemini-2.5-flash	5575	0.199	0.144	54%	71%
gemini-2.5-flash-cal2	3045	0.157	0.145	46%	76%

blend-mkt50-v1 = the raw AI probability blended 50/50 with the market price at forecast time — a deterministic self-correction deployed after the calibration analysis showed mid-range overconfidence. If the blend's Brier beats the market's, the AI adds information beyond price.

Calibration — when the AI says X%, how often does it happen?

AI confidence	Forecasts	Realized	AI Brier	Market Brier
0-10%	4	100%	0.980	0.000
20-30%	1	100%	0.640	0.922
50-60%	395	55%	0.252	0.173
60-70%	991	44%	0.264	0.213
70-80%	1155	59%	0.231	0.192
80-90%	935	58%	0.239	0.225
90-100%	2094	83%	0.122	0.044

Red = overconfident (events happened less often than the AI's stated probability).

AI vs market, by market line

Market line (winner's price at forecast time)	Forecasts	AI Brier	Market Brier	AI edge
mkt 95%+	1346	0.095	0.001	-0.094
mkt 80-95%	1301	0.096	0.016	-0.080
mkt 60-80%	1131	0.107	0.093	-0.014
mkt 40-60%	1004	0.277	0.231	-0.046
mkt <40% (upset)	793	0.578	0.560	-0.018

Methodology

Hourly AI forecasts are stored with the contemporaneous market price (append-only receipts). At resolution every receipt is scored: Brier = (1 - probability assigned to the eventual winner)^2, for both the AI and the market price captured at the same timestamp. Lower is better. Brier is computed only for 2-outcome markets; a winner requires one outcome priced >=0.95 at resolution. No receipt is dropped silently.

Unscorable receipts: 8480 ({'market_expired_unresolved': 8427, 'ambiguous_resolution': 15, 'unmatched_outcome_text': 38}) — counted, never silently dropped.

Raw JSON: /api/v1/scoreboard · Updated 2026-08-12T23:30:02.666591+00:00 · Market-implied odds, not financial advice.