How AI forecasts are made: five methods, their track records, and how to judge a prediction

"AGI by 2027." "Not this century." "Half of white-collar jobs gone in five years." Headlines about the future of AI are everywhere, and they contradict each other. Most arrive as a single date or number, with no hint of how it was produced or how often that kind of prediction has been right before.

This article opens up the machinery. It explains the five main ways people forecast AI, what the evidence says about each method's track record, and where even the best methods break down. By the end you should be able to look at any AI prediction and ask the five questions that tell you how much weight it deserves.

What a forecast is, and what it isn't

A forecast is a statement about the future that can be checked. "There is a 30% chance that X happens by 2030" is a forecast: when 2030 arrives, X either happened or it didn't, and if you make many such statements you can be scored on them. The standard score is the Brier score, which measures the gap between the probabilities you gave and what actually happened. Lower is better, and zero is perfect.

Two things are often confused with forecasts. A prediction without a probability ("AGI is coming soon") can't really be scored, because the speaker can always reinterpret "soon." A scenario is a story about one way the future could unfold. Scenarios are useful for thinking, but they aren't claims about what is most likely. Canada's federal foresight agency, Policy Horizons Canada, makes this explicit in its February 2026 report on AI, which sets out four futures with names like "AI everywhere all at once" and "Algorithms in the shadows." The report says the scenarios "are not predictions or forecasts, they illustrate plausible futures"¹⁰.

Keeping these three apart is the first skill. A lot of public confusion comes from reading a scenario as a forecast, or a vague prediction as a precise one.

Five ways people forecast AI

1. Ask the experts. The simplest method is to survey people who build AI. The largest such survey, by Katja Grace and colleagues, polled 2,778 researchers who had published at six top AI venues in October 2023. Aggregated, they gave a 50% chance that "unaided machines can accomplish every task better and more cheaply than human workers" by 2047, thirteen years earlier than the same question produced in 2022¹.

2. Extrapolate a trend. Pick a quantity that has moved steadily, fit a line, and extend it. The best-known example today comes from METR, a nonprofit that evaluates AI systems. It measures the length of software tasks (in human working time) that a model can finish with 50% reliability, called the time horizon. In a 2025 paper, that horizon had doubled roughly every seven months since 2019. The authors projected that, if the results generalize to real-world work, AI could handle many software tasks that take humans a month within about five years⁷. METR's January 2026 update found the same seven-month doubling over the whole period, and a faster pace (roughly every three months) for models since 2024⁸.

3. Use trained forecasters. In the early 2010s, five university-based research groups competed in a two-year geopolitical forecasting tournament. One of them, Philip Tetlock's team, found that training forecasters to avoid common biases, letting them work in teams, and grouping the top 2% into elite teams "dramatically increased the accuracy of forecasts"³. These superforecasters are now regularly asked about AI.

4. Aggregate many forecasters. Instead of trusting one person, combine many, either by averaging or through markets and forecasting platforms. The most systematic recent effort is the Longitudinal Expert AI Panel (LEAP) run by the Forecasting Research Institute (FRI). It asks the same detailed questions of 339 experts (computer scientists, economists, industry and policy professionals), 60 superforecasters and about 1,400 members of the public, and repeats them over time so changes in opinion can be tracked⁵.

5. Build a model or tell a story. Some forecasters build explicit models that link inputs (computing power, research speed) to outcomes. Others write detailed scenarios. AI 2027, published in April 2025 by Daniel Kokotajlo and colleagues, is the most widely read example: a month-by-month story of rapid progress, backed by quantitative models in its appendices⁹.

Each method answers a slightly different question, and each has a record we can now partly check.

The track record so far

Forecasts about AI are hard to grade, because most are about dates that haven't arrived. But enough near-term questions have now resolved to say something useful.

Expert surveys have been unstable. Between the 2016 and 2022 rounds of the Grace survey, the 50% date for human-level machine intelligence moved by one year. Between 2022 and 2023, after ChatGPT, it moved by thirteen¹. The same survey shows strong framing effects, meaning answers change with how the question is worded. When asked about "full automation of labour" rather than machines that can do every task, respondents put the 50% date at 2116, about 70 years later, for what is arguably the same event¹. The authors themselves warn that AI researchers "are experts in AI research, not AI forecasting" and that their answers "should not be seen as a reliable guide to objective truth"¹. Historically, a 2012 review of 95 AI timeline predictions found that expert predictions were "indistinguishable from non-expert predictions," and that the most common answer, from both groups, was that AI was 15 to 25 years away, whenever the prediction happened to be made².

Forecasters underestimated recent AI progress, and superforecasters did so most. In 2022, FRI's Existential Risk Persuasion Tournament (XPT) brought together 169 domain experts and superforecasters to forecast, among other things, AI benchmark scores. When FRI checked the results in September 2025, models had beaten the forecasts on the MATH, MMLU and QuALITY benchmarks, and an AI had achieved an International Mathematical Olympiad gold medal in July 2025. Averaged across those four questions, domain experts had given the outcome that actually happened a 24.6% probability. Superforecasters gave it 9.7%⁴. Overall accuracy across all resolved questions was nearly identical for the two groups, and neither group beat a simple trend-extrapolation baseline⁴. Notably, the errors weren't always in the "too slow" direction: the same forecasters were too optimistic about green hydrogen costs and direct air capture⁴.

Trend lines have held up, so far. The METR doubling trend has survived a change in task suite and testing software, and its post-2024 pace has been faster, not slower⁸. That is real evidence. But METR is careful about what it shows: its confidence intervals "are still very wide," the result is "somewhat sensitive to task composition," and it had human timing data for only 5 of its 31 longest tasks⁸. The 2025 paper also found models do worse on "messier" tasks that look more like real jobs⁷.

Scenario authors revise. The AI 2027 authors later clarified that 2027 was their single most likely year, while their medians (the year by which they gave 50% odds) were somewhat later, and their May 2025 update lengthened one author's median for a "superhuman coder" from 2027 to 2029⁹. Revising in public is good practice. It also shows how much a vivid scenario can be read as more certain than its authors intended.

AI is getting better at forecasting too. On ForecastBench, an FRI benchmark that scores AI systems on real questions whose answers aren't known yet, superforecasters were still first in January 2026, ahead of the best model by 0.017 Brier points. FRI's extrapolation put AI–superforecaster parity around November 2026, with a 95% range from January 2026 to November 2027¹¹. That is itself a trend forecast, and it is due for checking soon.

What today's careful forecasters actually say

With those track records in mind, here is the most structured recent picture.

In LEAP's first waves (July to September 2025), the median expert expected that by 2030 AI would assist in 18% of US work hours, up from about 4% in late 2024, and use 7% of US electricity⁵. On jobs, experts expected only 2% growth in US white-collar employment by 2030, against a 6.8% trend extrapolation⁵. Where experts and superforecasters disagreed, superforecasters usually expected less progress⁵, which is worth remembering given their XPT record.

LEAP's Wave 8, collected April to May 2026, asked directly about timelines. It defined AGI as commercial AI that can outperform the 90th-percentile professional in every mainly non-physical occupation, at no more than five times the cost of human labour. Experts and superforecasters both gave a median 80% chance this happens before 2100. Among those who expect it, the median year was 2050 for experts and 2047 for superforecasters, with a quarter of experts saying 2039 or earlier⁶. On METR's measure, experts' median expectation was that AI completes eight-hour tasks with 80% reliability by 2030. Superforecasters said 2028⁶.

Two features of these numbers matter more than the dates. First, the spread is wide: a quarter of experts put AGI before 2040 and the middle half of expert answers runs from 2039 to 2065⁶. Second, LEAP's authors note that the forecasts "fall far short of the warnings from AI lab leaders about imminent artificial superintelligence"⁵. So the most widely quoted predictions, from lab executives, sit at one edge of a broad expert distribution, not at its centre.

The limits: why AI is unusually hard to forecast

Definitions move. "AGI" and "human-level AI" mean different things to different people. The 70-year gap between two nearly equivalent survey questions¹ shows how much of a forecast can be the question rather than the answer.

Short-term accuracy may not transfer. FRI found correlations between forecasters' accuracy on near-term questions and their views on long-term AI risk were "all close to zero"⁴. Being right about 2025 benchmarks tells you little, so far, about who is right about 2040.

Benchmarks saturate and drift. Trend extrapolation needs a stable yardstick. AI benchmarks often get "solved" within a few years, and METR notes its own suite "has relatively few tasks that the latest generation of models cannot perform successfully"⁸. When the ruler runs out, the line has to be re-fit on a new one.

Capability is not impact. Forecasts of what models can do are better grounded than forecasts of how fast businesses and governments will use them. LEAP's modest jobs forecast next to its aggressive capability forecasts⁵⁶ is a reminder that adoption, regulation and cost sit between the two.

Self-fulfilling and self-defeating effects. Forecasts of rapid progress attract investment and talent, which can speed progress up. Forecasts of danger can trigger rules that slow it down. Unlike the weather, AI responds to what people predict about it.

None of this makes forecasting useless. It means that a forecast's method, spread and track record matter more than its headline number.

What this means for you

The next time you see an AI prediction, ask:

1. Is it a forecast, a scenario, or a slogan? Is there a probability and a date that could be checked? If not, treat it as an opinion or a story.
2. What method produced it? A survey, a trend line, a forecasting panel, a model, or one person's intuition? Each has a different record, summarized above.
3. What's the spread, not just the middle? A median of 2050 with a middle range from 2039 to 2065 says something very different from a flat "2050."
4. Who is making it, and what would change their mind? Good forecasters publish their reasoning and revise in public. Be cautious with predictions that never get updated or are never wrong.
5. Does the decision in front of you depend on the date at all? Often it doesn't. Whether to train staff, test a tool, or strengthen data security rarely hinges on whether AGI arrives in 2035 or 2050. Plan for a range and set signposts you'll watch, such as METR's time horizon or LEAP's next wave, rather than betting on a single year.

Sources

1. Grace, K., Stewart, H., Sandkühler, J. F., Thomas, S., Weinstein-Raun, B., & Brauner, J. (2024). Thousands of AI authors on the future of AI. arXiv:2401.02843. arxiv.org/abs/2401.02843
2. Armstrong, S., & Sotala, K. (2012). How we're predicting AI, or failing to. In J. Romportl, P. Ircing, E. Zackova, M. Polak, & R. Schuster (Eds.), Beyond AI: Artificial Dreams (pp. 52–75). University of West Bohemia. coefficientgiving.org/wp-content/uploads/ArmstrongandSotala2012.pdf
3. Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., Scott, S. E., Moore, D., Atanasov, P., Swift, S. A., Murray, T., Stone, E., & Tetlock, P. E. (2014). Psychological strategies for winning a geopolitical forecasting tournament. Psychological Science, 25(5), 1106–1115. doi.org/10.1177/0956797614524255
4. Forecasting Research Institute. (2025, September 2). What did forecasters get right and wrong in the largest existential risk forecasting tournament? forecastingresearch.substack.com/p/what-did-forecasters-get-right-and
5. Murphy, C., Rosenberg, J., Canedy, J., et al., Tetlock, P. E., & Karger, E. (2025, November 10). The Longitudinal Expert AI Panel: Understanding expert views on AI capabilities, adoption, and impact (FRI Working Paper No. 5), with Insights from Waves 1, 2, and 3. forecastingresearch.org/research/longitudinal-expert-ai-panel-leap-working-paper and leap.forecastingresearch.org/reports/waves-1-to-3-insights
6. Forecasting Research Institute. (2026, June 1). LEAP Wave 8: Timelines. leap.forecastingresearch.org/reports/wave8
7. Kwa, T., West, B., et al. (2025, v4 2026). Measuring AI ability to complete long software tasks. arXiv:2503.14499. arxiv.org/abs/2503.14499
8. METR. (2026, January 29). Time Horizon 1.1. metr.org/blog/2026-1-29-time-horizon-1-1
9. Kokotajlo, D., Alexander, S., Larsen, T., Lifland, E., & Dean, R. (2025, April 3, notes updated through December 2025). AI 2027, including Lifland, E., Jurkovic, N., & FutureSearch, Timelines forecast. ai-2027.com and ai-2027.com/research/timelines-forecast
10. Policy Horizons Canada. (2026, February 10). Foresight on AI: Scenarios for an AI-enabled world. Government of Canada. horizons.service.canada.ca/en/2026/02/10/scenarios-ai-enabled-world/index.shtml
11. Bastani, H., Kučinskas, S., & Reynolds, M. (2026, January 29). LLMs are closing the gap on human superforecasters. Forecasting Research Institute. forecastingresearch.substack.com/p/llms-are-closing-the-gap-on-human