<link rel="stylesheet" href="theme-light.css" />

Since GPT-4, AI predictions keep missing in the same direction

There are 4 camps on AI:

  1. It's a scam
  2. AI today — already does most digital work
  3. AGI tomorrow — does any work a human can
  4. ASI the day after — does anything cognition can theoretically do

People from different camps can't really argue; they're describing different worlds. But one thing is checkable: since GPT-4, serious predictions about model capabilities have missed in one direction only. Low. Progress runs on an exponential; we think in straight lines. And linear is comfortable — "Fable is the plateau, it slows down from here" is being said right now, the way it was said at every point of the curve so far.

The other comfortable belief is commoditization: the AI layer flattens out, roughly equal and roughly cheap for everyone, and your product just sits on top of it.

Look closer at the gap between the frontier and the second tier, though. Benchmarks understate it, because benchmarks measure puzzles — HLE questions professors spent a month composing. The economic effect lives somewhere else: how long an agent can run real work — code, support, sales, analytics — before it falls over. That's where the billions are, and there the gap is embarrassing.

And it won't close. Recursive self-improvement is no longer a thought experiment — labs use today's models to build tomorrow's. The frontier takes the whole Pareto curve: best at intelligence first, then at speed, then — competition handles this part — at price.

Which makes today's capabilities mostly irrelevant to your roadmap. The next 2 years will move further than the last 5 did.

Now map that back onto the camps:

  1. Fable can't feel or dream, useless toy
  2. Fable is neat — maybe a bit better than the rest
  3. What can a team of a thousand Sol agents, working the same hard problem, actually do?
  4. Sol holds a business problem for 10+ hours; GPT-3 lost the plot by paragraph 3. That distance took 5 years. Draw it again, forward.

If you're building, here's the practical read. The how keeps collapsing toward free, and every moat made of how collapses with it. Two things don't ride that exponential: knowing what's worth building, and proving that what you built is real. The first is why BHAG exists. The second is why the platform you're reading this on does.

Comments

Sign in to join the conversation.

@marcus-bell· 3 months ago

Camp 1 here, getting dragged toward camp 2 roughly quarterly. "Predictions keep missing in the same direction" is harder to argue with than any single benchmark, which is annoying, because I would like to argue.

@hannah-weiss· 3 months ago

The moat framing is the actionable bit: if the how keeps collapsing toward free, what is left is knowing what to build and proving what you built is real. Convenient that you happen to sell both — but the logic holds anyway.

Albert Metz
@albert· 3 months ago

Guilty on the convenience — though the order of operations matters: believe the logic first, then build the companies. Not the other way around.