We're Learning Backwards

The Scaling Hypothesis

Think back to the early days of GPT. The original GPT-1 (2018) often generated unintelligible text. With 117M parameters and ~5GB of training data, this was state of the art:

GPT-2 (2019) scaled to 1.5B parameters and 40GB of training data, and brought the first real inkling of “intelligence.” It could generate coherent paragraphs, and OpenAI initially withheld the full model out of concern of misuse. Even the authors seemed surprised by how much improvement they got from what was mostly just “GPT-1, but bigger.”

Gwern’s Scaling Hypothesis essay (2020) argued that intelligence is what you get when you apply simple neural units and learning algorithms at sufficient scale. Feed a model enough data and compute, and capabilities like reasoning, generalization, and understanding should emerge. Six years later, we have LLMs that hold conversations, write code, and solve nontrivial math problems. Most of this didn’t come from an architectural breakthrough. It came from bigger machines, more data, and improved training methods.

Just like Hays & Efros, there is a threshold where outputs *click* from “obvious nonsense” to “surprisingly convincing.”

Correlation, Causation, and Intelligence

But there’s a difference between “convincing” and “intelligent”. Consider the Hays & Efros case. When presented with a truly unique scene, the system fails abruptly. It doesn’t have a model of how objects interact, so it can’t smoothly fill in the blanks when out of distribution. With LLMs, you can see this by asking whether you should walk or drive to the car wash.²² Both Claude (Opus, Sonnet, Haiku) and ChatGPT (Thinking, Instant) confidently say to walk since it’s so close by. They even provide a list of situations where driving would be better (bad weather, carrying heavy items), but they fail to reason that a car wash without a car makes for a pretty bad car wash.

This is obviously spot-fixable with more post-training, but it’s not just an isolated quirk.³³ Especially as these models get smarter and seem to approach or exceed human intelligence, the absurdity of these failures becomes more glaring. It doesn’t make cognitive sense that something that explains graduate-level abstract mathematics simultaneously fails to reason through a simple counting problem.

This is the problem of spiky intelligence: models that are usually brilliant, but have sharp cliffs in their blind spots.

All of this starts to make sense when you consider correlation and causation, two concepts that are notoriously difficult to disentangle. With LLMs, the bet is that forcing enough correlations into a compressed format necessarily forces a learned causal model of the world. Backpropagation incentivizes memorization⁴⁴, but with enough data, it’s no longer possible to memorize everything. This is the core of the Scaling Hypothesis as well.

In other words, causality is emergent from correlation, given infinite data and compute.

I hope I’ve convinced you that today’s LLMs are, in fact, not truly causal. At the same time, they definitely feel intelligent and capable of “reasoning” (through chain-of-thought). So do we actually care about the distinction? If the output is useful, who cares how it got there?

For most situations, most of the time, it doesn’t matter. But on occasion, you get cosmically unlucky and encounter a problem that pushes the model into a region of confidently wrong correlation.

Novelty Is Hard

ARC Prize’s goal is to make “low noise, high signal AGI benchmarks”. Their most recent benchmark, ARC-AGI-3 (March 25th, 2026), tries to do this by putting models in completely new interactive puzzle environments with no instructions, no rules, and no explicit win conditions. The only way to succeed is to explore, test hypotheses, and most importantly, learn from the environment in real time. Scoring is based on action efficiency relative to a human baseline.

So far, the benchmark has been rough for LLMs, with frontier models currently scoring under 1%. I don’t doubt that they will get better, but this benchmark highlights the core problem: LLMs need their priors to be useful.⁵⁵

As a human, what’s striking about these puzzles is how conscious the problem-solving feels. You can watch yourself forming hypotheses, testing them, and revising. I built a daily Wordle-style version of this to share that experience with my friends. It’s challenging, but it gives you a deep appreciation for the complexity of “general intelligence” and the fact that we can do this with any consistency at all.

I’m deliberately avoiding saying that humans solve this “with no priors”. Despite an unseen environment, we have a lifetime of experience and a rich world model to draw on. But even a newborn has core perceptual biases⁶⁶ and an extremely strong ability to learn, refined over millions of years of evolution.

Fluid Intelligence

In 1943, psychologist Raymond Cattell proposed a distinction between two types of intelligence: “crystallized” (knowledge and skills acquired through experience), and “fluid” (the ability to solve novel problems without priors). Generally, humans start with high fluid intelligence that gradually crystallizes over time. Naturally, the division between the two is blurry, but it’s a useful framework for thinking about intelligence in general.

There’s evidence that humans have causal reasoning abilities preceding knowledge accumulation. Leslie & Keeble (1987) showed that 6-month-old infants, with essentially no accumulated experience, react differently to “possible” versus “impossible” events. ⁷⁷ With negligible experience and no opportunity to accumulate patterns over millions of examples, they’re still able to detect violations of a basic causal model of the world.

Fluid intelligence precedes, and likely enables, crystallized intelligence.

Things get interesting when you think about the implications for LLMs. It’s pretty clear that they have crystallized intelligence at extraordinary scale. They have arguably superhuman performance on an inarguably superhuman range of specializations, and it would be reductive to call this solely “pattern-matching”. And yet, they still fail on genuinely novel tasks, so we can’t say they’re AGI either.

The hardest part of artificial general intelligence is the generalization.

I don’t disagree with the Scaling Hypothesis; with infinite data and compute, we will eventually get fluid intelligence. But even if we keep scaling compute, we definitely don’t have infinite data. If data is the bottleneck, then the “simple model, lots of data” approach will necessarily have a ceiling.

Most of the recent progress in LLM capabilities hasn’t come from scaling pre-training. It’s come from increasingly clever post-training techniques: RLHF to align outputs with human preferences, tool use to offload tasks the model can’t do natively, chain-of-thought to simulate step-by-step reasoning, retrieval-augmented generation to access knowledge beyond the training set. These genuinely improve model performance, but they’re also exactly the kind of human-engineered, domain-specific interventions that Sutton’s Bitter Lesson warned us against.

Sutton’s argument was never “don’t be clever.” It was that cleverness doesn’t scale. Hand-engineered features give you short-term wins, but they lose to methods that scale with compute and data. The irony is that the Bitter Lesson might be right about the destination and the path, but we’re running out of fuel.

Are We AGI Yet?

So what could give us fluid intelligence?

If we revisit ARC-AGI-3, the results offer a clue. During the preview period, the best-performing models weren’t the ones with the most knowledge. They were the ones that explored, forming hypotheses, testing them, and revising. StochasticGoose scored 12.58% using CNNs and hierarchical sampling, while frontier language models scored under 1%. The systems that explored their environments, tracked what they did, and revised hypotheses outperformed the systems that pattern-matched from prior knowledge.

This mirrors what I’ve observed when I play the same environments. As you press buttons, you deliberately try to build a world model of the game’s mechanics by forming hypotheses and testing them. When a new mechanic is introduced, you revise your world model to accommodate it. Beyond the ARC benchmark, this is what we also see in the real world. Infants learn about physics by dropping things, pushing things, and seeing what happens, not by reading about physics in a textbook.

By contrast, LLMs have an absurd amount of knowledge but no mechanism to develop the fluid foundation it should rest on. This doesn’t mean they’re useless. Their introduction has dramatically transformed a huge number of fields, and even given us hope that AGI might be possible within our lifetimes. But everything I’ve seen suggests that the path to AGI will not be bigger LLMs.

I think the path requires a fundamentally new architecture that learns a dynamic world model through interaction, with the capability to revise that model in real time. Perhaps a different training signal that rewards exploration, testing hypotheses, and adapting. I don’t know what that looks like. I don’t know how to reconcile that with the Scaling Hypothesis, the Bitter Lesson, or The Unreasonable Effectiveness of Data.

But I do know that LLMs learned backwards.