On March 25, 2016, I stood in a lecture hall under three enormous projector screens and spent fifty-three minutes explaining how to build a thinking machine.
I was twenty-two. Almost no one was listening. So much of the next ten years was already up on those screens.
It was ten days after AlphaGo beat Lee Sedol, and the room was still warm from it. Here is that talk, played back through everything that happened next.
To place the moment: this is March 2016. The transformer did not exist. There was no GPT, no model you could simply ask. The paper that would quietly become the foundation of everything that came next, the one with the cocky little title “Attention Is All You Need,” was still fifteen months away.
So the vocabulary of the entire talk is convolutional networks, recurrent networks, and LSTMs. That was the whole toolkit anyone had. I stood there with a clear picture of the destination and the wrong vehicle parked underneath it. I knew where this was going. I had no idea how it would get there.
That gap is the whole story of the next decade. You can watch me stand right on the edge of it without knowing.
The whole fifty-three minutes, with the moments worth jumping to. Pick a chapter and it drops you straight in.
I did something a little reckless for a student talk. I taught the whole thing from the floor up. A neuron is a weighted sum and a squashing function. Stack them into layers and you get a matrix. Multiply, add a bias, squash, repeat, and you have a network that turns a handwritten digit into the number two.
Then I had to explain how it learns, which meant explaining backpropagation, which meant explaining gradient descent to a room that mostly had not taken calculus. So I put up a mountain range. The height is your error. You are dropped somewhere random in the dark, and you cannot see the valley. All you can do is feel the slope under your feet and step downhill. Do that a few million times and you arrive at the bottom without ever having seen the map.
I was proud of that analogy. I still am. It is the only honest way to explain the thing. So before you read another word, go feel it.
From there the talk climbs. Convolutional networks that learn their own edge detectors. Recurrent networks that remember. An LSTM that learns to forget, which is how it learns to remember. Reinforcement learning, on a slide titled “Reinforcement Learning! FINALLY,” which tells you exactly how long I had been waiting to get to the fun part.
And then the demos. This is where the kid shows his hand.
I trained a character-by-character network on my GTX 980 for eight hours and made it write in the voice of Friedrich Nietzsche. Then I put two passages on the screen and asked the room to spot the forgery. Try it yourself.
Then, over spring break, I fed a network every tweet from Trump and Hillary and built a thing I called Trillary Crump. I was a kid, it was an election year, and I thought this was the funniest possible use of a neural network. I was not wrong.
Here is what makes the tape worth playing back.
Every architecture in that talk is a convolutional network, a recurrent network, or an LSTM. The word transformer does not appear, because the transformer did not exist. My own demos, the character network and the tweet generator, are precisely the techniques it was about to make obsolete.
I had the destination in focus and the mechanism completely wrong. Right about the what. Blindsided by the how. That sentence turns out to describe almost everything on the next slide.
So let us actually score it. Here is the future as a twenty-two-year-old called it, with a decade of hindsight stapled to each card. Some of these still make the hair on my neck stand up. Tap any one to see what happened.
NVIDIA put tensor cores in everything and became, by 2024, one of the most valuable companies on Earth on exactly this thesis. The GTX 980 I trained on now sits next to a datacenter GPU like a candle next to a reactor.
That is ChatGPT, Copilot, and Gemini, described seven years early. The live example I gave, the machine settling a dinner-table argument, now happens at my actual dinner table. It shipped around 2023.
That is, almost word for word, the self-supervised pretraining that powers every large language model. I pointed dead at the destination. I just drew the wrong machine underneath the label.
I wrote the word Copilot without knowing the brand was coming. Augment the professional is the dominant framing of 2026, and it is the product I help build.
A decade later, figuring out why they work is a serious field called mechanistic interpretability. Still open. Still the best question in the room.
It foreshadows mixture-of-experts and today's multi-model agent systems. Right instinct, reaching for a shape the field would standardize on. It just did not arrive the way I drew it.
RL stayed central, but as the polish on top of language models: the part that taught them manners, and later the part that taught them to reason. Right that it mattered. Wrong about the form.
Here in 2026, none of it has arrived. I was conservative about language and optimistic about bodies. The field turned out to be the other way around.
The unlock was not depth. It was scale, on a simpler architecture that could swallow the entire internet as a training set. Deep gave way to wide, plus attention, plus data.
Data turned out to be the co-equal third pillar. The web itself became the training set. I was standing next to half the story and never named it.
The transformer, the thing that actually delivered every prediction on this scorecard, was fifteen months away. My own demos are precisely the techniques it made obsolete. Right about the what, blindsided by the how.
The misses are more interesting than the hits, because they are all the same miss wearing different clothes.
I thought progress meant depth. More layers. My headline example was a network that had just won by stacking a hundred and fifty-two of them. The real unlock was not depth. It was scale, and a simpler shape that could swallow the entire internet as a training set. I barely mention data in the whole talk. Data turned out to be half the story.
I was split on timing in a way I find almost funny now. The language, the assistants, the answers before you ask, I filed all of it under twenty years, and it took seven. The robots, the emulated visual cortex, the Teddy from the Spielberg movie, I filed under ten years, and here in 2026 none of it has arrived. I was conservative about words and optimistic about bodies. The field turned out to be the other way around.
And I bet the house on reinforcement learning as the road to general intelligence. It mattered, but not the way I drew it. It came back as the polish on top of the language models, the thing that taught them manners, and then again as the thing that taught them to reason. Right instinct. Wrong shape. Same as everything else.
Here is the part that gets me.
This talk is March 2016. GCNet, the GIF-captioning network sitting one folder over in this workspace, is November 2016. Watch them back to back and you can see the toolkit getting assembled in real time. The convolutional eyes. The recurrent memory. The single graphics card in a warm room. The talk is the prequel to the project.
But it is the prequel to more than that. A kid in a half-empty lecture hall, predicting ambient intelligence and quoting Kurzweil’s singularity, is the same person writing The Fortunate Fall a decade later. The themes did not change. The resolution did. Back then I was reading the tea leaves and hoping. Now I am reading them with a track record.
The thing this whole blog is about, the distance between what you want and what the machine gives you, I was already watching it close on those three projector screens. I just did not have the name for it yet.
Right about the what. Blindsided by the how. Early on the when.
I would give the talk again tomorrow.