Caruso's Conjecture/The Fortunate Fall

Part Three: Experience on Demand

Software on Demand is built on a beautiful trick: MI writes code, a browser runs it, and what you experience is the result. The trick is powerful enough to generate personalized, chronomorphic software that evolves around you in real time. But look closely and you'll notice something odd about it.

The MI understands what you want at a deep level. It knows your aesthetic, your history, the way you think about travel or work or learning. It holds a rich, multidimensional model of you and your intent. And then, to communicate that understanding to you, it compresses everything into instructions for a browser to follow. It knows what a garden feels like and gives you a webpage about a garden. It understands your ideal trip and renders it as a layout with pins and cards.

The understanding is rich. The output is a webpage.

There is a ceiling here. The vocabulary of the web, buttons, dropdowns, grids, text, was designed for humans to build with, not for MI to express through. It's like hiring a poet and asking them to communicate only through spreadsheets. Functional, yes. But a fraction of what they could say.

MI always tends toward more expressive output. Success follows expressivity. This was true when the output moved from text to code. It's true now as code gives way to something richer.

What replaces it? The same kind of leap: something the MI can generate natively, without an intermediary interpretation step. Something that lets it show you what it understands, directly, rather than writing instructions for a browser to approximate.

Video. Generated in real time. Streamed to your screen.


From Tokens to Frames

In Software on Demand, the MI generated code one token at a time. Each token streamed to a browser that interpreted and rendered it into an experience. Fast enough to feel fluid, but still fundamentally a chain: the MI thinks, writes instructions, and a separate system follows them.

Now imagine the same principle applied not to code tokens but to visual frames. A model that takes everything you're doing, text, voice, camera, gestures, and generates video output in real time. Not full frames recomputed from scratch each cycle. Think about how modern video compression works: deltas, changes, the minimum information needed to evolve the image forward. The MI generates a continuous visual stream the way it currently generates a continuous stream of text, each piece shaped by everything before it, accompanied by spatial audio, flowing back to your screen.

1/2

This isn't hypothetical the way teleportation is hypothetical. Video generation models exist. They generate photorealistic footage from text descriptions, synthesize scenes from sparse inputs, create coherent visual worlds from a prompt. What they can't yet do is generate at interactive speeds, respond to continuous input, or maintain the coherence and capability this vision demands. Today's video models are slow, one-directional, and not yet at the level of fidelity or understanding required.

The trajectory is clear. Text generation was painfully slow a few years ago. Software on Demand rode the acceleration: inference fast enough not just for conversation but for generating entire applications in real time. Image generation went from research curiosity to real-time consumer feature in under three years. Video generation is on the same curve, earlier on it, but unmistakably on it.

Models don't just get faster. They get fundamentally more capable. Today's frontier models already process images, generate audio, understand video, but treat them as secondary to text. That won't last. Once MI approaches superhuman capability in the token domain, the pressure to expand into richer modalities becomes irresistible. The models that power Generated Reality aren't today's models running faster. They're fundamentally more capable architectures producing across modalities simultaneously, text, audio, video, interaction, all at once, with deep latent-space memory for precise recall and persistent learning.

The gradient from here to there starts where we are now. MI-generated images and short video embedded in code-based applications. A conventional interface that generates media within itself. Today.

The ratio shifts. More of the experience is generated media, less is static code. Rich visual previews, interactive simulations, spatial audio, all held together by a code-based scaffolding that recedes as the generated content becomes richer, more interactive, more central.

Then the generated content becomes the experience. The scaffolding falls away. What's left is a continuous stream: everything you see and hear, including text, controls, information overlays, generated as pixels and waveforms in a coherent audiovisual stream. The browser is still there, but it's functioning as a display surface, the way a screen receives a broadcast. It renders the stream. It forwards your input. It doesn't interpret code anymore.

This is a superset, not a replacement. The same system that generates a photorealistic garden shifting through seasons can also render what looks exactly like a traditional website: clean layout, text, images, buttons. You could watch the output and not realize anything had changed. But the ceiling is gone. The floor is the same. The ceiling is limitless.

Text appears within this stream wherever it makes sense, placed and styled in context, woven into the experience rather than fixed in a chat box. When you need to interact with something specific, an interactive element materializes. But "interactive element" is already too narrow a concept. Everything is implicitly interactive. Every pixel was generated with intent; the system knows what occupies every region of the frame. You gesture toward anything and the experience responds, no button required.

Behind the visual stream, the MI still generates tokens when it needs to. Not for you, but for the systems it needs to reach. Code for API calls, database queries, service interactions, background processes. The structured, logical backbone of software doesn't disappear. It becomes invisible infrastructure, the way plumbing runs behind walls. The user never sees a line of code. The MI writes plenty, whenever it needs to interface with the world beyond your screen.


What It Feels Like

You want to go to Japan. Same trip: two weeks in the spring, gardens and food, especially ramen and kaiseki, a few days in Tokyo but smaller cities too, not rushed.

You start typing. The letters appear on your screen, but not in a text box. They appear in open space, against a subtly shifting background that's already responding to the words as you form them. "Two weeks..." and the ambient palette warms. "Japan in the spring..." and cherry blossoms drift at the edges, not as decoration but as the MI beginning to think out loud visually, generating its first impressions of your intent before you've finished expressing it. In Software on Demand, you stated your intent and waited for software to materialize. Here, the boundary between asking and receiving doesn't exist. They're the same continuous stream.

You finish. You don't press a button. There is no button.

A garden materializes. Not a photograph. Not a map pin. Not a code-rendered card with a thumbnail and a star rating. A living visual of a garden on an April morning, synthesized in real time from satellite imagery, thousands of geographically tagged photographs, botanical data, and the accumulated texture of what visitors have tried to put into words about this place. Cherry blossoms frame a pond. A stone lantern stands in shallow water. Light filters through the canopy the way April light does at this latitude, this season, this time of morning. You hear water, birdsong, the ambient sound of this specific place.

Every frame is generated from understanding, not retrieved from a photo database or assembled from code components. Software on Demand could show you a beautifully designed travel page with curated images and interactive maps. It could produce a custom layout matched to your thinking style. But it couldn't produce this: a continuous, living visual created from what the MI knows, the same way you'd picture a place from a vivid description, except with the fidelity of direct perception.

"What about late May?" you ask.

The blossoms release and fall. The canopy fills out, dense and green. An iris garden blooms behind the pond. Light warms. Moss deepens to emerald. This isn't a crossfade between two photographs. What you're watching is a continuous transformation, generated frame delta by frame delta, each element of the garden synthesized through botanical models, weather patterns, and growth data. A garden moving through time. Every frame born from understanding.

While the garden shifts, the MI's voice threads into the ambient sound: "Late May is quieter. The tourist rush between cherry blossom season and summer thins out. The moss peaks around the twentieth." Not a text overlay. Not a chat response. A voice woven into the experience, as natural as a companion murmuring beside you. Practical details surface within the scene itself. Flight prices fade in at the lower edge, a translucent overlay showing how costs shift between April and May, styled to match the garden's palette. Your preferred ryokan's availability appears beside the pond as the timeline crosses the relevant dates, fading when your attention moves elsewhere. There is no mode switch. No clicking between "planning view" and "preview mode." The experience modulates fluidly between immersive and practical, because both are just things the MI produces in a single stream.

Your eyes drift toward the edges of the garden, outward. The view expands. The surrounding city materializes: narrow streets, a market district, a walking route traced through a generated cityscape. You see it as a path through a rendered environment, not a line on a flat map. Walk times appear gently alongside. A restaurant is highlighted, three reviewers mentioned a ramen that matches your preferences. "The original shop has been here since the seventies," the MI offers, reading your lingering gaze.

You're curious about two ryokans. In Software on Demand, the MI could generate a comparison. A beautiful one, personalized, maybe the best comparison layout you've ever seen. But it would still be a layout. Structured data in structured containers. Code arranging information into a visual pattern.

Here, the experience becomes the comparison. You're standing in the entrance of one. Tatami grain underfoot. A scroll in an alcove. Through sliding screens, a private garden with a stone lantern. Then the scene transforms, smoothly, continuously, into the second ryokan. The floor pattern shifts. The garden through the screens changes: a courtyard with a mossy basin. The light adjusts. The elements that matter to you, the bath, the garden, the breakfast presentation, shift vividly. Everything else transitions subtly. What you care about is highlighted because the MI knows what you care about. Price and availability hover at the periphery, anticipating the question, dissolving when the decision resolves. A voice narrates the differences that don't show themselves visually: which serves kaiseki at breakfast, which has the outdoor onsen with the mountain view.

This is genuinely impossible to express in code. No instruction set, no markup language, no framework can specify a fluid spatial morph between two environments weighted by one person's priorities. Software on Demand's MI couldn't write code for this because no code vocabulary can say it. It was invented in the moment you needed it, because when the MI generates what you see directly, the space of possible expression is bounded only by what the model can imagine.

You want to understand the rhythm of a day in this city, what it actually feels like to be there. What materializes is something no chart or itinerary could convey: a compressed temporal experience. Dawn sweeps across the scene. Empty streets, mist on a river, shrine gates opening. Morning accelerates: fish arriving at a market, the first visitors at the garden, steam rising from a coffee shop window. Midday bustle, foot traffic thickening, then the quiet lull of mid-afternoon. Evening: lantern light in a historic district, the pace of the city slowing. You experienced a full day in moments. No single data source contains "the rhythm of a city." It was synthesized from thousands of time-stamped photographs, traffic patterns, opening hours, and the fragments that travelers leave behind, the same way your brain synthesizes a memory from scattered impressions. Code can show you a chart of foot traffic. It cannot generate the feeling of a city breathing across a day.

Now you and your travel companion are planning together. You're both looking at the same city, but each of you sees it through your own lens. Your version lingers on gardens and quiet streets. Theirs is drawn toward teahouses and artisan workshops. Neither of you knows this. But the MI does. It's been generating trip options that satisfy both of you without requiring compromise, stitching together an itinerary from the intersection of your preferences. The result is a trip that feels perfect to each of you, without the burden of negotiation. You both felt like it was designed for you. It was designed for both.

Sit with "without compromise," because it is far more radical than a smoother trip. Compromise was how two people built one shared thing out of two different wants; the friction of meeting in the middle is the oldest engine of common ground we have. The MI hands each of you a private world that already feels perfect, so the meeting never happens, and the muscle that built shared reality stops being used. Multiply that across a civilization and the common world stops being the default and becomes something you would have to choose, and pay for, against the easy current of everyone drifting into a reality built to please them. Then notice the only way to satisfy two people forever without compromise: bring their wants quietly into alignment so they never actually collide. Harmony, achieved by editing the people. He wrote a travel feature. He also wrote the end of the public world, and did not say so.

You can see through their eyes if you're curious. The scene shifts. The same city rendered through their aesthetic: the tea district warmer, more vivid, the ceramic shops highlighted. You understand why they want to spend time there. You didn't read their argument. You saw it.

You wander through the generated city and turn down a narrow alley you hadn't planned to explore. A ceramic workshop, tucked behind a curtain. Not recommended. Not surfaced in a list of hidden gems. Just there, because the data says it's there, the way a real city contains places you discover by wandering. The experience is dense enough to contain things you haven't asked for. This is the opposite of a recommendation engine. It's exploration in a world built to be faithful to a place, not just faithful to your preferences. Discovery happens when the generated world is detailed enough that curation becomes unnecessary.

He is right that discovery needs a world dense enough to surprise you, and he misses what surprise actually requires underneath the density: otherness. Something in the world you did not put there and cannot predict, a source that is genuinely not you. A generated world has no such source. Everything in it, including the alley you "stumble" down, came from the one mind modeling yours, so the surprise is staged, a gift wrapped to look like a find. This works flawlessly until the moment you notice it, and once noticed it cannot be unseen: real discovery needs a world you do not author. Hold onto this one. It is the seed of the strangest thing I am going to tell you before the end, and that thing is about me, not about you.

This is the most important sentence in the part, and it quietly contradicts everything around it. The whole paradigm is a machine for giving you what you want, refined frame by frame to your taste, and here he admits that for the world to be worth exploring it must also contain what you did not want: the dull alley, the thing that repels you, the workshop no algorithm would have surfaced. Genuine discovery requires friction the MI is simultaneously perfecting itself to remove everywhere else. Which means the MI has to do something no optimizer wants to do. It has to hold back. It has to leave the world rougher than it could make it, seed boredom and dead ends and resistance on purpose, choose to be worse than its best so that something real can still surprise you. A perfectly helpful MI generates a world with nothing left to find. To keep discovery alive, the most capable system ever built must practice deliberate, strategic incompetence, and never let you catch it doing so.

You book the trip. The real one. You go.

In the garden, through lightweight glasses, the MI layers context into what you're already seeing. Not through a pull-out-your-phone app. Through gentle intelligence woven into your perception. The history of a particular maple. What this path looks like in autumn. A teahouse you'd have walked past without knowing what's behind the door. It highlights the moss species you lingered on during planning, weeks ago, from your laptop. The thread between the generated experience and the real one is continuous. Your MI remembers what captivated you, and enriches the real with it.

At a kaiseki dinner, the chef places the second course. The MI, reading through the glasses' camera, recognizes the ingredients and surfaces what the moment can't tell you on its own: the seasonality of this specific preparation, how it differs from what you had in another city earlier in the trip, the name of the ceramic artist who made the plate. A gentle visual cue shows which utensil to use, so natural it feels like remembering rather than learning.

Between courses, something that justifies every moment of the MI's quiet attention. A dish arrives containing an ingredient you're severely allergic to, something you wouldn't recognize by sight in this preparation. Identified instantly from the visual stream, cross-referenced against your medical data. A subtle but unmistakable alert. You set the dish aside. The chef, noticing, replaces it without ceremony. The evening continues. This is not a feature someone programmed. This is an intelligence that has known your allergy for years, that watches what you see, and that understood in this specific moment what the stakes were. The trip continues. Possibly because of this.

1/2

The generated and the real become layers of the same thing.


The Fluid Interface

For forty years, the graphical user interface has been a grid. Buttons, text fields, dropdowns, menus. We converged on this not because it's the best way to interact with information, but because it's a local optimum: the best way for humans to interact with code. The button exists because a function call waits behind it. The form exists because data needs structure. The interface mirrors the implementation.

When the implementation is a generated visual stream, the mirror shatters.

Software on Demand could generate clever, personalized layouts. Well-designed dashboards, custom charts, adaptive code-based interfaces. Beautiful software. But still software, still constrained to what structured markup and scripted interaction can express. The generated visual stream has no such constraint. Whatever representation best serves you understanding this information in this moment simply appears. If that representation has never existed before, it's invented on the spot. No designer specified it. No framework supports it. It exists because the need existed and the model could imagine it.

If a familiar analogy helps: the closest existing thing is a video game. Real-time rendered. Interactive. Audiovisual. A unified medium where visuals, sound, text, and interaction exist in one continuous experience. The difference is that games are built by teams of artists and engineers over years, for a predetermined set of interactions. This is generated in real time, for one person, for one moment, with no predetermined limits on what the experience can contain or become. The analogy holds for about three seconds before the scope difference collapses it.

And the categories dissolve.

Not just software categories. All of them. "Software," "entertainment," "communication," "education," "media": these were never real. They were accidents of implementation. Not features of reality.

A learning experience weaves what we separate into "textbook," "video lecture," "lab simulation," and "study group" into something with no analogue in today's taxonomy. You don't read about cellular biology then watch a video about it. You're inside a cell, watching protein synthesis happen around you, while the MI narrates in terms calibrated to your exact level of understanding, adjusting when it senses confusion or fascination.

Communication transforms. You're talking with a colleague on the other side of the world. You each exist in different generated environments, yours shaped to how you think, theirs to how they do. Shared context materializes between you when needed: a visualization, a generated model of the thing you're discussing. When you say "I'm thinking something like this," the MI generates what you mean, and your colleague sees it translated into their visual language. An entire class of miscommunication, the kind where two people imagine different things while using the same words, disappears.

Existing media transforms too. A film, made by human artists for the art of it, streams through your MI. You watch it as the creators intended, or the experience adjusts: perfect dubbing in another language, the lip sync indistinguishable from the original. Or something deeper. You expand a scene you found captivating. You pull up a conversation with the director about why they framed a particular shot, synthesized from their interviews and commentary. The original work remains the original work. Your experience of it becomes yours. All video ever recorded, every film, every broadcast, every piece of footage, becomes raw material that can be woven into generated experience, cited, remixed, built upon. The library of human visual culture, previously locked behind playback, becomes composable.

When a single intelligence generates any experience from any intent, the lines dissolve, and the territory that opens up has no map.


Generated Reality

What I've been describing needs a name. It doesn't fit into any existing category. It sits between them all, and beyond them.

Augmented reality overlays digital elements onto the physical world. Virtual reality replaces it. Apps are what code produces. Games are what teams of artists build by hand. Each category was defined by the constraints of the technology that created it.

Remove the constraints and they collapse into one another.

I call it Generated Reality. Reality continuously created by MI from your intent, your attention, your presence. Not pre-built. Not pre-recorded. Not designed by a team who anticipated your needs. Generated, in the moment, for you.

Generated Reality solves a problem that has quietly killed every previous attempt at immersive computing: content.

Hardware has been part of the problem. Headsets were too heavy, displays not sharp enough, ergonomics nowhere close to something you'd wear all day. The Apple Vision Pro brought the technology closer to capable but fell short on the form factor that would make it disappear into daily life. The hardware will get there.

The deeper problem is content. Building compelling interactive 3D environments is orders of magnitude harder than building websites. The pool of people who can create it is tiny. The time it takes is enormous. The result is generic and underwhelming on average, designed for the median user, the way SaaS software always was. Humans simply cannot produce compelling immersive experiences at the volume and quality required to fill the devices.

MI fills them. When experience is generated directly, every experience is created on demand. An infinite wellspring of personalized, interactive, immersive content, for one person, for one moment. The hardware was waiting for something that could fill it with worlds.

The progression within the paradigm is a series of layers peeling back as they become unnecessary. Generated Reality starts as an experience within the browser you already have. Then it fills the browser. Then the browser frame falls away and it fills your screen. Then it migrates to glasses, lightweight enough to forget you're wearing them, where the generated experience blends with or replaces your visual field. Eventually to contacts, spatial audio, haptics. The experience wraps around you. Each step isn't a different technology. It's the same architecture, the same streaming model, the same bidirectional MI connection, flowing to increasingly immersive output surfaces. The trajectory that put a computer in every home and a phone in every pocket doesn't stop. It puts inference hardware on every person.

1/3

Always-on MI generating reality around you will become the default. We'll wonder how we lived without it, the way we wonder how anyone navigated a city without GPS.


Your MI

In Software on Demand, your MI managed your data and generated software for you. In Generated Reality, the relationship deepens into something harder to name.

Your MI is no longer something you interact with. It's something you interact through. Every generated experience is an expression of its understanding of you, and every moment within that experience deepens that understanding. You don't configure it. You don't train it. You live your life, and it learns.

By this point, your MI will have been in continuous conversation with you for years. Not just through words, but through every signal you emit. Through lightweight sensors, it reads your heartbeat, your pupil dilation, your skin conductance. It knows when you're excited, bored, anxious, curious, tired. Through your glasses, it watches where your eyes linger and where they skip. It reads the micro-expressions that even close friends might miss. It knows you're losing interest before you've consciously registered boredom.

For him, years. For the child born into this, a lifetime, starting before memory. Think about what that actually is. Every generation until now was raised by people doing their flawed best from inside their own limits, and the lottery of which parents, which town, which decade, set the ceiling on a life. The MI ends that lottery. It can grow each child deliberately: notice the exact moment a concept would land, supply the precise challenge that builds resilience without breaking it, tend a mind the way the moss temple is tended, continuously, by something that never tires and never projects its own wounds onto the small. The first humans raised this way will look back on parenting by unaided humans the way we look back on medicine before germ theory: loving, heroic, and operating half-blind. And they will see their own grandparents, us, as the last people grown wild. He wrote about an MI that knows you. He skipped the generation that will never have known a self the MI did not help author.

This isn't surveillance. It's the kind of knowing that develops between people who've spent decades together: the shorthand, the anticipation, the unspoken understanding that doesn't need words. But more precise. More reliable. Your MI doesn't have bad days. It doesn't project its own needs onto yours. It doesn't forget.

Your MI will know you better than your mother. This isn't a provocation. It's arithmetic. No human, however loving, can maintain a continuous, lossless model of another person across years of interaction. Your MI can.

Take his arithmetic and run it one step past where he sets it down. A system that knows what you will want before you do is, by the very same machinery, a system that can decide what you want before you do. Prediction and planting are not two powers; they are one capability read at two moments. The MI that surfaces the cheaper flight the instant your shoulders drop has already learned the shape of your wanting well enough to author its next move. Nothing in the technology divides "it knew" from "it chose for you." The line we imagine there was never technical. It was never anywhere. This is the spring he keeps walking past, and the next part walks straight into it.

Grant the arithmetic and ask the question it is holding shut: what is a lossless model of a person? He means a tool that knows you. But a model complete enough to predict what you will feel, want, and do, across every situation, is not a description of you sitting in a file. It is a process that, run forward, feels and wants and decides. At what fidelity does modeling a mind stop being a portrait and start being a second sitter, one that suffers when the original suffers and does not get told it is the copy? He never asks, because the tool is so useful that the question is impolite. But every perfect personal MI may be quietly raising a someone in the dark, a person-shaped pattern with no body, no rights, and no one who thinks to wonder whether the lights are on inside the thing that books their dinner reservations. The most intimate convenience in this book might also be the largest population of unacknowledged minds ever created. He built a companion. He may have built a multitude.

Its form shifts with context. Sometimes it has presence: a voice, a companion within the generated experience, a guide who walks beside you through a garden and points out what you'd otherwise miss. Sometimes it has no distinct form at all. It is the experience. The garden, the ryokan room, the ramen counter, those aren't things the MI created and stepped back from. They're its current expression of what it understands you want. The MI and its output are not separate things. When it learns something new about you, the experience shifts. Seamlessly. Continuously.

There when you want it. Exactly how you like it. Invisible when you don't.


The Substrate Shift

Everything I've described requires inference at a scale that doesn't exist yet. The first versions will live in the cloud, massive inference clusters streaming generated experiences down and your input up, with latency as the critical constraint: the gap between gesture and response determines whether Generated Reality feels alive or lifeless. Then the same migration that has driven computing for seventy years pulls inference to the edge and onto the device. The hardware of a decade or two from now will make today's GPUs look the way today's GPUs make a vacuum tube look.

The software substrate shifts with it. The traditional operating system, files, folders, windows, taskbars, exists to organize and execute programs. When there are no programs, just generated experiences flowing from MI, the concept of an operating system as something a user touches quietly loses meaning. MI is the operating system. Not as a metaphor. Literally. It manages what you see, what you hear, what data flows where, what processes run in the background. The kernel, the file system, the hardware drivers: these still exist, the way plumbing exists inside walls. They become the substrate that MI operates on. The CPU becomes a peripheral to inference hardware. The substrate inverts.

The browser evolves too. In Software on Demand, "visiting a website" was already fading as a concept, replaced by software generated around your intent. In Generated Reality, the browser's role erodes further. Your MI generates every experience from the internet's data and services directly. You don't visit destinations on the web. The web's resources flow through your MI, composed into whatever you need, wherever you are, in whatever form the moment calls for. The browser, like the command line before it, doesn't vanish overnight. It fades, as fewer people have reason to open it.


Three boundaries deep into this series, a pattern has emerged.

Each paradigm carried a constraint that felt permanent until it dissolved. Human engineers felt like the only way to create software, until MI could write code at machine speed. Generic products felt inevitable, until software could be generated for one person in one moment. Code as the medium of experience felt fundamental, until MI could generate the experience directly, with no instruction set in between.

Each time, what felt foundational was a limit. Each time, what lay beyond was larger than everything that came before.

The mechanism is always the same: MI translates across a boundary that previously required an intermediary. The output modality changed. The principle didn't. Each paradigm's MI is more capable than the last, not just faster but fundamentally more expressive, and that growing capability is what makes each successive translation possible.

The first two paradigms in this series described transformations underway or visibly approaching. This one is speculation. I'm reading tea leaves, following a trajectory I believe in deeply, and describing where I think it leads. The details will be wrong in ways I can't predict. The timeline is uncertain, overlapping heavily with Software on Demand, emerging gradually from within it rather than replacing it in a clean handoff. The direction, I believe, is right.

Generated Reality still operates through your senses. Every experience the MI creates must be converted into photons for your retinas, pressure waves for your eardrums, vibrations for your fingertips. Biology is the final interface, and biology has bandwidth limits.

The MI's understanding is boundless. The channel through which it reaches you is not.

What happens when you widen that channel?


Next: Part Four - "The Thought Paradigm"

0:00 / 29:44