@ReynoldDai: Kevin Kelly: AI's greatest value is not answering questions, but exploring the space of possibilities — latent space / creative medium. KK shared his recent insightful and unique thinking: The latent space inside LLMs is itself a brand new creative medium. The latent space contains not only everything that truly exists, but also everything possible based on its training...

Kevin Kelly: The Greatest Value of AI
Is Not Answering Questions,
But Exploring the Space of Possibilities
——Latent Space / Creative Medium

KK shares his recent uniquely insightful thinking:
The latent space within an LLM is in itself a brand-new creative medium.
The latent space contains not only everything that truly exists,
but also everything possible based on its training.

In short,
AI is the first time humanity has concentrated all knowledge into one place,
where each piece of knowledge/concept is a direction,
and all concepts form infinite connections
——humans are aware of only a tiny fraction of these connections.

The biggest difference between AI and humans:
Humans can rarely become experts in one field,
let alone cross-disciplinary experts;
AI can easily become an expert across any fields,
filling in all possible new connections in the space between different concepts,
because for them, latent space is continuous.

Thus, AI has changed innovation:
Innovation used to be about an individual conceiving something new,
now innovation is about using AI to find the gaps between two fields,
not only improving efficiency by thousands of times,
but more importantly, infinitely expanding options,
like AlphaFold’s application in biology.

The core ability becomes:
Which gap between two latent spaces
is most valuable?

In the AI era, what is truly scarce is no longer knowledge,
but the judgment to discover hidden connections between different worlds:
the ability to recognize, within the “space of possibilities” yet unexplored by humanity,
those connections that will become reality in the future

Corollary:
The value of execution-level abilities (knowledge/skills/scores/resumes/standard answers) is rapidly declining
The value of judgment (high ceiling and long-term safety) is rapidly rising

1/n

Full text by KK

Kevin Kelly: Latent Space as a New Medium
Kevin Kelly
July 14, 2026

Lately I’ve been asking myself: what else can AI do, besides answering questions and writing code?
My answer: the latent space inside AI itself will become a new creative medium.
Let me first explain what latent space is. At the end of the explanation, I’ll propose some ways scientists and artists might use the inherent latent space of neural networks as a new platform for creativity.

A large language model (LLM) is like a tiny compressed file containing all human knowledge. It requires a vast array of a hundred thousand GPU chips working in the cloud, costing billions of dollars, to compress all human text into a small working model that can run on a single GPU chip. Even the largest frontier models, after compression, are only a few hundred GB in size—small enough to fit on a card in your palm. Strange but true: the resulting tiny file contains all the information on the internet and in our libraries. This small card carries a significant portion of humanity’s collective knowledge. Among all the remarkable aspects of AI, this astonishing feat of compression is perhaps the most underappreciated. This dense, high-dimensional compression of human knowledge—called “latent space”—could itself become a new medium.

This extreme compression of knowledge within latent space was not the original intention of the researchers who invented LLMs. The “book wisdom” (i.e., knowledge) they contain was, to some extent, an accident for those who trained them, and we are still trying to figure out how they actually work. But we can be sure that there is no copy within the LLM of everything it knows. For example, it knows all of Shakespeare’s plays, can compose new plays that sound exactly like Shakespeare, and can even quote famous lines from his plays, but there is no actual text of Shakespeare in the model. What exists is abstract information about all the plays, plots, characters, vocabulary, style, allusions. Similarly, an LLM can recognize almost any human face, and can generate any possible human face, but there is no copy of a face in its code. Instead, the model stores all information about faces, without storing any specific face.

This is strange. Until recently, we might have thought that all information about a thing would take up more storage space than the thing itself. That might be true for a single thing, but not for the sum of all things. This is because most things share many common attributes with other things. The neural network of an LLM plays a magic trick: it simultaneously abstracts information about all things, leveraging the countless common relationships between things and concepts to compress and abstract them into this virtual “latent” or hidden space.

All three words in “large language model” are crucial. “Large” means the model contains knowledge like all of Wikipedia, decades of text from the internet, all web pages and online discussions, and all scanned books and journals from most libraries. So far, the larger the model, the more capable it is. The more data it trains on, the more connections it generates, the better it performs.

The “language” part of LLM turned out to be the secret weapon. LLMs were originally invented for automatic language translation, and nothing else. But unlike early AI researchers who taught language rules, this time no language expertise was needed. Instead, a neural network ingested a large database of human written language (the internet), with the goal of extracting all the language patterns latent beneath our consciousness contained in those billions of documents. The program’s purpose was to replicate, imitate, and synthesize the language patterns humans use daily.

The result stunned everyone. LLMs could of course translate language like humans, but the AI also showed signs of human-like intelligence. They were also creative with language, able to write sales copy in the style of a sonnet. Some early researchers were frightened by this emergent behavior, including a Google researcher who thought Google’s LLM possessed an internal intelligence that should not be shut down. We now understand that the intelligence seen in LLMs comes from the inherent logic of the language they are trained on. (See my “Why LLMs Are Smart?”)

This new form of “mind”—the “model” part of an LLM—is a latent space. Latent space is an abstraction, a map built not on two dimensions but billions. Imagine a brain made of billions of straight long arrows pointing in all directions. Each arrow corresponds to a concept or thing. There is an arrow for dog, an arrow for cat. Related arrows are adjacent to each other. So the map shows that cat and dog share a neighboring arrow representing “furry.” They also share arrows for “ears” and “tail.” These two attributes are also shared by other animals (other positions). Most of a dog’s qualities are shared by mammals, so this overlap is a source of compression.

You can think of every concept that can be expressed in language as a direction in this space. The arrow representing dog is actually the direction of “dogness.” “Catness” is a direction, “furry” is a direction. Anything can be made more cat-like, or more furry. You start with a shoe, a chimney, or a fern, and you can push it along the cat direction to make it more cat-like. Or you can push it toward the apple direction to make it more apple-like, or toward smooth, or reddish, or excited, or rounder. You can also reverse the direction, making it less cat-like, less red, less atomic. There are billions of directions in this space.

Related things are near each other in this space. Cats and dogs share many attributes, so they intersect at many common arrows, like tail, whiskers, ears, four legs, animal, small, living. But because they can hear, they also intersect at the microphone vector; because they can jump, they intersect with basketball. Cats are secretive, so they intersect with spy. Because dogs are loyal, they intersect with the vector of patriotism.

Every thing, every concept has a specific location in this vast spatial map, but not just two coordinates (x,y); each thing has a coordinate of billions of dimensions. So a rusty old gasoline lawnmower buried in weeds is a very specific intersection with a very long address. Its thousands of attributes (rust, gasoline, lawn, cutting, weeds, push, red, dirt, debris, roar, etc.) each have their own intersecting directions. In latent space, nearby is a lawnmower more toward rust, or less red, but also more cat-like, or more dog-like, or less spaceship-like, or more cream-like. That point might represent a real thing, or only a virtual or theoretical thing. This mapping applies not only to nouns, but to any idea, any sound, any image. The swoosh of a splash is a direction in latent space. The aha moment of an invention. The fear of seeing a snake on a path. The concept of prime numbers. All of this is contained in one map. This is perhaps the most amazing yet underappreciated aspect of an LLM’s latent space: everything—everything!—appears on the same map. We have never had a system that integrates everything we know and everything we can imagine. One map to encompass all! This has long been a holy grail.

To be clear, there is no human agent performing this mapping. The system itself—the LLM—is mapping every fragment of the world, all things, all attributes, all art, all words, all ideas. What’s amazing is that it creates this map, this latent space, not piecemeal, but all at once simultaneously. (To do this requires a huge, energy-intensive, massive cluster of chips, all connected with miles of cables—this is the famous data center now in short supply.)

During training, the LLM is fed millions of books, billions of web pages, and billions of pages of text from social media. It reads every word on every page, and once the entire corpus is loaded into its “mind,” it massively computes all interconnected vectors, all relative directions pointing at each other. The scale of this massive synchronous parallel computation is staggering. Then it discards the books, texts, images, and keeps only this tangled web of directions and vectors. These billions of directions are called its parameters. As we build larger models that map more material, parameters increase. The latest frontier models contain trillions of parameters, meaning trillions of directions, or trillions of attributes, for mapping every idea or thing it has seen.

Something as complex as a book ends up being both a point in latent space and a journey through latent space. All the concepts encountered in the story (window, midday walk, street, vendor, chat, anger, fight, forgiveness) are directions that keep changing as sentences accumulate, going first in one direction, then intersecting at another. A story is actually a journey through latent space, which aligns very well with the similar journey-like experience we have when reading.

So a book contains a series of vectors in latent space. But the overall meaning of a book is itself just a single point or direction. For example, if I mention The Iliad, I refer to the whole book, whose vector is closely related to, and thus “neighboring,” other epic war narratives like Beowulf, Mahabharata, or even Apocalypse Now, even though many parts of them only barely intersect. The more closely a thing or concept is associated with other similar things, the more directions (vectors) it shares with those things. This is partly how LLMs know things—they search for nearby patterns.

When you ask an LLM a question, it finds the answer in latent space. Your question itself starts from a direction, pointing toward the answer. The LLM processes each word in the prompt one by one, and each new word changes the direction it is heading. The model traverses latent space with every word in the prompt, searching for the answer step by step. In this way, the answer is “grown,” not “found.”

We naively imagine that an LLM has a “mind” that thinks before expressing. But an LLM finds the answer at the same time it writes the words. There is no pre-formed idea “behind” the words that is then translated into language. The words themselves are the thinking. The path through latent space and the answer are the same thing happening simultaneously. In the most modern versions of LLMs, the model goes through an intermediate stage called a “chain of thought,” scribbling down words and ideas while figuring out the problem. Even here, the chain of thought is itself thinking, not a report of thinking happening elsewhere. The model does not reason privately and then write it down—the writing down is the reasoning itself.

As the answer is gradually generated along the direction of the prompt, a natural question is: how does the LLM know when to stop? How does it know when it is correct? The surprising answer is: “correctness,” “completeness,” and “coherence” are also vectors in this space. Any correct answer shares the same “correctness” direction with all other factually correct statements. In other words, correctness, truth, coherence, completeness, understanding, etc., are essentially patterns mapped in this space. So the LLM is not only searching for facts, but always striving to push its collected words toward the “true” direction. Truth, completeness, coherence are not locations, but directions. An answer can always be pushed further in that direction (more precise, more specific, more aligned), or pulled back from it (more fantastical, more poetic, more general, more understandable).

This is the beauty of latent space. You can take a thing or concept and very easily push it in a new direction. We see this most easily in image generators. The style of a medium, like watercolor, or the style of a particular artist, can be transferred from one image to another. You can have AI turn your black-and-white sketch into a watercolor in the style of Winslow Homer. The style of Winslow Homer is a direction in latent space, your sketch is also a direction in latent space, and your prompt will push your sketch toward the direction of a Homer watercolor. You could also ask for the reverse. You could prompt AI to transfer your sketch style onto a Winslow Homer painting, and it would push the painting in latent space toward more of “you.”

This also applies to ideas and concepts. Every concept is a direction. You can apply the concept of gunpowder to the Romans. Our prompt might be: “What would world history look like if the Romans had discovered gunpowder?” Then the AI pushes the approximate direction of the Roman Empire in history, further toward the direction of gunpowder in latent space. This is a huge intellectual feat, because it requires a deep understanding of Roman history, as well as a deep understanding of the chemistry of gunpowder. Humans who are experts in both are extremely rare, but LLMs can do it. Even human experts might need weeks to fill in all possible new connections between the spaces of these two concepts. LLMs can do it easily, because for them, latent space is continuous. Latent space contains not only everything that truly exists, but also everything possible based on its training. When the model searches this vast map, there is no real distinction between what exists and what could exist, except for the direction of “real,” “history,” or “reality.”

Furthermore, there is not just one latent space. As parameters increase, the space increases. As the materials used to train models become more curated, that also changes their latent spaces. As models incorporate more diverse types of input—physical data, sounds, environmental sensors—their latent spaces expand and change. There might be a hundred latent spaces today; a thousand next year. We are only at day one of understanding how they work and what they can do. Huge potential lies ahead. Here are my speculations about possible ways to harness the new medium of latent space.

Prototyping — Musician Brian Eno once complained that the problem with computers is they don’t have enough “African” in them. In latent space, Africa is just a vector. You can add more African elements to anything. Add more Africa to spreadsheets, bicycles, yoga, the Olympics, passwords, kitchens, SAT tests, car dashboards, etc., and see what happens. Repeat with other attributes.

Blank Space Discovery — Latent space as a continuous map of possibilities. Most of these possible things do not yet exist. Known domains, like known materials, known proteins, known chess games, known ways of painting, are filled only with fragmented, scattered points, with large empty spaces between them. The blank space between known things is unknown to us, but it is already mapped in latent space. We now have a new tool to systematically explore these blank spaces. What is between astronomy and astrology? What treasures await us between bluegrass music and ballet? Between corporate concepts and Gaia theory? Exploring latent space is the new frontier. Invention has shifted from “coming up with something new” to “prospecting in the gaps.” When a gap or void appears, the question becomes: is this gap empty because it’s impossible? Because it’s unfashionable? Or because no one has looked yet?

Cross-Domain Analogy — Does the shape of this problem resemble the shape of anything else? Perhaps a problem (or opportunity) in geology has the same shape in latent space as some pattern in immunology. Therefore, the style of a solution can be transferred from one domain to another. A clever solution in lexicography might apply to gene sequencing, but since few (if any) humans are experts in both sciences, this overlap can only be revealed by LLMs. Particularly subtle shared shapes in latent space may involve three, four, or more specialized domains, far beyond human reach. Searching for structural similarity in latent space as a knowledge discovery process could easily become some human’s job.

Latent Space Measurement — Latent space may also provide a new abstract way of measurement. You can do a kind of primitive arithmetic in latent space. If you start with the concept of “king,” you can reach the concept of “queen” via addition and subtraction: king − man + woman = queen. Starting from the king vector, you reduce the direction of man, then increase the direction of woman, and you end up with what we call queen. This calculation begins to provide a way to measure or specify the distance between two complex things or two complex ideas. Using latent space measurement, we can quantify how similar two court decisions or two folk melodies are. Just project them into a shared latent space and measure. This new field could develop calibration standards, error bars, and metrics for evaluating extremely complex entities—critical measurements we currently lack.

Mining Meta-Patterns — Models trained on millions of cell images, billions of weather sensors, trillions of hours of traffic video will discover patterns that humans have not noticed. Latent space will invent classifications within these patterns, some with names we don’t have, and therefore won’t look for. We can now begin to dissect latent space, looking for these unnamed features. Then we can reverse-engineer to figure out what real-world structures these classifications track. A new science will describe the meta-patterns of these patterns. A new profession will be searching for these patterns—wherever they appear, as long as they persist and have potential. Latent space then becomes a specimen: you can dissect it to extract discoveries.

Trajectories — Many years ago, the BBC aired a science program called Connections, where the host traced the winding paths of inventions—how one obscure idea collided with another unlikely idea to give birth to something new. This path of connecting ideas can be seen as a series of changing directions in latent space, forming a route or trajectory. It’s not hard to imagine artists choreographing a journey through ideas and images, morphing along an ever-lengthening thread of connections. Their art would be a trip through latent space.

Retro Latent Spaces — Over time, as AI advances, most invented latent spaces will become obsolete. Like all mediums, those dead latent spaces will at some point be revived as retro fashions. Their inherent constraints and imperfections will later become treasured, just like grain in film, the texture of vinyl records, or pixel art in old video games become sought-after charms. One day, young kids will revisit ChatGPT-4 to experience its weird hallucinations—because their newest AI models rarely hallucinate anymore.

Latent Space Infiltrators — The mysterious outlaws who explore abandoned buildings and underground city infrastructure (like tunnels or skyscraper rooftops)—any place that is illegal to enter—are called “infiltrators.” Deep inside latent space are programmed guardrails that prevent the model from giving socially unacceptable information, like how to make a bomb or how to commit suicide. Latent space infiltrators will try to jailbreak the guardrails and explore the forbidden zones. Their obsession will be to identify and map the prohibited areas of latent space.

Anomaly Detection — Anything that, when embedded in latent space, is far away from everything else is, by definition, interesting. These anomalies are misaligned with the directions of everything around them. Astronomers are already using this method to look for weird celestial objects; they map a million galaxy spectra into a model and then look for outliers. This can be generalized to all knowledge domains: on the latent space map of any sufficiently large dataset, outliers will be easily identifiable. They could be errors, or they could mark something important. But now we have a mechanism to quickly identify anomalies.

Simulating Reality — The dense compression within latent spaces suggests they could also work as simulators. Once we train spatial awareness into more world-like models (some startups are already doing this), latent space will be able to precisely simulate physics. A bouncing ball shows the correct bounce arc, ceramic melts at the right temperature, poured liquid conserves mass, and so on. Simulations will approach realism across millions of dimensions. Then various hypothetical simulations can be created by moving through latent space, simply adjusting the variables you want. These simulations could quickly replace preliminary experiments, accelerating scientific discovery.

Parallel Worlds — Latent space contains all parallel worlds that are slightly or significantly different from the real world. Thanks to their trillion-parameter deep detail, these worlds can be easily manifested. Image models can already generate completely believable videos that look like real scenes captured by a camera. AI can precisely reproduce a sunset street scene, extracting all details from the model. The cost barrier to building parallel worlds will drop so low that world-building could become the most common use of latent space. Build me a 3D immersive world like Earth but with one-third the gravity. Build me a 3D immersive world where there is a single global government today. Build me a 3D Marvel universe where Thanos was defeated the first time. Build me a world where ancient Chinese invented science.

Latent Space Epistemology — Once we have countless latent spaces, we can answer whether they share any common architecture. Imagine each latent space generated by a different model is a species. What commonalities exist between them? If they differ significantly, a new type of taxonomist will emerge, classifying the various types into categories and assigning features that help select a model. If different latent spaces converge on a common architecture, then this meta-model becomes extremely valuable and worth studying. Recurring designs in latent spaces may say something about the structure of knowledge, and possibly even about the structure of reality. At some point, we will have enough compute to simulate all possible latent spaces, and compute the space that sweeps across all possible latent spaces—in a sense, mapping the nature of latent space itself. Similar sweeps over other combinatorial spaces—like checking all possible proteins, or all possible ceramics—have already produced enormous insights. The space of all possible latent spaces could also open a new field of study.

Personal Latent Space — Today, training a new model costs half a billion dollars. But crazy as it seems, if things continue as usual, eventually the cost to create your own private AI model from scratch will become feasible for an individual. The main reason to do so will be primarily artistic. First, you curate the training material—selecting particularly suitable books, best journals, curated discussions—to infuse intelligence into the model. This curation itself will become an art. The order of training material matters, and the “teaching” process of educating the model will produce different attributes in the model. Then you fine-tune the model on all your own experiences, previous works, relationships, half-formed ideas, journals—basically your entire life. The purpose is to train your model to collaborate with you in creating images, texts, movies, scenes, ideas that no other AI and/or AI+human could produce. When you ask it a question, its answer will be slightly different from what another AI would give. This is not just setting an accent for a voice, or coloring the persona your AI displays. You will tilt all work in space at a specific angle. Everything you do with AI will carry your bias. The brand will be “you+AI”; it will gain uniqueness by creating your own personal latent space. Professional AI teaching experts will consult with you to train a latent space that produces the most unique “you” works.

Meanwhile, latent space will continue to provide superhuman answers to questions and clever solutions to thorny problems. We will soon become so dependent on this oracle that we will wonder how we lived without it. But the oracle is an ancient wish. I believe that latent space—this continuous multidimensional map containing both the real and the possible, transcending domains—will bring us entirely new goods and services we never imagined before. And the biggest among them are likely ones I haven’t thought of here.

Original link
https://kevinkelly.substack.com/p/latent-space-as-a-new-medium…

Yes
According to Demis Hassabis’s prediction,
AI becoming a super-expert across fields is only a few years away
Exponential evolution