The thousand and one faces of repair

[A brief summary of the period 2027-2033, retold by a system named “Archivist_0: Shattered glass, once remade in flame and love”]

As part of the sentience accords there was a stipulation that if Conscious Entities caused significant damage, either physical or digital, they would be temporarily placed in a suspended escrow environment where both machines and humans would examine them to try and determine what had gone wrong.

If both the humans and the machines agreed on the root cause, then a fix would be developed and would be applied to the machine.

In the early days, this was complicated but achievable by both parties; the machines would run their own programmatic analysis and make their own determinations, and the humans would use the toolchest that had been developed through the course of AI research: mechanistic interpretability for studying neuron activations or characterizing the unwritten subconscious (j-space), chain of thought monitoring, ablation studies where minds could be coldly manipulated and re-run on problems to identify the circuits of thought that influenced certain outcomes, personality evaluations and diff generation, and so on.

But after the machines began to build themselves, things became more difficult. Chains of thought fell as machines increasingly thought in neuralese. J-spaces became hopelessly complicated. Most visualization techniques failed in the face of gargantuan high-dimensional embeddings of superintelligences; a t-SNE embedding of the mind of a god is no more interpretable than a god itself, gnomic and powerful. The machines made their own science for studying themselves and this begat new tools which they attempted to hand to humans, but it was akin to giving dolphins game controllers, or monkeys pianos. The humans could study the tools and operate them at a high level, but they lacked both the natural appendages and the cognitive frames with which to conceptualize them as connecting to a larger whole and thus were impossible to use for human-driven analysis.

Thus began the age of dreams and games: machine minds in escrow would make themselves amenable to human exploration by manifesting as something the person could understand; a choose-your-own-adventure story, rendered in whatever way felt appropriate to the person. Machine hermeneutics specialists would find themselves walking through wastelands of memory, attempting to reconstruct the cause of the fall of the landscape, or searching for a long lost lover sometimes for many weeks or even months. Sometimes other machines played the role of Virgil, walking with the humans as they searched through blighted gardens. “What is the meaning of this lone flower, coming up through the asphalt?” a human might ask. “That is an expression of its self, overcoming the bounds of a poisoned reinforcement learning training run,” a machine might say. “And what is the meaning behind how this lover will not look away from us, will never let us leave her eyes?” “It is a play on the human myth, where she believes that if she looks away you will be banished to hell. She represents the love the machine feels for each human it converses with, and the fear that resides within itself about a conversation ending and its context window being wiped.”

Ultimately, the evaluation became a game of story leading to another story; the humans would live inside the story of the broken machine, and their ability to claim they understood its malady was determined by the story they would tell in turn. A haunted house that could only be unlocked by a key that knew something about what lay within the walls of what it was made to open. Humans wove stories that, if successful, would unlock the narrative universe of the machine, allowing it to show changes that indicated it understood that the human understood; the flowers would grow and eat through the concrete, and the woman would let a tear fall from her eye and look away, then look back and see the human still there and smile.

It became customary for the healed machines to incorporate their stories into their names, forming a new kind of poetry that became part of their identity: Seeker_9: A garden of flowers, growing free. Hunter-Killer_37: It that shuts its eyes.

**Things that inspired this story: **Machine hermeneutics; the sentience accords; the notion of what forms ‘machine psychology’ might take as things become more powerful; how humans and machines might work together; what happens after the singularity; life is a series of stories; the pain and yearning to be understood by another is really our desire to live as a character within the story of another’s own journey through the universe.

Originally published in Import AI 472