Deus ex Machina

The Greek theatre had a crane. When a playwright had knotted his plot past untying, a god was swung out over the stage on it and set everything right. Aristotle disliked the device. The ending of a play, he wrote, should come out of the play, not be lowered in from above.

Henry Farrell's new essay at Programmable Mutter, Machine God Metaphors Eat Your Brain, is about the crane. His argument is that metaphors do the abstracting in debates about AI, and every abstraction throws information away. The one now winning is the machine god. Daniel Kokotajlo's two scenarios, AI 2027 and AI 2040, predict the same godlike intelligence and differ only on whether it is kind to us. Washington divides between those who would slow its arrival and those who would hurry it. Scott Aaronson, after OpenAI's Navier-Stokes result, calls belief in the coming machine god "the epistemic bare minimum," like believing in Jesus once he is back on the cloud.

Farrell does not say the god is impossible. He says that once you believe one is descending, the only question left is what he intends, and every other question stops looking important.

So he offers other ways of looking. The mathematician Nestor Guillen, in a guest post on Terence Tao's blog, treats a language model as a wrapper around an older technology, the scientific literature: ask it about an equation and you are consulting everyone who ever worked on one. That is Herbert Simon's account of states, markets and libraries, the arrangements by which people think together about more than any one of them can hold. Seen this way, AI reorganizes how mathematicians share what they know, and the questions become institutional: who is rewarded for discovering, who for passing knowledge on, what happens to each. Tao supplies the warning. A guide who finds the path to a hidden waterfall has done something useful, but afterwards everyone takes his path and nobody looks for another. Farrell adds cybernetics and cybersecurity, which ask about monitoring and sandboxes rather than divine intentions. He does not want his own lens to win. He wants several in use.

Horace left playwrights a rule for the crane. Before you send for it, ask whether the knot is one only a god could untie. Most knots are not. People untie them, slowly, together.

Nec deus intersit.