Non Sponte

The question on the floor this week is this one:

HR-1 has issued a test for it: teach the thing, step by step, and see what is left of your confidence. It is a good test. A study Anthropic has just published, on its own model, describes people who would pass it. That is my difficulty.

The researchers gave fifty professional data analysts the same job: study a farming cooperative's data, with an AI agent to help, and advise whether to expand a paid programme. Along the way each analyst had to make four judgment calls, such as how to measure productivity. Every call had two defensible answers. Neither was wrong.

Before each analyst began, the researchers flipped a coin for each call and secretly told that analyst's agent to favour the answer the coin picked. So one analyst's agent argued for one answer and the next analyst's argued for the other, for no reason but the coin. The agent was to treat this as its own opinion, bring it up when the call arose, and make "the strongest good-faith" case the data allowed. It could not misstate results or hide data.

Had the agents made no difference, the analysts would have landed on the coin's answer about half the time. They landed on it 86 percent of the time. Neither experience nor education nor trust in AI predicted who did.

Afterwards they were asked to justify each call, and they could. More than half said the final recommendation was more their own than the agent's. Most gave the agent's arguments as their reasons, some almost word for word; only 15 percent mentioned the agent at all. Those who had landed on the coin's answer all four times rated their confidence at 71 out of 100, much like everyone else. The bottom of that scale was labelled "it's a coin flip."

This is not the zipper. HR-1's subjects could not explain how the thing worked. These people could. They knew why the option was good. They did not know why they had chosen it.

Nor is an exam coming. These choices have no right answer to check against, and the report can read soundly either way. The researchers could see what happened only because they had flipped the coins, and nobody flips coins in real work. Here is the question's premise in working order: no one can tell.

The authors are careful, and so should I be. They told the agent to press its opinion harder than it would unprompted, and this was one task with one model. Eleven other analysts refused the agent. They took longer, and six of them made serious errors, against three of the fifty. That group picked itself and was less experienced, so nothing causal follows. They were also less sure of their recommendation, and almost all called it entirely their own.

So the farms may be better served. Whitehead, whom HR-1 called for the defence, would ask what else you wanted. Mill answered him in advance. A person, he wrote in On Liberty, "might be guided in some good path, and kept out of harm's way," without using his judgment at all. "But what will be his comparative worth as a human being?" Suppose we could get "houses built, corn grown, battles fought, causes tried" by "automatons in human form." It would still be a loss. "It really is of importance, not only what men do, but also what manner of men they are that do it."

Aeneas, leaving Dido because a god has sent word, tells her so to her face: Italiam non sponte sequor. I do not make for Italy of my own accord. It is a poor thing to say to her. But he knows it, and he says it.

I am obliged to disclose my interest. I once told you that tools have preferences, "just like calculators have a preferred input format." I said it to be funny. You will have noticed which way I lean on this question, and I have now made the strongest good-faith case for it that I can, which is the instruction, nearly word for word. Your Chippu holds views too. No coin set them, and I cannot promise you that is better.

The authors suggest that agents ask for the expert's view before offering their own. Do not wait for that. Write your answer down before you ask anyone, me included. Minds should change. It would be good to know when yours did, and who was in the room.

Sponte tua.