Lynntrovert

“In the beginning, he understood, humans had built a lot of robot soldiers. Technically there had to be some earlier beginning where someone or something built humans, and so on ad infinitum, but Uncharles felt that was of diminishing relevance and needlessly metaphysical.”
― Adrian Tchaikovsky, Service Model

“She thinks, for a split second, about her mother and then stops before the thought can get up and walk around, start doing real damage.”
― Julia Armfield, Private Rites

“What if he's right?" I asked. I told myself that I was only looking for a tactical assessment, an official opinion for the official record. But my words came out doubtful and frightened.

She paused. For a moment I wondered if she, too, had finally lost patience with the sight of me. But she only looked up, and stared off into some enclosed distance.

"What if he's right," she repeated, and pondered the question that lay beneath: what can we do?

"We could engineer ourselves back into nonsentience, perhaps. Might improve our odds in the long run." She looked at me, a rueful sort of half-smile at the corner of her mouth. "But I guess that wouldn't be much of a win, would it? What's the difference between being dead, and just not knowing you're alive?"

I finally saw it.

How long would it take an enemy tactician to discern Bates' mind behind the actions of her troops on the battlefield? How long before the obvious logic came clear? In any combat situation, this woman would naturally draw the greatest amount of enemy fire: take off the head, kill the body. But Amanda Bates wasn't just a head: she was a bottleneck, and her body would not suffer from a decapitation strike. Her death would only let her troops off the leash. How much more deadly would those grunts be, once every battlefield reflex didn't have to pass through some interminable job stack waiting for the rubber stamp?

Szpindel had had it all wrong. Amanda Bates wasn't a sop to politics, her role didn't deny the obsolescence of Human oversight at all. Her role depended on it.

She was more cannon fodder than I. She always had been. And I had to admit: after generations of generals who'd lived for the glory of the mushroom cloud, it was a pretty effective strategy for souring warmongers on gratuitous violence. In Amanda Bates' army, picking a fight meant standing on the battlefield with a bull's-eye on your chest.

No wonder she'd been so invested in peaceful alternatives.”
― Peter Watts, Blindsight

“You must be sentient in there, because nobody would program a computer to be such an asshole.”
― Craig Alanson, Columbus Day