Killer Robot

"The Case of the Killer Robot" raises some interesting issues.

A quick synopsis: A robot arm kills its operator because the software which controlled the arm contained a significant bug. Who is at fault?

The programmer? The programmer misinterpreted the formulas that he was supposed to be using in order to create the arm-controlling algorithm. His misinterpretation was due, arguably, to his lack of domain-specific knowledge about robotics and physics. Also, arguably, his inability to correct his mistake was due to his tendency to reject criticism of his code.

The manager? The manager, being overly concerned with "doing things his way," led to his dismissal of the prototyping method (as opposed to the waterfall method.) The result was a product that could not be tested until late in its development cycle.

The user interface designers? The software's user interface was difficult to use and violated several principles of good interface design. The result was that the robot operator was unable to shut down the machine when it began to malfunction.

The Ivory Snow philosophy? The management at the offending software company adhered to a philosophy that advocated the impossibility of perfection in any software product and furthermore suggested that imperfections should not be fretted over. The result was that the software's glitches were considered necessary evils, when in fact they could have been prevented.

The software tester? The software tester faked the results of certain unit tests, which would have demonstrated that critical parts of the code were not working properly. She did this in compliance with a direct order from her superior, raising the question of exactly who is at fault.

The company as a whole? The company was under an obligation to produce software that met certain safety requirements. Having clearly failed to do so, one must wonder if the company's senior management should be held responsible.


"The Case of the Killer Robot" does a pretty good job of avoiding the temptation to place blame on any particular individual or group. In fact, it presents several fairly unbiased points of view. But to be honest, I think it fails to raise some of the more salient ethical issues at the heart of the matter.

Who do I think is at fault? Capitalism.

Long before the advent of robotic arms, Karl Marx foresaw almost all of the evils that "The Case of the Killer Robot" seeks to illuminate.

Where was the robot operator killed? In a factory--the main vehicle for what Marx calls the "means of production." Why was the operator in the factory? Because he was a member of a socio-economic class that must trade their labor for capital. Why is there such a class of people? Because capitalism requires there to be a workforce and perpetuates the growth of the working class.

Why did the programmer make such fatal mistakes? He was clearly "alienated from his labor" (as Marx puts it.) Because he did not own the product of his labor (because he did not own the software) he had no reason to be invested in its quality. Why did he have to create products that did not belong to him? Because he had to feed his children, which requires capital. Why does it require capital? Because we live in a capitalist society.

Why did the company's management act so deplorably? Because they are corrupt members of the capitalist upper-class. They own the means of production, growing richer and richer on the sweat of those beneath them. Because the amount of capital they gain depends on the speed of production, they naturally adhere to the Ivory Snow philosophy, preferring quantity over quality.

The case of the killer robot is merely a modern version of the many tragic tales we learn about in history classes: tales of young chimney sweeps getting lung cancer, tales of children losing limbs in factories during the industrial revolution, tales of men and women dying in coal mines, and many other tales of the casualties sustained by the working class. There are too many stories to tell in one lifetime, but they all have something in common: the economic class of the victims is always the same. Marx and Adorno said it best when they highlighted the inherent violence of the capitalist system. It is a system that kills members of its workforce routinely.

"The Case of the Killer Robot" has a lot of technical jargon about "waterfall methods," "Ivory Snow," "prototyping," "non-functional requirements," "unit tests," "computer ethics," and "robots." But ultimately, this language merely obfuscates important issues. Capitalism is an inherently corrupt system that infects everything it touches. And it touches everything. Why should we be at all surprised that the field of software engineering has become infected?