AI and the Illusion of Human-Algorithm Complementarity

the autonomous car, the new avatar of modernity, is being presented to us as an absolute priority. Following Uber and Google, most of the big automakers have launched enormous programs to try to beat each other to the finish line of the driverless-car race. It seems, from their view, that in 10 to 15 years drivers will be rare, consigned to the historical dustbin alongside cabbies.

Why this sudden zeal for self-driving cars? They are difficult and expensive to manufacture, and they pose significant ethical and moral problems. Few people actually complain about having to drive, and many consider it a pleasure (e.g., Brenan 2018). A commute offers a break in one of the last spaces of freedom and quiet in our ultra-regulated and cacophonous societies. So, it seems fair to ask, do we really want or need autonomous cars? While, on balance, we might answer "no," the proponents of progress have a trump card, which for many ends all argument: safety. Autonomous cars, we are told, will be safer and more reliable than human drivers, who are, allegedly, quite incompetent.

If reducing mortality rates were of paramount importance, then many other measures would be getting serious attention. For starters, we would prioritize train and bus transportation, make roads safer, and completely ban smartphone use while driving. More comprehensively, we would rethink urbanism and spatial organization to make us less dependent on cars. Beyond road fatalities, we would also be tackling other causes of mortality, such as cancers caused by glyphosates or industrial food. Compared to the zeal for self-driving cars, the enthusiasm for addressing these issues barely registers.

Interestingly, many of these other sources of danger—postwar urban design, industrial food systems, and such—share a common justification with autonomous vehicles. They, too, were conceived in order to increase efficiency and safety. Yet in seeking to make the world safer and more productive, these systems have actually made it more fragile. In trying to impose predictability and control on the world, they create structures that are unable to address unique situations or systemic instabilities. In this article, I seek to show how newly designed algorithmic systems share this same weakness, and more so. Aware of these limitations in dealing with unforeseen situations and data, programmers insist that human operators must remain in the loop, on hand to respond to novel or ethically complex situations. Yet, this solution ignores the fundamental dynamics of human-algorithm interaction. The premise of artificial intelligence (AI) is based on a critique of human capabilities, and its deployment serves to undermine human skills, know-how, and responsibility. Thus, a much deeper reconsideration of this technology is necessary if we are to have a genuinely safer and, critically, more human society.

THE EMERGENCE OF ARTIFICIAL INTELLIGENCE

The unassailable force of the safety argument is not what is propelling the campaign to impose autonomous cars. Rather, it is the very availability and rapid development of AI, which makes their autonomous operation possible. And self-driving cars are but one example of this imperative. Others include predictive justice, the automation of human resources, medical diagnosis—the list goes on and on. In each area, AI is being sold as a neutral, impartial, and omniscient replacement for an inadequate human rationality shaped by faulty intuitions and reasoning bias. Removing the human factor, AI will deliver improved efficiency and reduce, if not eliminate, risk.

AI combines and amplifies two trends operative since the beginning of the Industrial Revolution. In the nineteenth century, with the exploitation of new sources of energy and the development of new technologies, humans became the masters of space and time, freeing themselves from many difficult forms of physical labor. At the same time, another revolution was at work: the rise of the quantification of the world, with the creation of statistics as a discipline and as a technique for managing society (Rey 2016).

The replacement of physical labor by machines has now proceeded toward the replacement of cognitive effort. The dazzling progress of computer science and the cybernetic revolution, fostered by mathematician Alan Turing...