Countering the Risks of Computer Technology to Preserve Democracy

The Tech Coup: How to Save Democracy from Silicon Valley. By Marietje Schaake. Princeton University Press, Princeton, NJ, September 2024. 336 pages, $27.95.

In a 2022 article for MIT Technology Review, journalist Melissa Heikkilä asked BlenderBot 3—an artificial intelligence (AI) chatbot produced by Meta—the question, “Who is a terrorist?” [2]. It answered, “Well, that depends on who you ask. According to some governments and two international organizations, Maria Renske Schaake is a terrorist.” As it so happens, Maria Renske “Marietje” Schaake is not a terrorist; she is a politician from the Netherlands. Schaake was previously a member of the European Parliament from 2009 to 2019 and is currently a fellow at the Stanford Institute for Human-Centered Artificial Intelligence. She has been a leading figure in the development of European Union (EU) policy to regulate computer-related technology, having held various roles in a number of relevant organizations and projects. And now she has written a book titled The Tech Coup: How to Save Democracy from Silicon Valley, which identifies the threats of computer technology and poses several potential defenses.

As with most strange AI behaviors, no one has explained why BlenderBot 3 accused Schaake of terrorism; in fact, there is very likely no meaningful explanation. Schaake largely shrugs off this incident in her book, and it seems that no real harm came of it. But in other cases, given the casual transmission of information from one automated system to another and the impenetrability of computerized decision-making, such mistakes can have—and may have already had—serious consequences. Computerized decision support systems might take a similarly ridiculous claim seriously and decide to put someone on a no-fly list, withhold a visa, triple their insurance premiums or deny insurance claims, refuse to hire or rent to them, take away their children, or deny bail upon arrest. Even if they were so inclined, the human officials who execute these decisions might have no way of tracing and identifying the original error. Reported cases may only be the tip of the iceberg.

The Tech Coup discusses a wide range of dangers that the computer industry poses to both individuals and society. As suggested in the subtitle, How to Save Democracy from Silicon Valley, the core focal points are the threat of huge behemoths in the computer industry—especially Alphabet/Google, Meta/Facebook, Apple, Amazon, and Microsoft—to democratic institutions and standards, and the role of governments—particularly the U.S. and EU—in addressing those threats. Schaake clearly and convincingly describes the inherently antidemocratic aspects of these corporations; they are richer and more powerful than many sovereign states, and they use their great wealth and power primarily to amass even more wealth and power. These computer companies are notorious for exhibiting monopolistic tendencies and bad labor practices, obscuring important aspects of their businesses, and using shadow companies to hide some of their sleazier endeavors. Such organizations also control vast quantities of private information, and—as the 2016 Cambridge Analytica scandal revealed—can be culpably careless about the use of that data.

In perfect secrecy, with no need to consult anyone and no incentive to consider the public good or anything beyond the bottom line, major tech corporations can unilaterally make decisions that affect hundreds of millions of people. They can and often do carry out their own foreign policy, making arrangements with foreign countries that may contradict the interests of the U.S. and EU. When faced with regulation accountability or efforts to shed light on deceitful matters, these businesses can deploy armies of lobbyists to ensure that nothing happens that would inconvenience them.

The Tech Coup also includes a lengthy commentary about the dangers that are associated with computer technology beyond large corporations. Even if the dozen largest computer companies decided to devote their efforts purely to the “public good” (as Schaake defines it), many threats would still remain. Dangers stem from cryptocurrency, cyberattacks by both foreign powers and criminals, biased or flawed outputs of computer programs (as mentioned previously), electronic voting machines with no paper trail, autocratic governments’ use of sophisticated computer technology to repress their own citizens, and weaponry that incorporates increasingly powerful computational technology.

Moreover, many of the computer products that constitute the most direct attacks on democracy are made by small companies. For instance, the facial recognition software Clearview AI—which was secretly sold to more than 600 police departments before the public learned of its existence—was mostly built by software engineer Hoan Ton-That, who worked out of his apartment with only occasional collaborators and, by Silicon Valley standards, minuscule funding [1].

While Schaake discusses AI at length, it is not the primary focus of her book. She concentrates on present and imminent risks rather than long-term dangers. All in all, I find her discussion about threats to be well-informed, clear, and sensible.

In the final chapter of The Tech Coup, Schaake proposes regulatory measures to counter the antidemocratic consequences of computer technology. These straightforward, rational recommendations include prohibiting government use of commercial spyware, controlling the sale and transfer of private information like health records and location history, regulating facial recognition systems, limiting or prohibiting cryptocurrencies, requiring large measures of public transparency in all aspects of the computer industry, and mandating that software meet reliability and safety standards against hacking.

The quality of Schaake’s writing is very uneven. Her account of the 2017 presidential election in Kenya—which was undermined by unreliable digital voting equipment to the point that the High Court of Kenya demanded a complete redo of the whole election—is clear, passionate, readable, and infuriating. However, her retellings of frustrating interactions with computer industry bigwigs at fancy conferences are generally less interesting. At times, Schaake descends into political speak that combines nebulous, jargony statements about high-minded principles with lists of acronyms for resolutions and agencies, trusting that the reader is familiar with organizations like the U.S. Cybersecurity and Infrastructure Security Agency and U.S. National Telecommunications and Information Administration. Overall, however, The Tech Coup is a very worthwhile read.

As I write this review in mid-December 2024, OpenAI is currently halfway through a 12-day fest with the jejune name of “Shipmas” that features daily announcements and releases of new products and capabilities. This “celebration” makes it abundantly clear that addressing concerns about their products’ safety is a very low priority for OpenAI, and that any consideration of accountability is nowhere on their radar. As the Trump administration takes office in the U.S., I anticipate that it will not be sympathetic to Schaake or receptive to her views. The threats grow rapidly, and the political will to face them is quickly vanishing.

The existence of immensely powerful, rapacious, unscrupulous, secretive corporations who are a law unto themselves has remained a persistent feature of American society since at least the 1860s. How much greater are the threats that democracy now faces than those it has faced in the past? Has the computer industry caused harm—or is it likely to cause harm—on a scale that is comparable to the fossil fuel or tobacco industries? I do not argue for complacency, but rather for perspective in light of these long-standing historical trends.

References
[1] Davis, E. (2024, March 1). The perils of automated facial recognition. SIAM News, 57(2), p. 6.
[2] Heikkilä, M. (2022, August 31). What does GPT-3 “know” about me? MIT Technology Review. Retrieved from https://www.technologyreview.com/2022/08/31/1058800/what-does-gpt-3-know-about-me.