From 09efdd97a91e68933f3c1a4fa683716f035cbd0a Mon Sep 17 00:00:00 2001 From: Silvio Rhatto Date: Fri, 19 Apr 2019 13:07:45 -0300 Subject: Updates books/sociology/age-of-surveillance-capitalism.md --- books/sociology/age-of-surveillance-capitalism.md | 3231 ++++++++++++++++++++- 1 file changed, 3230 insertions(+), 1 deletion(-) (limited to 'books') diff --git a/books/sociology/age-of-surveillance-capitalism.md b/books/sociology/age-of-surveillance-capitalism.md index c8b791a..18ad2ae 100644 --- a/books/sociology/age-of-surveillance-capitalism.md +++ b/books/sociology/age-of-surveillance-capitalism.md @@ -239,7 +239,6 @@ mystique, closeted homosexuals, church-going atheists, and back-alley abortions. Eventually, though, they even produced people like you and me. - [...] The free-market creed originated in Europe as a sweeping defense against @@ -280,3 +279,3233 @@ competition for scarce resources. The disciplines of competitive markets promised to quiet unruly individuals and even transform them back into subjects too preoccupied with survival to complain. + + + [...] + + In Capital in the Twenty-First Century, the French economist Thomas + Piketty integrated years of income data to derive a general law of + accumulation: the rate of return on capital tends to exceed the rate of + economic growth. This tendency, summarized as r > g, is a dynamic that + produces ever-more-extreme income divergence and with it a range of + antidemocratic social consequences long predicted as harbingers of an + eventual crisis of capitalism. + + [...] + + Many scholars have taken to describing these new conditions as + neofeudalism, marked by the consolidation of elite wealth and power far + beyond the control of ordinary people and the mechanisms of democratic + consent.55 Piketty calls it a return to “patrimonial capitalism,” a + reversion to a premodern society in which one’s life chances depend + upon inherited wealth rather than meritocratic achievement.56 + + [...] + + We now have the tools to grasp the collision in all of its destructive + complexity: what is unbearable is that economic and social inequalities + have reverted to the preindustrial “feudal” pattern but that we, the + people, have not. We are not illiterate peasants, serfs, or slaves. + Whether “middle class” or “marginalized,” we share + + [...] + + Nevertheless, Occupy revealed a similar conflict between inequality’s + facts and inequality’s feelings, expressed in a creatively + individualized political culture that insisted on “direct democracy” + and “horizontal leadership.”60 Some analysts concluded that it was this + conflict that ultimately crippled the movement, with its “inner core” + of leaders unwilling to compromise their highly individualized approach + in favor of the strategies and tactics required for a durable mass + movement.61 However + + [...] + + This is the existential contradiction of the second modernity that + defines our conditions of existence: we want to exercise control over + our own lives, but everywhere that control is thwarted. + Individualization has sent each one of us on the prowl for the + resources we need to ensure effective life, but at each turn we are + forced to do battle with an economics and politics from whose vantage + point we are but ciphers. We live in the knowledge that our lives have + unique value, but we are treated as invisible + + [...] + + The deepest contradiction of our time, the social philosopher Zygmunt + Bauman wrote, is “the yawning gap between the right of self-assertion + and the capacity to control the social settings which render such + self-assertion feasible. It is from that abysmal gap that the most + poisonous effluvia contaminating the lives of contemporary individuals + emanate.” + + [...] + + When it comes to genuine economic mutation, there is always a tension + between the new features of the form and its mother ship. A combination + of old and new is reconfigured in an unprecedented pattern. + Occasionally, the elements of a mutation find the right environment in + which to be “selected” for propagation. This is when the new form + stands a chance of becoming fully institutionalized and establishes its + unique migratory path toward the future. But it’s even more likely that + potential mutations meet their fate in “transition failure,” drawn back + by the gravitational pull of established practices.63 + + [...] + + Among the many violations of advocacy expectations, ubiquitous + “terms-of-service agreements” were among the most pernicious.67 Legal + experts call these “contracts of adhesion” because they impose + take-it-or-leave-it conditions on users that stick to them whether they + like it or not. + + [...] + + These “contracts” impose an unwinnable infinite regress upon the user + that law professor Nancy Kim describes as “sadistic.” + + [...] + + The digital milieu has been essential to these degradations. Kim points + out that paper documents once imposed natural restraints on contracting + behavior simply by virtue of their cost to produce, distribute, and + archive. Paper contracts require a physical signature, limiting the + burden a firm is likely to impose on a customer by requiring her to + read multiple pages of fine print. Digital terms, in contrast, are + “weightless. + + [...] + + Radin calls this “private eminent domain,” a unilateral seizure of + rights without consent. She + + [...] + + Once firms understood that the courts were disposed to validate their + click-wrap and browse-wrap agreements, there was nothing to stop them + from expanding the reach of these degraded contracts “to extract from + consumers additional benefits unrelated to the transaction.”73 This + coincided with the discovery of behavioral surplus that we examine in + Chapter 3, as + + [...] + + 2008 two Carnegie Mellon professors calculated that a reasonable + reading of all the privacy policies that one encounters in a year would + require 76 full workdays at a national opportunity cost of $781 + billion.75 The numbers are much higher today. Still, most + + [...] + + These developments reflect the simple truth that genuine economic + reformation takes time and that the internet world, its investors and + shareholders, were and are in a hurry. The credo of digital innovation + quickly turned to the language of disruption and an obsession with + speed, its campaigns conducted under the flag of “creative + destruction.” That famous, fateful phrase coined by evolutionary + economist Joseph Schumpeter was seized upon as a way to legitimate what + Silicon Valley euphemistically calls “permissionless innovation.”77 + Destruction rhetoric promoted what I think of as a “boys and their + toys” theory of history, as if the winning hand in capitalism is about + blowing things up with new technologies. Schumpeter’s analysis was, in + fact, far more nuanced and complex than modern destruction rhetoric + suggests. + + [...] + + was as if a shark had been silently circling the depths all along, just + below the surface of the action, only to occasionally leap glistening + from the water in pursuit of a fresh bite of flesh + + [...] + + Over time, the shark revealed itself as a rapidly multiplying, + systemic, internally consistent new variant of information capitalism + that had set its sights on domination. An unprecedented formulation of + capitalism was elbowing its way into history: surveillance capitalism. + + [...] + + As we shall explore in detail throughout the coming chapters, thanks to + surveillance capitalism the resources for effective life that we seek + in the digital realm now come encumbered with a new breed of menace. + Under this new regime, the precise moment at which our needs are met is + also the precise moment at which our lives are plundered for behavioral + data, and all for the sake of others’ gain. The result is a perverse + amalgam of empowerment inextricably layered with diminishment. In the + absence of a decisive societal response that constrains or outlaws this + logic of accumulation, surveillance capitalism appears poised to become + the dominant form of capitalism in our time. + + [...] + + This left us wholly unprepared to defend ourselves from new companies + with imaginative names run by young geniuses that seemed able to + provide us with exactly what we yearn for at little or no cost. This + new regime’s most poignant harms, now and later, have been difficult to + grasp or theorize, blurred by extreme velocity and camouflaged by + expensive and illegible machine operations, secretive corporate + practices, masterful rhetorical misdirection, and purposeful cultural + misappropriation. On this road, terms whose meanings we take to be + positive or at least banal—“the open internet,” “interoperability,” and + “connectivity”—have been quietly harnessed to a market process in which + individuals are definitively cast as the means to others’ market ends. + + [...] + + The new harms we face entail challenges to the sanctity of the + individual, and chief among these challenges I count the elemental + rights that bear on individual sovereignty, including the right to the + future tense and the right to sanctuary. Each of these rights invokes + claims to individual agency and personal autonomy as essential + prerequisites to freedom of will and to the very concept of democratic + order. + + [...] + + The Spanish Data Protection Agency recognized that not all information + is worthy of immortality + + [...] + + As for the Spanish people, their Data Protection Agency, and the + European Court of Justice, the passage of time is likely to reveal + their achievements as a stirring early chapter in the longer story of + our fight for a third modern that is first and foremost a human future, + rooted in an inclusive democracy and committed to the individual’s + right to effective life. Their message is carefully inscribed for our + children to ponder: technological inevitability is as light as + democracy is heavy, as temporary as the scent of rose petals and the + taste of honey are enduring. + + [...] + + The point for us is that every successful vaccine begins with a close + understanding of the enemy disease. The mental models, vocabularies, + and tools distilled from past catastrophes obstruct progress. We smell + smoke and rush to close doors to rooms that are already fated to + vanish. The result is like hurling snowballs at a smooth marble wall + only to watch them slide down its facade, leaving nothing but a wet + smear: a fine paid here, an operational detour there, a new encryption + package there. + + [...] + + It is the habitat for progress “at the speed of dreams,” as one Google + engineer vividly describes it.100 My aim here is to slow down the + action in order to enlarge the space for such debate and unmask the + tendencies of these new creations as they amplify inequality, intensify + social hierarchy, exacerbate exclusion + + [...] + + Peter Drucker’s field studies for his seminal Concept of the + Corporation, the 1946 book that codified the practices of the twentieth + + [...] + + The closest thing we have to a Buck Weaver or James Couzens codifying + Google’s practices and objectives is the company’s longtime chief + economist, Hal Varian, who aids the cause of understanding with + scholarly articles that explore important themes. Varian has been + described as “the Adam Smith of the discipline of Googlenomics” and the + “godfather” of its advertising model.6 It is in + + [...] + + Nowadays there is a computer in the middle of virtually every + transaction… now that they are available these computers have several + other uses.”8 He then identifies four such new uses: “data extraction + and analysis,” “new contractual forms due to better monitoring,” + “personalization and customization,” and “continuous + experiments.”Varian’s discussions of + + [...] + + Data extraction and analysis,” Varian writes, “is what everyone is + talking about when they talk about big data.” “Data” are the raw + material necessary for surveillance capitalism’s novel manufacturing + processes. “Extraction” describes the social relations and material + infrastructure with which the firm asserts authority over those raw + materials to achieve economies of scale in its raw-material supply + operations. “Analysis” refers to the complex of highly specialized + computational systems that I will generally refer to in these chapters + as “machine intelligence.” I like this umbrella phrase because it + trains us on the forest rather than the trees, helping us decenter from + technology to its objectives. But in choosing this phrase I also follow + Google’s lead. The company describes itself “at the forefront of + innovation in machine intelligence,” a term in which it includes + machine learning as well as “classical” algorithmic production, along + with many computational operations that + + [...] + + Amit Patel, a young Stanford graduate student with a special interest + in “data mining,” is frequently credited with the groundbreaking + insight into the significance of Google’s accidental data caches. His + + [...] + + Google’s engineers soon grasped that the continuous flows of collateral + behavioral data could turn the search engine into a recursive learning + system that constantly improved search results and spurred product + innovations such as spell check, + + [...] + + was Google that recognized the gold dust in the detritus of its + interactions with its users and took the trouble to collect it up.… + Google exploits information that is a by-product of user interactions, + or data exhaust, which is automatically recycled + + [...] + + that early stage of Google’s development, the feedback loops involved + in improving its Search functions produced a balance of power: Search + needed people to learn from, and people needed Search + + [...] + + Hal Varian, who joined Google as its chief economist in 2002, would + note, “Every action a user performs is considered a signal to be + analyzed and fed back into the system.”16 The Page Rank algorithm, + named after its founder + + [...] + + The key point for us rests on a critical distinction. During this early + period, behavioral data were put to work entirely on the user’s behalf. + User data provided value at no cost, and that value was reinvested in + the user experience in the form of improved services: enhancements that + were also offered at no cost to users. Users provided the raw material + in the form of behavioral data, and those data were harvested to + improve speed, accuracy, and relevance and to help build ancillary + products such as translation. I call this the behavioral value + reinvestment cycle, in which all behavioral data are reinvested in the + improvement of the product or service (see Figure 1). + + [...] + + cycle was similarly oriented toward the individual as its subject, but + without a physical product to sell, it floated outside the marketplace, + an interaction with “users” rather than a market transaction with + customers. + + [...] + + Users are not paid for their labor, nor do they operate the means of + production, as we’ll discuss in more depth later in this chapter. + Finally, people often say that the user is the “product.” This is also + misleading, and it is a point that we will revisit more than once. For + now let’s say that users are not products, but rather we are the + sources of raw-material supply. As we shall see, surveillance + capitalism + + [...] + + impatient money + + [...] + + These behavioral data available for uses beyond service improvement + constituted a surplus, and it was on the strength of this behavioral + surplus that the young company would find its way to the “sustained and + exponential profits” that would be necessary for survival. Thanks to a + perceived + + [...] + + the New York Times reported, “The precision of the Carol Brady data was + eye-opening for some.” Even Brin was stunned by the clarity of Search’s + predictive power, revealing events and trends before they “hit the + radar” of traditional media. As he told the Times, “It was like trying + an electron microscope for the first time. It was like a + moment-by-moment barometer + + [...] + + Google maximizes the revenue it gets from that precious real estate by + giving its best position to the advertiser who is likely to pay Google + the most in total, based on the price per click multiplied by Google’s + estimate of the likelihood that someone will actually click on the + ad.”42 That pivotal multiplier was the result of Google’s advanced + computational capabilities trained on its most significant and secret + discovery: behavioral surplus + + [...] + + competitors, in which ads were targeted to keywords or content, were + unable to identify relevant ads “for a particular user.” Now the + inventors offered a scientific solution that exceeded the + most-ambitious dreams of any advertising executive + + [...] + + This new Google assures its actual customers that it will do whatever + it takes to transform the natural obscurity of human desire into + scientific fact. This Google is the superpower that establishes its own + values and pursues its own purposes above and beyond the social + contracts to which others are bound. + + [...] + + Google’s unique auction methods and capabilities earned a great deal of + attention, which distracted observers from reflecting on exactly what + was being auctioned: derivatives of behavioral surplus. Click-through + metrics institutionalized “customer” demand for these prediction + products and thus established the central importance of economies of + scale in surplus supply operations. Surplus capture would have to + become automatic and ubiquitous if the new logic was to succeed, as + measured by the successful trading of behavioral futures. + + [...] + + kind of commerce that depended upon online surveillance at scale. + Insiders referred to Google’s new science of behavioral + + [...] + + here was an unprecedented and lucrative brew: behavioral surplus, data + science, material infrastructure, computational power, algorithmic + systems, and automated platforms. This convergence produced + unprecedented “relevance” and billions of auctions. Click-through rates + skyrocketed. Work on AdWords and AdSense became just as important as + work on Search + + [...] + + their community effectively declared a “state of exception” in which it + was judged necessary to suspend the values and principles that had + guided Google’s founding and early practices. + + [...] + + Google’s inventions, their origins in emergency, and the 180-degree + turn from serving users to surveilling them. Most of all, he credited + the discovery of behavioral surplus as the game-changing asset that + turned Google into a fortune-telling giant, pinpointing Google’s + breakthrough transformation of the Overture model, when the young + company first applied its analytics of behavioral surplus to predict + the likelihood of a click: + + [...] + + Google loosed a new incarnation of capitalism upon the world, a + Pandora’s box whose contents we are only beginning + + [...] + + On the strength of Google’s inventions, discoveries, and strategies, it + became the mother ship and ideal type of a new economic logic based on + fortune-telling and selling—an ancient and eternally lucrative craft + that has fed on humanity’s confrontation with uncertainty from the + beginning of the human story. + + [...] + + The scientific and material complexity that supported the capture and + analysis of behavioral surplus also enabled the hiding strategy, an + invisibility cloak over the whole operation. “Managing search at our + scale is a very serious barrier to entry,” Schmidt warned would-be + competitors.79 To be sure, there are always sound business + + [...] + + public were told that Google’s magic derived from its exclusive + capabilities in unilateral surveillance of online behavior and its + methods specifically designed to override individual decision rights? + Google policies had to enforce secrecy in order to protect operations + that were designed to be undetectable because they took things from + users without asking and employed those unilaterally claimed resources + to work in the service of others’ purposes. + + [...] + + George Orwell once observed that euphemisms are used in politics, war, + and business as instruments that “make lies sound truthful and murder + respectable.”81 Google has been careful to camouflage the significance + of + + [...] + + Google discovered this necessary element of the new logic of + accumulation: it must assert the rights to take the information upon + which its success depends. + + [...] + + signing on with Facebook, the talented Sandberg became the “Typhoid + Mary” of surveillance capitalism as she led Facebook’s transformation + from a social networking site to an advertising behemoth. Sandberg + understood that Facebook’s social graph represented an awe-inspiring + source of behavioral surplus: the extractor’s equivalent of a + nineteenth-century prospector stumbling into a valley that sheltered + the largest diamond mine and the deepest gold mine ever to be + discovered. “We have better information than anyone else. We know + gender, age, location, and it’s real data as opposed to the stuff other + people infer,” Sandberg + + [...] + + Sandberg understood that through the artful manipulation of Facebook’s + culture of intimacy and sharing, it would be possible to use behavioral + surplus not only to satisfy demand but also to create demand. For + starters, that meant inserting advertisers into the fabric of + Facebook’s online culture, where they could + + [...] + + This new market form declares that serving the genuine needs of people + is less lucrative, and therefore less important, than selling + predictions of their behavior. Google discovered that we are less + valuable than others’ bets on our future behavior. This changed + everything. + + [...] + + VIII. Summarizing the Logic and Operations of Surveillance Capitalism + + [...] + + is obscene to suppose that this harm can be reduced to the obvious fact + that users receive no fee for the raw material they supply. That + critique is a feat of misdirection that would use a + + [...] + + remarkable questions here concern the facts that our lives are rendered + as behavioral data in the first place; that ignorance is a condition of + this ubiquitous rendition; that decision rights vanish before one even + knows that there is a decision to make; that there are consequences to + this diminishment of rights that we can neither see nor foretell; that + there is no exit, no voice, and no loyalty, only helplessness, + resignation, and psychic numbing; and that encryption is the only + positive action left to discuss when we sit around the dinner table and + casually ponder how to hide from the forces that hide from us. + + [...] + + Social theorist David Harvey builds on Arendt’s insight with his notion + of “accumulation by dispossession”: “What accumulation by dispossession + does is to release a set of assets… at very low (and in some instances + zero) cost. Overaccumulated capital can seize hold of such assets and + immediately turn them to profitable use.” He adds that entrepreneurs + who are determined to “join the system” and enjoy “the benefits of + capital accumulation” are often the ones who drive this + + [...] + + Even when knowledge derived from our behavior is fed back to us as a + quid pro quo for participation, as in the case of so-called + “personalization,” parallel secret operations pursue the conversion of + surplus into sales that point far beyond our interests. We have no + formal control because we are not essential to this market action. In + this future we are exiles from our own behavior, denied access to or + control over knowledge derived from its dispossession by others for + others. Knowledge + + [...] + + When asked about government regulation, Schmidt said that technology + moves so fast that governments really shouldn’t try to regulate it + because it will change too fast, and any problem will be solved by + technology. ‘We’ll move much faster than any government.’”26 Both Brin + and Page are even more candid in their contempt + + [...] + + Economic historians describe the dedication to lawlessness among the + Gilded Age “robber barons” for whom Herbert Spencer’s social Darwinism + played the same role that Hayek, Jensen, and even Ayn Rand play for + today’s digital barons. In the same way that surveillance capitalists + excuse their corporations’ unprecedented + + [...] + + There was no need for law, they argued, when one had the “law of + evolution,” the “laws of capital,” and the “laws of industrial + society.” John Rockefeller insisted that his outsized oil fortune was + the result of “the natural law of trade development.” Jay Gould, when + questioned by Congress on the need for federal regulation of railroad + rates, replied that rates were already regulated by “the laws of supply + and demand, production and consumption.”31 The millionaires mobilized + in 1896 to defeat the populist Democrat William Jennings Bryan, who had + vowed to tether economic policy to the political realm, including + regulating the railroads and protecting the people from “robbery + + [...] + + Surveillance After September 11, surveillance scholar David Lyon + + [...] + + After several decades in which data-protection officials, privacy + watchdogs, civil rights groups, and others have tried to mitigate + negative social effects of surveillance, we are witnessing a sharp tilt + toward more exclusionary and intrusive surveillance practices.”56 This + abrupt refocusing of governmental power and policy after the 9/11 + attacks in New York City and Washington, DC + + [...] + + With the attacks of September 11, 2001, everything changed. The new + focus was overwhelmingly on security rather than privacy.”61 The + privacy provisions debated just months earlier vanished from the + conversation more or less overnight. In both the US Congress and across + the EU, legislation was + + [...] + + including Germany (a country that had been highly sensitized to + surveillance under the hammer of both Nazi and Stalinist + totalitarianism), the UK, and France.62 In the US the failure to + “connect the dots” on the terrorist attack was a source of shame and + dismay that overwhelmed other concerns. Policy guidelines shifted from + “need to know” to “need to share” as agencies were urged to tear down + walls and blend databases for comprehensive information and analysis.63 + In a parallel development, privacy scholar Chris Jay + + [...] + + The elective affinity between public and private missions was evident + as early as 2002, when former NSA Chief Admiral John Poindexter + proposed his Total Information Awareness (TIA) program with a vision + that reads like an early guide to the foundational mechanisms of + behavioral surplus capture and analysis: + + [...] + + secret public-private intelligence collaborations that tend to be + “orchestrated around handshakes rather than legal formalities, such as + search warrants, and may be arranged this way to evade oversight and, + at times, to defy the law.”88 He observed that intelligence agencies + are irresistibly drawn to “and in some respects dependent upon” firms’ + privately held data resources.89 + + [...] + + former NSA Director Mike McConnell offered another glimpse into the + elective affinities between Google and the intelligence community. + Writing in the Washington Post, McConnell made clear that Google’s + surveillance-based operations in data capture, extraction, and analysis + were both taken for granted and coveted. Here the boundaries of private + and public melt in the intense heat of new threats and their + high-velocity demands that must be met in “milliseconds.” In + McConnell’s future there is one “seamless” surveillance empire in which + the requirements of self-preservation leave no opportunity for the + amenities of + + [...] + + Once again, history offers us no control groups + + [...] + + 1) the demonstration of Google’s unique capabilities as a source of + competitive advantage in electoral politics; (2) a deliberate blurring + of public and private interests through relationships and aggressive + lobbying activities; (3) a revolving door of personnel who migrated + between Google and the Obama administration, united by elective + affinities during Google’s crucial growth years of 2009–2016; and (4) + Google’s intentional campaign of influence over academic work and the + larger cultural conversation so vital to policy + + [...] + + Obama used his proximity to Schmidt to cement his own identity as the + innovation candidate poised to disrupt business as usual in + Washington.98 Once elected, Schmidt joined the Transition Economic + Advisory Board and appeared next to Obama at + + [...] + + Political correspondent Jim Rutenberg’s New York Times account of the + data scientists’ seminal role in the 2012 Obama victory offers a vivid + picture of the capture and analysis of behavioral surplus as a + political methodology. The campaign knew “every single wavering voter + in the country that it needed to persuade to vote for Obama, by name, + address, race, sex, and income,” and it had figured out how to target + its television ads to these individuals. One breakthrough was the + “persuasion score” that identified + + [...] + + According to the Center for Media and Democracy’s investigatory + research report, “The Googlization of the Far Right,” the corporation’s + 2012 list of grantees featured a new + + [...] + + Meanwhile, a list of Google Policy Fellows for 2014 included + individuals from a range of nonprofit organizations whom one would + expect to be leading the fight against that corporation’s + concentrations of information and power, including the Center for + Democracy and Technology, the Electronic Frontier Foundation, the + Future of Privacy Forum, the National Consumers League, the Citizen + Lab, and the Asociación por los Derechos Civiles.116 In July 2017 the + Wall Street Journal reported that + + [...] + + That summer, one of the New America Foundation’s most highly regarded + scholars and a specialist in digital monopolies, Barry Lynn, posted a + statement praising the EU’s historic decision to levy a $2.7 billion + fine on Google as the result of a multiyear antitrust investigation. + According to the New York Times and Lynn’s own account, New America’s + director bent to pressure from Schmidt, firing Lynn and his Open + Markets team of ten researchers. “Google is very aggressive in throwing + its money around Washington and Brussels, and then pulling strings,” + Lynn told the New York Times. “People are so afraid of Google now.” The + reporters cite Google + + [...] + + Google in the lead, surveillance capitalism vastly expanded the market + dynamic as it learned to expropriate human experience and translate it + into coveted behavioral predictions. Google and this larger + surveillance project have been birthed, sheltered, and nurtured to + success by the historical conditions of their era—second-modernity + needs, the neoliberal inheritance, and the realpolitik of surveillance + exceptionalism—as well as by their own purpose-built fortifications + designed to protect supply chain operations from scrutiny through + political and cultural capture. + + [...] + + the capture of behavioral surplus and the acquisition of decision + rights. Like a river running to the sea, if one route is blocked + + [...] + + increasingly ruthless cycle of kidnapping human experience, cornering + surplus supplies, and competing in new behavioral futures markets + + [...] + + The extraction imperative demands that everything be possessed. In this + new context, goods and services are merely surveillance-bound supply + routes. It’s not the car; it’s the behavioral data from driving the + car. It’s not the map; it’s the behavioral data from interacting with + the map. The ideal here is continuously expanding borders that + eventually describe the world and everything in it, all the time. + + [...] + + Traditionally, monopolies on goods and services disfigure markets by + unfairly eliminating competition in order to raise prices at will. + Under surveillance capitalism, however, many of the practices defined + as monopolistic actually function as means of cornering user-derived + raw-material supplies. There is no monetary price for the user to pay, + only an opportunity + + [...] + + The corporation unfairly impedes competitors in Search in order to + protect the dominance of its most important supply route, not primarily + to fix prices. These cornering operations are not abstractions, + + [...] + + products such as Android are valued more for supply than for sales. + Disconnect, Inc., founded in 2011 by two former Google engineers and a + privacy-rights attorney, developed + + [...] + + Google executive, noting that if other manufacturers switched to + Skyhook, it “would be awful for Google, because it will cut off our + ability to continue collecting data” for the company’s Wi-Fi location + database. Court documents from Skyhook’s eventual lawsuit against + Motorola (and Samsung) include an e-mail from Google’s senior vice + president of Mobile + + [...] + + Finally, extraordinary research from the French nonprofit Exodus + Privacy and the Yale Privacy Lab in 2017 documented the exponential + proliferation of tracking software. Exodus identified 44 trackers in + more than 300 apps for Google’s Android platform, some + + [...] + + For example, the ad tracker FidZup developed “communication between a + sonic emitter and a mobile phone.…” It can detect the presence of + mobile phones and therefore their owners by diffusing a tone, inaudible + to the human ear, inside a building: “Users installing + + [...] + + a pattern foreshadowed by the Google patent that we examined in Chapter + 3 and that we shall see repeatedly in the coming chapters, the research + findings emphasize that the always-on tracking is impervious to the + Android “permissions system,” despite its promises of user control.17 + + [...] + + Disconnect software was banned from Google Play’s vast catalog of + mobile apps, leading to Disconnect’s lawsuit against Google in 2015. + The startup’s complaint explains that “advertising companies including + Google use these invisible + + [...] + + dispossession operations reveal a predictable sequence of stages that + must be crafted and orchestrated in great detail in order to achieve + their ultimate destination as a system of facts through which surplus + extraction is normalized.The four stages of the cycle are incursion, + habituation, adaptation, and redirection. Taken together, these stages + constitute a “theory of change” that describes and predicts + dispossession as a political and cultural + + [...] + + with Google’s wider practice: it’s great to empower people, but not too + much, lest they notice the pilfering of their decision rights and try + to reclaim them. The firm wants to enable people to make + + [...] + + Google’s ideal society is a population of distant users, not a + citizenry. It idealizes people who are informed, but only in the ways + that the corporation chooses. It means for us to be docile, harmonious, + and, above all, grateful. + + [...] + + Within days, an independent analysis by German security experts proved + decisively that Street View’s cars were extracting unencrypted personal + information from homes. Google was forced to concede that it had + intercepted and stored “payload data,” personal information grabbed + from unencrypted Wi-Fi transmissions. As its apologetic blog post + noted, + + [...] + + Google’s “Spy-Fi” scandal filled headlines around the world. Many + believed that the Street View revelations would inflict irreparable + + [...] + + April 2012 FCC report is heart wrenching in its way, a melancholic + depiction of democracy’s vulnerability in the face-off with a wealthy, + determined, and audacious surveillance capitalist opponent. In November + 2010 the FCC sent Google a letter of inquiry + + [...] + + The second point is that in retrospect, one sees that the very idea of + a single rogue engineer was designed and elaborated as a brilliant + piece of misdirection, a classic scapegoating ploy. It directed + attention away from the ambitious and controversial agenda of the + extraction imperative toward a different narrative of a single infected + cell excised from the flesh of an enormous but innocent organism. All + that was left was to excise the infected flesh and let the organism + declare itself cured of its privacy kleptomania. Then—a return to the + streets + + [...] + + This is to say that her job was a logical impossibility. That she may + have nevertheless taken it seriously is suggested by + + [...] + + Street View’s redirection and elaboration announced a critical shift in + the orientation and ambition of the surveillance program: it would no + longer be only about routes, but about routing + + [...] + + For now, suffice to say that Street View and the larger project of + Google Maps illustrate the new and even more ambitious goals toward + which this cycle of dispossession would soon point: the migration from + an online data source to a real-world monitor to an advisor to an + active shepherd—from knowledge to influence to control. Ultimately, + Street View’s elaborate data would become the basis for another complex + of spectacular Google incursions: the self-driving car and “Google + City,” which we learn more about in Chapter 7. Those programs aim to + take surplus capture to new levels while opening up substantial new + frontiers for the establishment of behavioral futures markets in the + real world of goods and services. It is important to understand that + each level of innovation builds on the one before and that all are + united in one aim, the extraction of behavioral surplus at scale.In + this progression, Google perceives an opportunity + + [...] + + Google discovered by chance or intention the source of every mapmaker’s + power + + [...] + + The first US rectangular land survey captured this language perfectly + in its slogan: “Order upon the Land.”72 The cartographer is the + instrument of power as the author of that order, reducing reality to + only two conditions: the map and oblivion. The cartographer’s truth + crystallizes the message that Google and all surveillance capitalists + must impress upon all humans: if you are not on our map, you do not + exist + + [...] + + Google has done incrementally and furtively what would plainly be + illegal + + [...] + + done all at once.”98 + + [...] + + Nevertheless, the company made a canny decision not to disclose the + true extent of Cortana’s knowledge to its users. It wants to know + everything about you, but it does not want you to know how much it + knows or that its operations are entirely geared to continuously + learning more. Instead, the “bot” is programmed to ask for permission + and confirmation. The idea is to avoid spooking the public by + presenting Cortana’s intelligence as “progressive” rather than + “autonomous,” according to the project’s group program manager, who + noted that people do not want to be surprised by how much their phones + are starting to take over: “We made an explicit decision to be a little + less ‘magical’ and little + + [...] + + The Siren Song of Surveillance Revenues + + [...] + + PrecisionID + + [...] + + ID is then broadcast to every “unencrypted website a Verizon customer + visits from a mobile device. It allows third-party advertisers and + websites to assemble a deep, permanent profile of visitors’ web + browsing habits without their consent.”126 Alarmed by the threat of + fresh competition, Google, posing as a privacy advocate, launched a + campaign for a new internet protocol that would prevent “header + injections” such as Verizon’s PrecisionID.127 Privacy expert and + journalist Julia Angwin and + + [...] + + UIDH [unique identifier header], and expect that to be available soon.” + The New + + [...] + + capitalism gene therapy. As Verizon’s president of Operations told + investors, “For + + [...] + + The companies understood, and they persuaded Republican senators, that + the principle of consent would strike a serious blow to the + foundational mechanisms of the new capitalism: the legitimacy of + unilateral surplus dispossession, ownership rights to surplus, decision + rights over surplus, and the right to lawless space for the prosecution + of these activities.146 To this end the resolution also prevented + + [...] + + another trend, surveillance in the interest of behavioral surplus + capture and sale has become a service in its own right. Such companies + are often referred to as “software-as-a-service” or SaaS, but they are + more accurately termed “surveillance as a service,” or “SVaaS.” For + example, a new app-based approach to lending instantly establishes + creditworthiness based on detailed + + [...] + + You’re able to get in and really understand the daily life of these + customers,” explained the CEO of one lending company that analyzes + 10,000 signals per customer.151 Such methods were originally + + [...] + + Surveillance capitalism was born digital, but as we shall see in + following chapters, it is no longer confined to born-digital companies. + This logic for translating investment into revenue is highly adaptive + and exceptionally lucrative as long as raw-material supplies are free + and law is kept at bay. The rapid migration to surveillance revenues + that is now underway recalls the late-twentieth-century shift from + revenues derived from goods and services to revenues derived from + mastering the speculative and shareholder-value-maximizing + + [...] + + Who knows? Who decides? Who decides who decides + + [...] + + According to the philosopher of language John Searle, a declaration is + a particular way of speaking and acting that establishes facts out of + thin air, creating a new reality where there was nothing. Here is how + it works: sometimes we speak to simply describe the world—“you have + brown eyes”—or to change it—“Shut the door.” A declaration combines + both, asserting a new reality by describing the world as if a desired + change were already true: “All humans are created equal.” “They are + yours to command.” As Searle writes, “We + + [...] + + Searle concludes, “All of institutional reality, and therefore… all of + human civilization is created by… declarations + + [...] + + Instead, Durkheim trained his sights on the social transformation + already gathering around him, observing that “specialization” was + gaining “influence” in politics, administration, the judiciary, + science, and the arts. He concluded that the division of labor was no + longer quarantined in the industrial workplace. Instead, it had burst + through those factory walls to becoming the critical organizing + principle of industrial society. This is also an example of Edison’s + insight: that the principles of capitalism initially aimed at + production eventually shape the wider social and moral milieu. + “Whatever opinion one has about the division of labor,” Durkheim wrote, + “everyone knows that it exists, and is more and more becoming one of + the fundamental bases of the social order.”17 Economic imperatives + predictably mandated the division of labor in production, but what was + the purpose of the division of labor in society? This was the question + that motivated Durkheim’s analysis, and his century + + [...] + + What would hold society together in the absence of the rules and + rituals of clan and kin? Durkheim’s answer was the division of labor. + People’s needs for a coherent new source of meaning and structure were + the cause, and the effect was an ordering + + [...] + + conclusions are still relevant for us now. He argued that the division + of labor accounts for the interdependencies and reciprocities that link + the many diverse members of a modern industrial society in a larger + prospect of solidarity. Reciprocities breed mutual need, engagement, + and respect, all of which imbue this new ordering principle with moral + force + + [...] + + Britain, university administrators are already talking about a “missing + generation” of data scientists. The huge salaries of the tech firms + have lured so many professionals that there is no one left to teach the + next generation of students. As one scholar described it, “The real + problem is these people are not dispersed through society. The + intellect and expertise is concentrated in a small number of + companies.”32 + + [...] + + Under the regime of surveillance capitalism, the corporation’s + scientists are not recruited to solve world hunger or eliminate + carbon-based fuels. Instead, their genius is meant to storm the gates + of human experience, transforming it into data and translating it into + a new market colossus that creates wealth by predicting, influencing + + [...] + + under scrutiny, those long-awaited delivery trucks look more like + automated vehicles of invasion and conquest: more Mad Max than Red + Cross, more Black Sails than Carnival Cruise. The wizards behind their + steering wheels careen across every hill and hollow, learning how to + scrape and stockpile our behavior + + [...] + + Schmidt was, in fact, merely paraphrasing computer scientist Mark + Weiser’s seminal 1991 article, “The Computer for the 21st Century,” + which has framed Silicon Valley’s technology objectives for nearly + three decades. Weiser introduced what he called “ubiquitous computing” + with two legendary sentences: “The most profound technologies are those + that disappear. They weave themselves into the fabric of everyday life + until they are indistinguishable from it.” He described a new way of + thinking “that allows the computers themselves to vanish into the + background.… Machines that fit the human environment instead of forcing + humans to enter theirs will make using a computer as refreshing as + taking a walk in the woods.”2 + + [...] + + new phase, supply operations were enlarged and intensified to + accommodate economies of scope and economies of action. What does this + entail? The shift toward economies of scope defines a new set of aims: + behavioral surplus must be vast, but it must also be varied. These + variations are developed along two dimensions. The first is the + extension of extraction operations from the virtual world into the + “real + + [...] + + call economies of action. In order to achieve these economies, machine + processes are configured to intervene in the state of play in the real + world among real people and things. These interventions are designed to + enhance certainty by doing things: they nudge, tune, herd, manipulate, + and modify behavior in specific directions by executing actions as + subtle as inserting a specific phrase into your Facebook news feed, + timing the appearance of a BUY button on your phone, or shutting down + your car engine when an insurance payment is late. + + [...] + + means of behavioral modification.” The aim of this undertaking is not + to impose behavioral norms, such as conformity or obedience, but rather + to produce behavior that reliably, definitively, and certainly leads to + desired commercial results. The research director of Gartner, the + well-respected business advisory and research firm, makes the point + unambiguously when he observes + + [...] + + shadow text.4As the prediction imperative gathers force, it gradually + becomes clear that extraction was the first phase of a + far-more-ambitious project. Economies of action mean that real-world + machine architectures must be able to know as well as to do. Extraction + is not enough; now it must be twinned with execution. The extraction + architecture is combined with a new execution architecture, through + which hidden economic objectives are imposed upon the vast and varied + field of behavior.5 Gradually, as surveillance capitalism’s imperatives + + [...] + + is an extraordinary statement because there can be no such guarantees + in the absence of the power to make it so. This wider complex that we + refer to as the “means of behavioral modification” is the expression of + this gathering power. The prospect of guaranteed outcomes alerts us to + the force of the prediction imperative, which demands that surveillance + capitalists make the future for the sake of predicting it. Under this + regime, ubiquitous computing is not just a knowing machine; it is an + actuating machine designed to produce more certainty about us and for + them. + + [...] + + Finally, I want to underscore that although it may be possible to + imagine something like the “internet of things” without surveillance + capitalism, it is impossible to imagine surveillance capitalism without + something like the “internet of things.” Every command arising from the + prediction imperative requires this pervasive real-world material + “knowing and doing” presence. The new apparatus is the material + expression of the prediction imperative, and it represents a new kind + of power animated by the economic compulsion toward certainty. Two + vectors converge in this fact: the early ideals of ubiquitous computing + and the economic imperatives of surveillance capitalism. This + convergence signals the metamorphosis of the digital infrastructure + from a thing that we have to a thing that has us. + + [...] + + was coaxed to life nearly sixty years ago under the warm equatorial sun + of the Galapagos Islands, when a giant tortoise stirred from her torpor + to swallow a succulent chunk of cactus into which a dedicated scientist + had wedged a small machine. It was a time when scientists reckoned with + the obstinacy of free-roaming animals and concluded that surveillance + was the necessary price of knowledge. Locking these creatures in a zoo + would only eliminate the very behavior that scientists wanted to study, + but how were they to be surveilled? The solutions once concocted by + scholars of elk herds, sea turtles, and geese have been refurbished by + surveillance capitalists and presented as an inevitable feature of + twenty-first-century life on Earth. All that has changed is that now we + are the animals + + [...] + + If you’re not in the system, you don’t exist + + [...] + + dark data + + [...] + + This means that surplus must be both plentiful (economies of scale) and + varied (economies of scope) in both range and depth. + + [...] + + gamification + + [...] + + all else fails, insurers are advised to induce a sense of inevitability + and helplessness in their customers. Deloitte counsels companies to + emphasize “the multitude of other technologies already in play to + monitor driving” and that “enhanced surveillance and/or geo-location + capabilities are part of the world we live in now, for better or + worse.”46 + + [...] + + Behavioral data drawn from their experience are processed, and the + results flow in two directions. First, they return to the drivers, + executing procedures to interrupt and shape behavior in order to + enhance the certainty, and therefore profitability, of predictions + (economies of action). Second, prediction products that rank and sort + driver behavior flow into newly convened behavioral futures markets in + which third parties lay bets on what drivers will do now, soon, and + later: Will + + [...] + + uncontract + + [...] + + This is not the automation of society, as some might think, but rather + the replacement of society with machine action dictated by economic + imperatives. The uncontract is not + + [...] + + Despite its pervasiveness both in Silicon Valley and in the wider + culture of data scientists and technology developers, inevitabilism is + rarely discussed or critically evaluated. Paradiso + + [...] + + Paradiso imagines a society in which it falls to each individual to + protect herself from the omniscient ubiquitous sensate computational + systems of the new apparatus. Rather than paradise, it seems a recipe + for a new breed of madness. Yet this is precisely the world that is now + under construction around us, and this madness appears to be a happy + feature of the plan. + + [...] + + Cisco Kinetic gets the right data to the right applications at the + right time… while executing policies to enforce data ownership, + privacy, security and even data sovereignty laws.”73 But, as is so + often the case, the most audacious effort to transform the urban + commons into the surveillance capitalist’s equivalent of Paradiso’s + 250-acre marsh comes from Google, which has introduced and legitimated + the concept of the “for-profit city.” Just as MacKay had counseled and + Weiser proselytized, the computer would be operational everywhere and + detectable nowhere, always beyond the edge of individual awareness. In + 2015, shortly after Google reorganized + + [...] + + We fund it all… through a very novel advertising model.… We can + actually then target ads to people in proximity, and then obviously + over time track them through things like beacons and location services + as well as their browsing activity + + [...] + + As we are shorn of alternatives, we are forced to purchase products + that we can never own while our payments fund our own surveillance and + coercion. Adding insult to injury, data rendered by this wave of things + + [...] + + life pattern marketing” based on techniques derived from military + intelligence known as “patterns of life analysis + + [...] + + It allows you to tap into people’s compulsive nature by encouraging + impulse buys with the notifications you send out.… It also allows you + to gain insight on your current customers by reading what they’re + saying on Yelp and Facebook.…”24 Another mobile marketing firm + recommends + + [...] + + November 2017 Quartz investigative reporters discovered that since + early 2017, Android phones had been collecting location information by + triangulating the nearest cell towers, even when location services were + disabled, no apps were running, and no carrier SIM card was installed + in the phone. The information was used to manage Google’s “push” + notifications and messages sent to users on their Android phones, + enabling the company to track “whether an individual with + + [...] + + databases of ruin.”34 + + [...] + + The company built an “employment index” for the national economy as + well as a “consumption index.” It also touted its ability to generate + quite-specific predictions such + + [...] + + The agencies’ well-meaning guidelines overlook the inconvenient truth + that transparency and privacy represent friction for surveillance + capitalists in much the same way that improving working conditions, + rejecting child labor, or shortening the working day represented + friction for the early industrial capitalists. It took targeted laws to + change working conditions back + + [...] + + The only real protection is when an app randomly but regularly + generates a new MAC address for your phone, but of the nine trackers, + only Apple’s performed this operation. The report also identifies a + general pattern of careless + + [...] + + other words, privacy policies are more aptly referred to as + surveillance policies, and that is what I suggest we call them. There + are many new territories of body rendition + + [...] + + Nobody reckoned with the fact that the prediction imperative makes + individual ignorance the preferred condition for rendition operations, + just as Arendt had observed and Mackay had prescribed for animals in + the wild. Original sin prefers the dark. The talks continued without + the advocates, and + + [...] + + 1767 the political economist Nathaniel Forster worried that + “fashionable luxury” was spreading “like a contagion,” and he + complained of the “perpetual restless ambition in each of the inferior + ranks to raise themselves to the level of those immediately above + them.”4 Adam Smith wrote insightfully on this social process, noting + that upper-class luxuries can in time be recast as “necessaries.” This + occurs as “the established rules of decency” change to reflect new + customs introduced by elites, triggering lower-cost production methods + that transform what was once unattainable into newly affordable goods + and services.5 Ford’s Model T + + [...] + + Conversation” stands alone in its promise to dominate raw-material + supply, and the rewards to the One Voice would be astronomical. Casual + talk helps to blur the boundaries between “it”—the apparatus saturated + with commercial agents—and us. In conversation we imagine friendship. + The more we fancy the apparatus as our confidante, nanny, governess, + and support system—a disembodied, pervasive “Mrs. Doubtfire” for each + person—the more experience we allow it to render, and the richer its + supply operations grow. Communication is the first human joy, and a + conversational interface is prized for the frictionless ease in which a + mere utterance can trigger action + + [...] + + 2018, Amazon had inked deals with home builders, installing its Dot + speakers directly into ceilings throughout the house as well as Echo + devices and + + [...] + + Mark Zuckerberg’s unilateral upending of established privacy norms in + 2010, when he famously announced that Facebook users no longer have an + expectation of privacy. Zuckerberg had described the corporation’s + decision to unilaterally release users’ personal information, + declaring, “We decided that these would be the social norms now, and we + just went for it.”55 Despite + + [...] + + The personalization project descends deeper toward the ocean floor with + these new tools, where they lay claim to yet a new frontier of + rendition trained not only on your personality but also on your + emotional life. If this project of surplus from the depths is to + succeed, then your unconscious—where feelings form before there are + words to express them—must be recast as simply one more source of + raw-material supply for machine rendition and analysis, all of it for + the sake of more-perfect prediction. As a market research report on + affective computing explains, “Knowing the real-time emotional state + can help businesses to sell their product and thereby increase + revenue.”88 Emotion analytics products such as SEWA use + + [...] + + Conditioning” is a well-known approach to inducing behavior change, + primarily associated with the famous Harvard behaviorist B. F. Skinner. + He argued that behavior modification should mimic the evolutionary + process, in which naturally occurring behaviors are “selected” for + success by environmental conditions. Instead of the earlier, more + simplistic model of stimulus/response, associated with behaviorists + such as Watson and Pavlov, Skinner interpolated a third variable: + “reinforcement.” In his laboratory work with mice and pigeons, Skinner + learned how to observe a range of naturally occurring behaviors in the + experimental animal and then reinforce the specific action, or + “operant,” that he wanted the animal to reproduce. Ultimately, he + mastered intricate designs or “schedules” of reinforcement that could + reliably shape precise behavioral routines. Skinner called the + application + + [...] + + Conditioning at scale is essential to the new science of massively + engineered human behavior.” He believes that smartphones, wearable + devices, and the larger + + [...] + + Varian endorsed and celebrated this self-authorizing experimental role, + warning that all the data in the world “can only measure correlation, + not causality.”3 Data tell what happened but not why it happened + + [...] + + element in the construction of high-quality prediction products—i.e., + those that approximate guaranteed outcomes—depends upon causal + knowledge. As Varian says, “If you really want to understand causality, + you have to run experiments. And if you run experiments continuously, + you can continuously improve your system.”4 + + [...] + + Psychologists have found that the more a person can project himself or + herself into the feelings of another and take the other’s perspective, + the more likely he or she is to be influenced by subliminal cues, + including hypnosis. Empathy orients people toward other people. It + allows one to get absorbed in emotional experience and to resonate with + others’ experiences + + [...] + + Facebook’s persistence warns us again of the dispossession cycle’s + stubborn march. Facebook had publicly acknowledged and apologized for + its overt experimental incursions into behavior modification and + emotional manipulation, and it promised adaptations to curb or mitigate + these practices. Meanwhile, a new threshold of intimate life had been + breached. Facebook’s potential mastery of emotional manipulation became + discussable and even taken for granted as habituation set in. From + Princeton’s Fiske to critic Grimmelmann and supporter Meyer, the + experts believed that if Facebook’s activities were to be forced into a + new regulatory regime, the corporation would merely continue in secret + + [...] + + Individual awareness is the + + [...] + + The evasion of individual and group awareness was critical to + Facebook’s behavior-modification success, just as MacKay had + stipulated. The first paragraph of the research article on emotional + contagion celebrates this evasion: “Emotional states can be transferred + to others via emotional contagion, leading people to experience the + same emotions without their awareness.” Nor do the young adults of + Australia’s great cities suspect that the precise measure of their + fears and fantasies is exploited for commercial result at the hour and + moment of their greatest vulnerability. + + [...] + + enemy of telestimulation because it is the necessary condition for the + mobilization of cognitive and existential resources. There is no + autonomous judgment without awareness. Agreement and disagreement, + participation and withdrawal, resistance or collaboration: none of + these self-regulating choices can exist without awareness. + + [...] + + Indeed, some theorists have suggested that the primary purpose of self + awareness is to enable self-regulation.” Every threat to human autonomy + begins with an assault on awareness, “tearing down our capacity to + regulate our thoughts, emotions, and desires.”22 + + [...] + + salience of self-awareness as a bulwark against self-regulatory failure + is also underscored in the work of two Cambridge University researchers + who developed a scale to measure a person’s “susceptibility to + persuasion.” They found that the single most important determinant of + one’s ability to resist persuasion is what they call “the ability to + premeditate.”23 This means that people who harness self-awareness to + think through the consequences of their actions are more disposed to + chart their own course and are significantly less vulnerable to + persuasion techniques. Self-awareness also figures in the + second-highest-ranking factor on their scale: commitment. People who + are consciously committed to a course of action or set of principles + + [...] + + We have seen already that democracy threatens surveillance revenues. + Facebook’s practices suggest an equally disturbing conclusion: human + consciousness itself is a threat to surveillance revenues, as awareness + endangers the larger project of behavior modification. Philosophers + recognize “self-regulation,” “self-determination,” and “autonomy” as + “freedom of will.” The word autonomy derives from the Greek and + literally means “regulation by the self.” It stands in contrast to + heteronomy, which means “regulation by others.” The competitive + necessity of economies of action means that surveillance capitalists + must use all means available to supplant autonomous action with + heteronomous action. + + [...] + + However, it would be dangerous to nurse the notion that today’s + surveillance capitalists simply represent more of the same. This + structural requirement of economies of action turns the means of + behavioral modification into an engine of growth. At no other time in + history have private corporations of unprecedented wealth and power + enjoyed the free exercise of economies of action supported by a + pervasive global architecture of ubiquitous computational knowledge and + control constructed and maintained by all the advanced scientific + know-how that money can buy. + + [...] + + Most research on games concludes that these structures can be effective + at motivating action, and researchers generally predict that games will + increasingly be used as the methodology of choice to change individual + behavior.34 In practice, this has meant that the power of games to + change behavior is shamelessly instrumentalized as gamification spreads + to thousands of situations in which a company merely wants to tune, + herd, and condition the behavior of its customers or employees toward + its own objectives + + [...] + + One analyst compiled a survey of more than ninety such “gamification + cases,” complete with return-on-investment statistics.35 Ian Bogost, a + professor of interactive computing at Georgia Tech and a digital + culture observer, insists that these systems should be called + “exploitationware” rather than games because their sole aim is behavior + manipulation and modification.36 + + [...] + + The zeal for Pokémon Go gradually diminished, but the impact of Hanke’s + accomplishments is indelible. “We’ve only just scratched the surface,” + Hanke told a crowd of fans.46 The game had demonstrated that it was + possible to achieve economies of action on a global scale while + simultaneously directing specific individual actions toward precise + local market opportunities where high bidders enjoy an ever-closer + approximation of guaranteed outcomes. Niantic’s distinctive + accomplishment + + [...] + + TechCrunch noted the game’s “precise location tracking” and “ability to + perform audio fingerprinting” through its access to your camera and + microphone, concluding, “So it’s prudent to expect some of your + location data to end up in Google’s hands.”48 The Electronic Privacy + Information Center noted in a letter of complaint to the Federal + + [...] + + However, it does not acknowledge that its services operate on two + levels: game services for players and prediction services for Niantic’s + customers. The company concedes that it uses third-party services, + including Google’s, to “collect and interpret data,” but it is careful + to sidestep the aims of those analyses.51 The seven-page letter + mentions “sponsored + + [...] + + The genius of Pokémon Go was to transform the game you see into a + higher-order game of surveillance capitalism, a game about a game + + [...] + + the end we recognize that the probe was designed to explore the next + frontier: the means of behavioral modification. The game about the game + is, in fact, an experimental facsimile of surveillance capitalism’s + design for our future + + [...] + + Thus began a morbidly fascinating and often bizarre chapter in the + history of American spy craft.55 Much of the new work was conducted in + the context of the CIA’s highly classified MKUltra project, which was + tasked with “research and development of chemical, biological, and + radiological materials capable for employment in clandestine operations + to control human behavior + + [...] + + Another factor was the 1971 publication of B. F. Skinner’s incendiary + social meditation Beyond Freedom & Dignity. Skinner prescribed a future + based on behavioral control, rejecting the very idea of freedom (as + well as every tenet of a liberal society) and cast the notion of human + dignity as an accident of self-serving narcissism + + [...] + + First Amendment, the subcommittee argued, “must equally protect the + individual’s right to generate ideas,” and the right to privacy should + protect citizens from intrusions into their thoughts, behavior, + personality, and identity lest these concepts “become meaningless.” It + was in this context that Skinnerian behavioral engineering was singled + out for critical examination + + [...] + + Where is the hammer of democracy now, when the threat comes from your + phone, your digital assistant, your Facebook login? Who will stand for + freedom now, when Facebook threatens to retreat into the shadows if we + dare to be the friction that disrupts economies of action that have + been carefully, elaborately, and expensively constructed to exploit our + natural empathy, elude our awareness, and circumvent our prospects for + self-determination? If we fail to take notice now, how long before we + are numb to this incursion and to all the incursions? How long until we + notice nothing at all? How long before we forget who we were before + they owned us, bent over the old texts of self-determination in the dim + + [...] + + Now we know that surveillance capitalists’ ability to evade our + awareness is an essential condition for knowledge production. We are + excluded because we are friction that impedes + + [...] + + The commodification of behavior under the conditions of surveillance + capitalism pivots us toward a societal future in which an exclusive + division of learning is protected by secrecy, indecipherability, and + expertise. Even when knowledge derived from your behavior is fed back + to you in the first text as a quid pro quo for participation, the + parallel secret operations of the shadow text capture surplus for + crafting into prediction products destined for other marketplaces that + are about you rather than for you. These markets do not depend upon you + except first as a source of raw material from which surplus is derived, + and then as a target for guaranteed outcomes + + [...] + + this future we are exiles from our own behavior, denied access to or + control over knowledge derived from our experience. Knowledge, + authority, and power rest with surveillance capital, for which we are + merely “human natural resources + + [...] + + Centuries of debate have been levied on the notion of free will, but + too often their effect has been to silence our own declarations of + will, as if we are embarrassed to assert this most fundamental human + fact. I recognize my direct experience of freedom as an inviolate truth + that cannot be reduced to the behaviorists’ formulations of life as + necessarily accidental and random, shaped by external stimuli beyond my + knowledge or + + [...] + + influence and haunted by irrational and untrustworthy mental processes + that I can neither discern nor avoid + + [...] + + American philosopher John Searle, whose work on the “declaration” we + discussed in Chapter 6, comes to a similar conclusion in his + examination of “free will.” He points to the “causal gap” between the + reasons for our actions and their enactment. We may have good reasons + to do something, he observes, but that does not necessarily mean it + will be done. “The traditional name of this gap in philosophy is ‘the + freedom of the will.’” In response to the “sordid history” of this + concept, he reasons, “even if the gap is an illusion it is one we + cannot shake off.… The notion of making and keeping promises + presupposes the gap.… [It] requires consciousness and a sense of + freedom on the part of the promise-making and promise-keeping agent + + [...] + + Our freedom flourishes only as we steadily will ourselves to close the + gap between making promises and keeping them. Implicit in this action + is an assertion that through my will I can influence the future. It + does not imply total authority over the future, of course, only over my + piece + + [...] + + should an experience as elemental as this claim on the future tense be + cast as a human right? The short answer is that it is only necessary + now because it is imperiled. Searle argues that such elemental + “features of human life” rights are crystallized as formal human rights + only at that moment in history when they come under systematic threat. + So, for example, the ability to speak is elemental. The concept of + “freedom of speech” as a formal right emerged only when society evolved + to a degree of political complexity that the freedom to speak came + under threat. The philosopher observes that speech is not more + elemental to human life than breathing or being able to move one’s + body. No one has declared a “right to breathe” or a “right to bodily + movement” because these elemental rights have not come under attack and + therefore do not require formal protection. What counts as a basic + right, Searle argues, is both “historically contingent” and “pragmatic + + [...] + + Most simply put, there is no freedom without uncertainty; it is the + medium in which human will is expressed in promises. Of course, we do + not only make promises to ourselves; we also make promises to one + another. When we join our wills and our promises, we create the + possibility of collective action toward a shared future, linked in + determination to make our vision real in the world. This is the origin + of the institution we call “contract,” beginning with the ancient + Romans.6 Contracts originated as shared “islands of predictability” + intended to mitigate uncertainty for the human community, and they + still retain this meaning. “The simplest way + + [...] + + The uncontract aims instead for a condition that the economist Oliver + Williamson describes as “contract utopia”: a state of perfect + information known to perfectly rational people who always perform + exactly as promised.12 The problem is, as Williamson writes, “All + complex contracts are unavoidably incomplete + + [...] + + you have ever seen a house built according to architectural plans, then + you have a good idea of what Williamson means. There is no blueprint + that sufficiently details everything needed to convert drawings and + specifications into an actual house. No plan anticipates every problem + that might arise, and most do not come close. The builders’ skills are + a function of how they collaborate to invent the actions that fulfill + the intention of the drawings as they solve the unexpected but + inevitable complications that arise along the way. They work together + to construct a reality from the uncertainty of the plan + + [...] + + Were “contract utopia” to exist, Williamson says, it would best be + described as a “plan” that, like other “utopian modes,” requires “deep + commitment to collective purposes” and “personal subordination.” + Subordination to what? To the plan. Contract in this context of perfect + rationality is what Williamson describes as “a world of planning.” Such + planning was the basic institution of socialist economics, where the + “new man” was idealized as possessing “a high level of cognitive + competence” and therefore, it was espoused, could design highly + effective plans.14 Varian deftly swaps out socialism’s “new man” and + installs instead a market defined by surveillance capitalism’s economic + imperatives, expressed through a ubiquitous computational architecture, + the machine intelligence capabilities to which data are continuously + supplied, the analytics that discern patterns, and the algorithms that + convert them into rules. This is the essence of the uncontract, which + + [...] + + Uncertainty is not + + [...] + + chaos but rather the necessary habitat of the present tense. We choose + the fallibility of shared promises and problem solving over the certain + tyranny imposed by a dominant power or plan because this is the price + we pay for the freedom to will, which founds our right to the future + tense. In the absence of this freedom, the future collapses into an + infinite present of mere behavior, in which there can be no subjects + and no projects: only objects. In the future + + [...] + + Life inclines us to take action and to make commitments even when the + future is unknown. Anyone who has brought a child into the world or + + [...] + + the real world of human endeavor, there is no perfect information and + no perfect rationality + + [...] + + improve their approximation to guaranteed outcomes. Just as industrial + capitalism was driven to the continuous intensification of the means of + production, so surveillance capitalists are now locked in a cycle of + continuous intensification of the means of behavioral modification + + [...] + + Surveillance capitalists’ interests have shifted from using automated + machine processes to know about your behavior to using machine + processes to shape your behavior according to their interests. In other + words, this decade-and-a-half trajectory has taken us from automating + information flows about you to automating you. Given the conditions of + increasing ubiquity, it has become difficult if not impossible to + escape this audacious, implacable web + + [...] + + In order to reestablish our bearings, I have asked for a rebirth of + astonishment and outrage. Most of all, I have asked that we reject the + Faustian pact of participation for dispossession that requires our + submission to the means of behavioral modification built on the + foundation of the Google declarations. I am also mindful, though, that + when we ask How did they get away with it? there are many compelling + reasons to consider, no one of which stands alone + + [...] + + need laws that reject the fundamental legitimacy of surveillance + capitalism’s declarations and interrupt its most basic operations, + including the illegitimate rendition of human experience as behavioral + data; the use of behavioral surplus as free raw material; extreme + concentrations of the new means of production + + [...] + + shock and awe + + [...] + + withdrawal of agreement takes two broad forms, a distinction that will + be useful as we move into Part III. The first is what I call the + counter-declaration. These are defensive measures such as encryption + and other privacy tools, or arguments for “data ownership.” Such + measures may be effective in discrete situations + + [...] + + turn to the history of the Berlin Wall as an illustration of these two + forms of disagreement + + [...] + + industrial capitalism dangerously disrupted nature, what havoc might + surveillance capitalism wreak on human nature? The answer to this + question requires a return to imperatives + + [...] + + Industrial capitalism brought us to the brink of epic peril, but not as + a consequence of an evil lust for destruction or runaway technology. + Rather, this result was ineluctably driven by its own inner logic of + accumulation, with its imperatives of profit maximization, competition, + the relentless drive for labor productivity through the technological + elaboration of production, and growth funded by the continuous + reinvestment of + + [...] + + Similarly, the meaning of Polanyi’s prophecy for us now can be grasped + only through the lens of surveillance capitalism’s economic imperatives + as they frame its claim to human experience. If we are to rediscover + our sense of astonishment, then let it be here: if industrial + civilization flourished at the expense of nature and now threatens to + cost us the Earth, an information civilization shaped by surveillance + capitalism will thrive at the expense of human nature and threatens to + cost us our humanity + + [...] + + The idea from the start was that naming and taming are inextricable, + that fresh and careful naming can better equip us to intercept these + mechanisms of dispossession, reverse their action, produce urgently + needed friction, challenge the pathological division of learning, and + ultimately synthesize new forms of information capitalism that + genuinely meet our needs for effective life + + [...] + + the heart of Gentile’s political philosophy is the concept of the + “total.”3 The state was to be understood as an inclusive organic unity + that transcends individual + + [...] + + secret plans executed by secret police, the silent complicities and + hidden atrocities, the ceaseless transformation of who or what was up + or down, the intentional torsion of facts into anti-facts accompanied + by a perpetual deluge of propaganda, misinformation, euphemism, and + mendacity. The authoritative leader, or “egocrat,” to use the French + philosopher Claude Lefort’s term, displaces the rule of law and + “common” sense to become the quixotic judge of what is just or unjust, + truth or lie, at each moment.9 + + [...] + + Great Terror + + [...] + + murders of whole sectors of the Soviet population, from poets to + diplomats, generals to political loyalists. According to Soviet + historian Robert Conquest, that two-year period saw seven million + arrests, one million executions, two million deaths in labor camps, one + million people imprisoned, and another seven million people still in + camps by the end of 1938.11 Despite the immediacy of catastrophic + + [...] + + Until the rise of surveillance capitalism, the prospect of + instrumentarian power was relegated to a gauzy world of dream and + delusion. This new species of power follows the logic of Planck, Meyer, + and Skinner in the forfeit of freedom for knowledge, but those + scientists each failed to anticipate the actual terms of this + surrender. The knowledge that now displaces our freedom is proprietary. + The knowledge is theirs, but the lost freedom belongs solely to us. + With this origin story in + + [...] + + Instrumentarian power cultivates an unusual “way of knowing” that + combines the “formal indifference” of the neoliberal worldview with the + observational perspective of radical behaviorism + + [...] + + Forget the cliché that if it’s free, “You are the product.” You are not + the product; you are the abandoned carcass. The “product” derives from + the surplus that is ripped from your life.Big Other finally enables the + universal technology of behavior that, as Skinner, Stuart MacKay, Mark + Weiser, and Joe Paradiso each insisted, accomplishes its aims quietly + and persistently, using methods that intentionally bypass our + awareness, disappearing into the background of all things. Recall that + Alphabet/Google’s Eric Schmidt provoked uproar in 2015 when in response + to a question on the future of the web, he said, “The internet will + disappear.” What he really meant was that “The internet will disappear + into Big Other.” + + [...] + + We may confuse Big Other with the behaviorist god of the vortex, but + only because it effectively conceals the machinations of surveillance + capital that are the wizard behind the digital curtain + + [...] + + Under the regime of instrumentarian power, the mental agency and + self-possession of the right to the future tense are gradually + submerged beneath a new kind of automaticity: a lived experience of + stimulus-response-reinforcement aggregated as the comings and goings of + mere organisms. Our conformity is irrelevant to instrumentarianism’s + success. There is no need for mass submission to social norms, no loss + of self to the collective induced by terror and compulsion, no offers + of acceptance + + [...] + + Take one wrong step, one deviation from the path of seamless + frictionless predictability, and that same voice turns acid in an + instant as it instructs “the vehicular monitoring system not to allow + the car to be started.” + + [...] + + belonging as a reward for bending to the group. All of that is + superseded by a digital order that thrives within things and bodies, + transforming volition into reinforcement and action into conditioned + response. In this way instrumentarian power produces endlessly accruing + knowledge for surveillance capitalists and endlessly diminishing + freedom for us as it continuously renews surveillance capitalism’s + domination of the division of learning in society. False consciousness + is no longer produced by the hidden facts of class and their relation + to production but rather by the hidden facts of instrumentarian power’s + command over the division of learning in society as it usurps the + rights to answer the essential questions: Who knows? Who decides? Who + decides who decides? Power was once identified with the ownership of + the means of production, but it is now identified with ownership of the + means of behavioral modification that is Big Other. + + [...] + + The last stage of the laboring society, the society of jobholders, + demands of its members a sheer automatic functioning, as though + individual life had actually been submerged in the over-all life + process of the species and the only active decision still required of + the individual were to let go, so to speak, to abandon his + individuality, the still individually sensed pain and trouble of + living, and acquiesce in a dazed + + [...] + + tranquilized,” functional type of behavior. The trouble with modern + theories of behaviorism is not that they are wrong but that they could + become true, that they actually are the best possible conceptualization + of certain obvious trends in modern society. It is quite conceivable + that the modern age—which began with such an unprecedented and + promising outburst of human activity—may end in the deadliest, most + sterile passivity history has ever known.5 Is this to be our home + + [...] + + Now imagine, decades hence, another thinker meditating on the + “disturbing relevance” of instrumentarian power, observing that “the + true problems of our time cannot be understood, let alone solved, + without acknowledgement that instrumentarianism became this century’s + curse only because it so terrifyingly took care of its problems.” What + problems? I have + + [...] + + In the age of surveillance capitalism it is instrumentarian power that + fills the void, substituting machines for social relations, which + amounts to the substitution of certainty for society. In this imagined + collective life, freedom is forfeit to others’ knowledge, an + achievement that is only possible with the resources of the shadow text + + [...] + + private institutions of capital led the way in this ambitious + reformation of collective life and individual experience, but they + found necessary support from public institutions, especially as the + declaration of a “war on terror” legitimated every inclination to + enshrine machine-produced certainty as the ultimate solution to + societal uncertainty. These mutual affinities assured that + instrumentarian power would not be a stepchild but rather an equal + partner or even, with increasing regularity, the lord and master upon + whom the state depends in its quest for “total awareness.” That + instrumentarian power is regarded as the certain + + [...] + + White House briefing memo encouraged the companies to develop a + “radicalism algorithm” that would digest social media and other sources + of surplus to produce something comparable to a credit score, but aimed + at evaluating the “radicalness” of online content.14 The turn to + instrumentarian + + [...] + + Global Internet Forum to Counter Terrorism. The objective was to + tighten the net of instrumentarian power through + + [...] + + One startup, Geofeedia, specializes in detailed location tracking of + activists and protesters, such as Greenpeace members or union + organizers, and the computation of individualized + + [...] + + ACLU attorney countered that the government is using tech companies “to + build massive dossiers on people” based on nothing more than their + constitutionally protected speech.26 Another, more prominent + surveillance-as-a-service company, Palantir, once touted by Bloomberg + Businessweek as “the war on terror’s secret weapon,” was found to be in + a secret collaboration with the New Orleans Police Department to test + its “predictive policing” technology. Palantir’s software not only + identified gang members but also “traced people’s ties to other gang + members, outlined + + [...] + + thrust their scores into an inexorable downward spiral: “First your + score drops. Then your friends hear you are on the blacklist and, + fearful that their scores might be affected, quietly drop you as a + contact. The algorithm notices, and your + + [...] + + places less value on privacy than does Western culture and that most + Chinese have accommodated to the certain knowledge of online government + surveillance and censorship. The most common word for privacy, yinsi, + didn’t even appear in popular Chinese dictionaries until the + mid-1990s.42 Chinese citizens have accepted national ID cards with + biometric chips, “birth permits,” and now social credit rankings + because their society has been saturated with surveillance and + profiling for decades. For example, the “dang’an” is a wide-ranging + personal dossier compiled on hundreds of millions of urban residents + from childhood and maintained throughout life. This “Mao-era system for + recording the most intimate details of life” is updated by teachers, + Communist Party officials, and employers. Citizens have no rights to + see its contents, let alone contest them. The dossier is only one + feature of long-institutionalized + + [...] + + government urges the tech companies to train their algorithms for a + “radicalism” score. Indeed, the work of the shadow text is to evaluate, + categorize, and predict our behavior in millions of ways that we can + neither know nor combat—these are our digital dossiers. When it comes + to credit scoring, US and UK banks and + + [...] + + the Chinese context, the state will run the show and own it, not as a + market project but as a political one, a machine solution that shapes a + new society of automated behavior for guaranteed political and social + outcomes: certainty without terror. All the pipes from all the supply + chains will carry behavioral surplus to this new, complex means of + behavioral modification. The state will assume the role of the + behaviorist god, owning the shadow text and determining the schedule of + reinforcements and the behavioral routines that it will shape. Freedom + will be forfeit to knowledge, but it will be the state’s knowledge that + it exercises, not for the sake of revenue but for the sake of its own + perpetuation. + + [...] + + Joe” Stalin + + [...] + + The road from Shenzhen to an American or European airport also leads to + the Roomba vacuum cleaner mapping your living room and your breakfast + with Alexa + + [...] + + one direction lies the possibility of a synthetic declaration for a + third modernity based on the strengthening of democratic institutions + and the creative construction of a double movement for our time. On + this road we harness the digital to forms of information capitalism + that reunite supply and demand in ways that are both genuinely + productive of effective life and compatible with a flourishing + democratic social order. The first step down this road begins with + naming, establishing our bearings, reawakening our astonishment, and + sharing a sense of righteous indignity. + + [...] + + They aim to fashion a new society that emulates machine learning in + much the same way that industrial society was patterned on the + disciplines and methods of factory production. In their vision, + instrumentarian power replaces social trust, Big Other substitutes + certainty for social relations, and society as we know it shades into + obsolescence + + [...] + + Citing Abraham Lincoln, Facebook’s founder located his company’s + mission in the evolutionary time line of civilization, during which + humanity organized itself first in tribes, then cities, then nations. + The next phase of social evolution would be “global community,” and + Facebook was to lead the way, constructing the means and overseeing the + ends.14 Speaking at Facebook’s 2017 developers’ conference, Zuckerberg + linked his assertion of the company’s historic role in establishing a + “global community” to the standard myth of the modern utopia, assuring + his followers, “In the future, technology is going to… free us up to + spend more time on the things we all care about, like enjoying and + interacting with each other and expressing ourselves in new ways.… A + lot more of us are gonna do what today is considered the arts, and + that’s gonna form the basis of a lot of our communities + + [...] + + The “societal goal” articulated by the leading surveillance capitalists + fits snugly into the notion of limitless technological progress that + dominated utopian thought from the late eighteenth century through the + late nineteenth century, culminating with Marx. Indeed, surveillance + capitalists such as Nadella, Page, and Zuckerberg conform to five of + the six elements with which the great scholars of utopian thought, + Frank and Fritzie Manuel, define the classic profile of the most + ambitious modern utopianists: (1) a tendency toward highly focused + tunnel vision that simplifies the utopian challenge, (2) an earlier and + more trenchant grasp of a “new state of being” than other + contemporaries, (3) the obsessive pursuit and defense of an idée fixe, + (4) an unshakable belief in the inevitability of one’s ideas coming to + fruition, and (5) the drive for total reformation at the level of the + species and the entire world system + + [...] + + Often a utopian foresees the later evolution and consequences of + technological development already present in an embryonic state; he may + have antennae sensitive to the future. His gadgets, however, rarely go + beyond the mechanical potentialities of his age. Try as he may to + invent something wholly new, he cannot make a world out of nothing.”18 + In our time, however, surveillance capitalists can and do make such a + world—a genuinely historic deviation from the norm. Individually and + collectively, the + + [...] + + The only way to grasp the theory advanced in their applied utopistics + is to reverse engineer their operations and scrutinize their meaning, + as we have done throughout these chapters. + + [...] + + Microsoft’s instrumentarian society, the factories and workplaces are + like Skinner’s labs, and the machines replace his pigeons and rats. + These are the settings where the architecture and velocities of + instrumentarian power are readied for translation to society in a + digital-age iteration of Walden Two in which machine relations are the + model for social relations. Nadella’s construction site exemplifies the + grand confluence in which machines and humans are united as objects in + the cloud, all instrumented and orchestrated in accordance with the + “policies.” The magnificence of “policies” lies precisely in the fact + + [...] + + result is that “policies” are functionally equivalent to plans, as Big + Other directs human and machine action. It ensures that doors will be + locked or unlocked, car engines will shut down or come to life, the + jackhammer will scream “no” in suicidal self-sacrifice, the worker will + adhere to norms, the group will swarm to defeat anomalies. We will all + be safe as each organism hums in harmony with every other organism, + less a society than a population that ebbs and flows in perfect + frictionless confluence, shaped by the means of behavioral modification + that elude our awareness and thus can neither be mourned nor resisted. + + [...] + + the twentieth century the critical success factors of industrial + capitalism—efficiency, productivity, standardization, + interchangeability, the minute division of labor, discipline, + attention, scheduling, conformity, hierarchical administration, the + separation of knowing and doing, and so forth—were discovered and + crafted in the workplace and then transposed to society, where they + were institutionalized in schools, hospitals, family life, and + personality. As generations of scholars have documented, society became + more factory-like so that we might train and socialize the youngest + among us to fit the new requirements of a mass production order. + + [...] + + With conspicuously thin theory complemented by thick practice, the + patented device is designed to monitor user behavior in order to + preemptively detect “any deviation from normal or acceptable behavior + that is likely to affect the + + [...] + + Alternatively, the behavior could be assessed in relation to a “feature + distribution representing normal and/or acceptable behavior for an + average member of a population + + [...] + + user’s mental state + + [...] + + the circle widens as the patent specifications unfold. The scientists + note the utility of alerts for health care providers, insurance + companies, and law-enforcement personnel. Here is a new + surveillance-as-a-service opportunity geared to preempt whatever + behavior clients choose. Microsoft’s patent returns us to Planck, + + [...] + + In each case, corporate objectives define the “policies” toward which + confluent behavior harmoniously streams. + + [...] + + The machine hive—the confluent mind created by machine learning—is the + material means to the final elimination of the chaotic elements that + interfere with guaranteed outcomes + + [...] + + Instead of the typical assurances that machines can be designed to be + more like human beings and therefore less threatening, Schmidt and + Thrun argue just the opposite: it is necessary for people to become + more machine-like. + + [...] + + In this world the “correct” outcomes are known in advance and + guaranteed in action. The same ubiquitous instrumentation and + transparency that define the machine system must also define the social + system, which in the end is simply another way of describing the ground + truth of instrumentarian society. + + [...] + + this human hive, individual freedom is forfeit to collective knowledge + and action. Nonharmonious elements are preemptively targeted with high + doses of tuning, herding, and conditioning, including the full + seductive force of social persuasion and influence. We march in + certainty, like the smart machines. We learn to sacrifice our freedom + to collective knowledge imposed by others and for the sake of their + guaranteed outcomes. This is the signature of the third modernity + offered up by surveillance capital as its answer to our quest for + effective life together + + [...] + + Pentland is often referred to as the “godfather of wearables,” + especially Google Glass. In 1998 he predicted that wearables “can + extend one’s senses, improve memory, aid the wearer’s social life and + even help him or her stay calm and collected + + [...] + + Most noteworthy is that Pentland “completes” Skinner, fulfilling his + social vision with big data, ubiquitous digital instrumentation, + advanced mathematics, sweeping theory, numerous esteemed coauthors, + institutional legitimacy, lavish funding, and corporate friends in high + places without having attracted the worldwide backlash, moral + revulsion, and naked vitriol once heaped on Harvard’s outspoken + behaviorist. This fact alone suggests the depth of psychic numbing to + which we have succumbed and the loss of our collective bearings. + + [...] + + ’s like watching beavers from outer space, like Jane Goodall watching + gorillas. You observe from a distance.”7 (This is a slur on Goodall, of + course, whose seminal genius was her ability to understand the gorillas + she studied not as “other ones” but rather as “one of us.”) + + [...] + + The team saw that it would be possible to exploit the increasingly + “ubiquitous infrastructure” of mobile phones and combine those data + with new streams of information from their wearable behavioral + monitors. The result was a radical new solution that Pentland and Eagle + called “reality mining + + [...] + + Pentland argued that information gathered by his + sociometers—“unobtrusive wearable sensors” measuring communication, + voice tones, and body language—“could help managers understand who is + working with whom and infer the relationships between colleagues” and + “would be an efficient way to find people who might work well + together.”20 + + [...] + + people analytics + + [...] + + Pentland appeared in 2016 at a conference organized by Singularity + University, a Silicon Valley hub of instrumentarian ideology funded in + part by Larry Page. An interviewer tasked to write about Pentland + explains, “Though people are one of the most valuable assets in an + organization, many companies are still approaching management with a + 20th century mentality.… Pentland saw the factor that was always + messing things up was—the people.”29 Like Nadella, Pentland described + his aims as developing the social systems that would work along the + same lines as the machine systems, using behavioral data flows to judge + the “correctness” of action patterns and to intervene when it is + necessary to change “bad” action to “correct” action. “If people aren’t + interacting correctly and information isn’t spreading correctly,” + Pentland warns, “people + + [...] + + Pentland articulated his ambitions for the capabilities and objectives + of this new milieu in a series of papers, published primarily between + 2011 and 2014, but one remarkable 2011 essay of which he is the sole + author stands out: “Society’s Nervous System: Building Effective + Government, Energy, and Public Health Systems.”31 + + [...] + + The initial premise is reasonable enough: industrial-age technology + once revolutionized the world with reliable systems for water, food, + waste, energy, transportation, police, health care, education, and so + forth, but these systems are now hopelessly “old,” “centralized,” + “obsolete,” and “unsustainable.” New digital systems are required that + must be “integrated,” “holistic,” “responsive,” “dynamic,” and + “self-regulating”: “We need a radical rethinking of societies’ systems. + We must create a nervous system for humanity that maintains the + stability of our societies’ systems + + [...] + + What is missing… are the dynamic models of demand and reaction,” along + with an architecture that guarantees “safety, stability, and + efficiency.… The models required must describe human + + [...] + + Regarding incentives, Pentland outlines a principle of “social + efficiency,” which means that participation must provide value to the + individual but also to the system as a whole.37 For the sake of this + wholeness, it is believed, each of us will surrender to a totally + measured life of instrumentarian order + + [...] + + Skinner advocated, via Frazier, that the virtue of a “planned society” + is “to keep intelligence on the right track, for the good of society + rather than of the intelligent individual.… It does this by making sure + that the individual will not forget his + + [...] + + Pentland says that “continuous streams of data about human behavior” + mean that everything from traffic, to energy use, to disease, to street + crime will be accurately forecast, enabling a “world without war or + financial crashes, in which infectious disease is quickly detected and + stopped, in which energy, water, and other resources are no longer + wasted, and in which governments are part of the solution rather than + part of the problem.”48 This new “collective intelligence” operates to + serve the greater good as we learn to act “in a coordinated manner” + based on “social universals.” “Great leaps in health care, + transportation + + [...] + + The main barriers are privacy concerns and the fact that we don’t yet + have any consensus around the trade-offs between personal and social + values.” Like Skinner, he is emphatic that these attachments to a + bygone era of imperfect knowledge threaten to undermine the prospect of + a perfectly engineered future society: “We cannot ignore the public + goods that such a nervous system could provide.…”49 Pentland avoids the + question “Whose greater good?” How is the greater good determined when + surveillance capitalism owns the machines and the means of behavioral + modification? “Goodness” arrives already oriented toward the interests + of the owners of the means of behavioral modification and the clients + whose guaranteed outcomes they seek to achieve. The greater good is + someone’s, but it may not be ours + + [...] + + Capitalism and socialism are equally tainted by their shared emphasis + on economic growth, which breeds overconsumption and pollution. Skinner + is intrigued by the Chinese system but rejects it on the grounds of the + bloody revolution that any effort to convert Westerners would entail. + “Fortunately,” Skinner concludes in the preface to Walden Two, “there + is another possibility.” This option is Skinner’s version of a + behaviorist society that provides a way in which “political action is + to be avoided.” In Walden Two a “plan” replaces politics, overseen by a + “noncompetitive” group of “Planners” who eschew power in favor of the + dispassionate administration of the schedules of reinforcement aimed at + the greater good.52 Planners exercise unique control over society but + “only because that control is necessary for the proper functioning of + the community + + [...] + + Pentland worries that our political-economic constructs such as + “market” and “class” hail from an old, slow world of the eighteenth and + nineteenth centuries. The new, “light-speed hyperconnected world” + leaves no time for the kind of rational deliberation and face-to-face + negotiation and compromise that characterized the social milieu in + which such political concepts originated + + [...] + + There is no room for politics in this instrumentarian society because + politics means establishing and asserting our bearings. Individual + moral and political bearings are a source of friction that wastes + precious time and diverts behavior from confluence + + [...] + + Computation thus replaces the political life of the community as the + basis for governance. The depth and breadth of instrumentation make it + possible, Pentland says, to calculate idea flow, social network + structure, the degree of social influence between people, and even + “individual susceptibilities to new ideas.” Most important, + instrumentation makes it possible for those with the God view to modify + others’ behavior. The data provide a “reliable prediction of how + changing any of these variables will change the performance of all the + people + + [...] + + Frazier acknowledges that you cannot coerce people into doing the right + thing. The solution is far more subtle and sophisticated, based upon + scientifically calibrated schedules of reinforcement: “Instead you have + to set up certain behavioral processes which will lead the individual + to design his own ‘good’ conduct.… We call that sort of thing + ‘self-control.’ But don’t be misled, the control always rests in the + last analysis in the hands of society + + [...] + + Pentland’s idea is comparable: “The social physics approach to getting + everyone to cooperate” is “social network incentives,” his version of + “reinforcement.” With such incentives, he explains, “we focus on + changing the connections between people rather than focusing on getting + people individually to change their behavior.… We can leverage those + exchanges to generate social pressure for change.”60 Social media is + critical to establishing these tuning capabilities, Pentland believes, + because this is the environment in which social pressure can best be + controlled, directed, manipulated, and scaled + + [...] + + Pentland ignores the role of empathy in emulation because empathy is a + felt experience that is not subject to the observable metrics required + for computational governance. Instead, Pentland subscribes to the label + Homo imitans to convey that it is mimicry, not empathy, and certainly + not politics, which defines human existence + + [...] + + stream of ideas as a swarm or collective intelligence, flowing through + time, with all the humans in it learning from each other’s experiences + in order to jointly discover the patterns of preferences and habits of + action that best + + [...] + + What is being abolished is autonomous man—the inner man, the + homunculus, the possessing demon, the man defended by the literatures + of freedom and dignity + + [...] + + One important study of Bitcoin, the cryptocurrency that relies on + blockchain, suggests that such machine solutions both express and + contribute to the general erosion of the social fabric in ways that are + both consistent with instrumentarianism and further pave the way for + its success. Information scholars Primavera De Filippi and Benjamin + Loveluck conclude that contrary to popular belief, “Bitcoin is neither + anonymous nor privacy-friendly + + [...] + + We can begin by asking our children. Without knowing it, we sent the + least formed and most vulnerable among us to scout the hive and settle + its wilderness. Now their messages are filtering in from the frontier + + [...] + + Indeed, Facebook’s early advantage in this work arose in no small + measure from the simple fact that its founders and original designers + were themselves adolescents and emerging adults. They designed + practices for an imagined universe of adolescent users and college + + [...] + + contrary to Pentland’s belief that “class” divisions would disappear, + life in the hive produces new cleavages and forms of stratification: + not only tune or be tuned but also pressure or be pressured + + [...] + + Schüll learned that addictive players seek neither entertainment nor + the mythical jackpot of cash. Instead, they chase what Harvard Medical + School addiction researcher Howard Shaffer calls “the capacity of the + drug or gamble to shift subjective experience,” pursuing an + experiential state that Schüll calls the “machine zone,” a state of + self-forgetting in which one is carried along by an irresistible + momentum that feels like one is “played by the machine.”12 The machine + zone achieves a sense of complete immersion + + [...] + + Addiction by Design + + [...] + + Shaffer, the addiction researcher, has identified five elements that + characterize this state of compulsion: frequency of use, duration of + action, potency, route of administration, and player attributes + + [...] + + Perhaps the most difficult quality to capture is that in this period + that precedes the hard bargaining, an “inner” sense of “self” simply + does not yet exist. It is a time when “I” am whatever the “others” + think of me, and how “I” feel is a function of how the “others” treat + me. Instead of a stable sense of identity, there is only a chameleon + that reinvents itself depending upon the social mirror into which it is + drawn. In this condition, the “others” are not individuals but the + audience for whom I perform. Who “I” am depends upon the audience. This + state + + [...] + + Research shows that these big leaps in self-construction are stimulated + by experiences such as structured reflection, conflict, dissonance, + crisis, and failure. The people who help trigger this new inward + connection refuse to act as our mirrors. They reject fusion in favor of + genuine reciprocity + + [...] + + What are the consequences of the failure to win a healthy balance + between inner and outer, self and relationship? Clinical studies + identify specific patterns associated with this developmental + stagnation. Not surprisingly, these include an inability to tolerate + solitude, the feeling of being merged with others, an unstable sense of + self, and even an excessive need to control others as a way of keeping + the mirror close. Loss of the mirror is the felt equivalent of + extinction + + [...] + + The cultivation of inner resources is thus critical to the capacity for + intimacy and relationship, challenges that have become more + time-consuming with each new phase of the modern era. And while young + people are bound as ever to the enduring existential task of + self-making, our story suggests three critical ways in which this task + now converges with history and the unique conditions of existence in + our time + + [...] + + For example, Evil by Design author Chris Nodder, a user-experience + consultant, explains that evil design aims to exploit human weakness by + creating interfaces that “make users emotionally involved in doing + something that benefits the designer more than them.” He coaches his + readers in psychic numbing, urging them to accept the fact that such + practices have become the standard suggesting that consumers and + designers find ways to “turn them to your advantage + + [...] + + Facebook’s precocious mastery of “social proof”: “Much of our behavior + is determined by our impressions of what is the correct thing to do… + based on what we observe others doing.… This influence is known as + social proof + + [...] + + Most critical is that the more the need for the “others” is fed, the + less able one is to engage the work of self-construction. So + devastating is the failure to attain that positive equilibrium between + inner and outer life that Lapsley and Woodbury say it is “at the heart” + of most adult personality disorders + + [...] + + more that a user “liked,” the more that she informed Facebook about the + precise shape and composition of her “hand,” thus allowing the company + to continuously tighten the glove and increase the predictive value of + her signals. + + [...] + + On the demand side, Facebook’s “likes” were quickly coveted and craved, + morphing into a universal reward system or what one young app designer + called “our generation’s crack cocaine.” “Likes” became those variably + timed dopamine shots, driving users to double down on their bets “every + time they + + [...] + + News Feed is also the fulcrum of the social mirror. In the years + between revulsion and reverence, News Feed became Facebook’s most + intensely scrutinized object of data science and the subject of + extensive organizational innovation, all of it undertaken at a level of + sophistication and capital intensity that one might more naturally + associate with the drive to solve world hunger, cure cancer, or avert + climate destruction + + [...] + + laugh, cry, smile, click, like, share, or comment.”40 The glove + tightens around the hand with closed feedback loops enabled by the God + view, which favors posts from people with whom you have already + interacted, posts that have drawn high levels of engagement from + others, and posts that are like the ones with which you have already + engaged + + [...] + + According to the 302 most significant quantitative research studies on + the relationships between social media use and mental health (most of + them produced since 2013), the psychological process that most defines + the Facebook experience is what psychologists call “social + comparison.”45 It is usually considered a natural and virtually + + [...] + + One study found an increase in criminal larceny as television diffused + across society, awakening an awareness of and desire for consumer + goods. A related issue was that increased exposure to television + programs depicting affluence led to “the overestimation of others’ + wealth and more dissatisfaction with one’s own life + + [...] + + Both television and social media deprive us of real-life encounters, in + which we sense the other’s inwardness and share something of our own, + thus establishing some threads of communality. Unlike + + [...] + + consequence of the new density of social comparison triggers and their + negative feedback loops is a psychological condition known as FOMO + (“fear of missing out”). It + + [...] + + Profile inflation triggers more negative self-evaluation among + individuals as people compare themselves to others, which then leads to + more profile inflation, especially among larger networks that include + more “distant friends.” As one study concluded, “Expanding one’s social + network by adding a number of distant friends through Facebook may be + detrimental by stimulating negative emotions for users + + [...] + + This compulsive behavior is intended to produce relief in the form of + social reassurance, but it predictably breeds more anxiety and more + searching.52 Social comparison + + [...] + + When considered from the vantage point of the self-other balance, + positive social comparisons are just as pernicious as negative + comparisons. Both are substitutes for the “hard bargain” of carving out + a self that is capable of reciprocity rather than fusion + + [...] + + Facebook use does not promote well-being.… Individual social media + users might do well to curtail their use of social media and focus + instead on real-world relationships + + [...] + + This is the world of Pentland’s “social learning,” his theory of + “tuning” little more than the systematic manipulation of the rewards + and punishments of inclusion and exclusion. It succeeds through the + natural human inclination to avoid psychological pain + + [...] + + confluence,” in which harmonies are achieved at the expense of the + psychological integrity of participants + + [...] + + This synthetic hive is a devilish pact for a young person. In terms of + sheer everyday effectiveness—contact, logistics, transactions, + communications—turn away, and you are lost. And if you simply crave the + fusion juice that is proof of life at a certain age and stage—turn + away, and you are extinguished + + [...] + + Just as Pentland stipulated, these closed loops are imposed outside the + realm of politics and individual volition. They move in stealth, + crafting their effects at the level of automatic psychological + responses and tipping the self-other balance toward the + pseudo-harmonies of the hive mind. In this process, the inwardness that + is the necessary source of autonomous action and moral judgment suffers + and suffocates. These are the preparatory steps toward the death of + individuality that Pentland advocates. In fact, this + + [...] + + the eighteenth century’s political ideal of the individual as the + repository of inalienable dignity, rights, and obligations; (2) the + early twentieth century’s individualized + + [...] + + human being called into existence by history, embarking on Machado’s + road because she must, destined to create “a life of one’s own” in a + world of ever-intensifying social complexity and receding traditions; + and (3) the late twentieth century’s psychologically autonomous + individual whose inner resources and capacity for moral judgment rise + to the challenges of self-authorship that history demands and act as a + bulwark against the predations of power. The self-authorship toward + which young people strive + + [...] + + post-political societal processes that bind the hive rely on social + comparison and social pressure for their durability and predictive + certainty, eliminating the need for trust + + [...] + + the closing lines of Jean-Paul Sartre’s existential drama No Exit, the + character Garcin arrives at his famous realization, “Hell is other + people.” This was not intended as a statement of misanthropy but rather + a recognition that the self-other balance can never be adequately + struck as long as the “others” are constantly “watching.” Another + mid-century social psychologist, Erving Goffman, took up these themes + in his immortal The Presentation of Self in Everyday Life. Goffman + developed the idea of the “backstage” as the region in which the self + retreats from the performative demands of social life. + + [...] + + work as in life, “control of the backstage” allows individuals “to + buffer themselves from the deterministic demands that surround them.” + Backstage, the language is one of reciprocity, familiarity, intimacy, + humor. It offers the seclusion in which one can surrender to the + “uncomposed” face in sleep, defecation, sex, “whistling, chewing, + nibbling, belching, and flatulence.” Perhaps most of all, it is an + opportunity for “regression,” in which we don’t have to be “nice”: “The + surest sign of backstage solidarity is to feel that it is safe to lapse + into an asociable mood of sullen, silent irritability.” In the absence + of such respite where a “real” self can incubate and grow, Sartre’s + idea of hell begins to make sense.62 + + [...] + + Milgram identified three key themes in the subway experiment as he and + his students debriefed their experiences. The first was a new sense of + gravitas toward “the enormous inhibitory anxiety that ordinarily + prevents us from breaching social norms.” Second was that the reactions + of the “breacher” are not an expression of individual personality but + rather are “a compelled playing out of the logic of social relations + + [...] + + Embarrassment and the fear of violating apparently trivial norms often + lock us into intolerable predicaments.… These are not minor regulatory + forces in social life, but basic ones.” Finally, Milgram understood + that any confrontation of social norms crucially depends upon the + ability to escape. It was not an adolescent who boarded the subway that + day. Milgram was an erudite adult and an expert on human behavior, + especially the mechanisms entailed in obedience to authority, social + influence, and conformity. The subway was just an ordinary slice of + life, not a capital-intensive architecture of surveillance and behavior + modification, not a “personalized reward device.” Still, Milgram could + not fight off the anxiety of the situation. The only thing that made it + tolerable was the possibility of an exit. Unlike Milgram, we face an + intolerable situation + + [...] + + are meant to fuse with the system and play to extinction: not the + extinction of our funds but rather the extinction of our selves. + Extinction is a design feature formalized in the conditions of no exit. + The aim of the tuners is to contain us within “the power of immediate + circumstances” as we are compelled by the “logic of social relations” + in the hive to bow to social pressure exerted in calculated patterns + that exploit our natural empathy. Continuously tightening feedback + loops cut off the means of exit, creating impossible levels of anxiety + that further drive the loops toward confluence. What is to be killed + here is the inner impulse toward autonomy and the arduous, exciting + elaboration of the autonomous self as a source of moral judgment and + authority capable of asking for a subway seat or standing against rogue + power. + + [...] + + To exit means to enter the place where a self can be birthed and + nurtured. History has a name for that kind of place: sanctuary + + [...] + + We know that nothing guarantees safety and certainty in this world, but + we are comforted by the serenity of this home and its layered silences. + The days unfurl now + + [...] + + In the march of institutional interests intent on implementing Big + Other, the very first citadel to fall is the most ancient: the + principle of sanctuary. The sanctuary privilege has stood as an + antidote to power since the beginning of the human story. Even in + ancient societies where tyranny prevailed, the right of sanctuary stood + as a fail-safe. There was an exit from totalizing power, and that exit + was the entrance to a sanctuary in the form of a city, a community, or + a temple.4 By the time of the Greeks, sanctuaries were sacred sites + built across the ancient Greek world and consecrated to the purposes of + asylum and religious sacrifice. The Greek word asylon means + “unplunderable” and founds the notion + + [...] + + sanctuary as an inviolable space.5 The right of asylum survived into + the eighteenth century in many parts of Europe, attached to holy sites, + churches, and monasteries. The demise of the sanctuary privilege was + not a repudiation but rather a reflection of social evolution and the + firm establishment of the rule of law. One historian summarized this + transformation: “justice as sanctuary.”6 In the modern + + [...] + + empirical study makes the point. In “Psychological Functions of + Privacy,” Darhl Pedersen defines privacy as a “boundary control + process” that invokes the decision rights associated with “restricting + and seeking interaction + + [...] + + The same themes appear from the perspective of psychology. Those who + would eviscerate sanctuary are keen to take the offensive, putting us + off guard with the guilt-inducing question “What have you got to hide?” + But as we have seen, the crucial developmental challenges of the + self-other balance cannot be negotiated adequately without the sanctity + of “disconnected” time and space for the ripening of inward awareness + and the possibility of reflexivity: reflection on and by oneself. The + real psychological truth is this: If you’ve got nothing to hide, you + are nothing. + + [...] + + six categories of privacy behaviors: solitude, isolation + + [...] + + proper realm of inaccessibility or secrecy with respect to the world at + large as well as a recognition of the important social dimension of + such protected inner space.…”7 + + [...] + + contemplation, autonomy, rejuvenation, confiding, freedom, creativity, + recovery, catharsis, and concealment + + [...] + + anonymity, reserve, intimacy with friends, and intimacy with family + + [...] + + billions of sensors filled with personal data fall outside of Fourth + Amendment protections, a large-scale surveillance network will exist + without constitutional limits + + [...] + + This theme is illustrated in the odyssey of Belgian mathematician and + data protection activist Paul-Olivier Dehaye, who in December 2016 + initiated a request for his personal data collected through Facebook’s + Custom Audiences and tracking Pixel tools, which would reveal the web + pages where Facebook had tracked him. Dehaye probably knew more about + the rogue data operations of Cambridge Analytica than anyone in the + world, outside of its own staff and masterminds + + [...] + + of the right to contest “automatic decision making.” If the algorithms + are to be contestable in any meaningful way, it will require new + countervailing authority and power, including machine resources and + expertise to reach into the core disciplines of machine intelligence + and construct new approaches that are available for inspection, debate, + and combat. Indeed, one expert has already proposed the creation of a + government agency—an “FDA for algorithms + + [...] + + is already possible to see a new awakening to empowering collective + action, at least in the privacy domain. One example is None of Your + Business (NOYB), a nonprofit organization led by privacy activist Max + Schrems. After many years of legal contest, Schrems made history in + 2015 when his challenge to Facebook’s data-collection and + data-retention practices—which he asserted were in violation of EU + privacy law—led the Court of Justice of the European Union to + invalidate the Safe Harbor agreement that governed data transfers + + [...] + + the absence of synthetic declarations that secure the road to a human + future, the intolerability of glass life turns us toward a societal + arms race of counter-declarations in which we search for and embrace + increasingly complex ways to hide in our own lives, seeking respite + from lawless machines and their masters. We do this to satisfy our + enduring need for sanctuary and as an act of resistance with which to + reject the instrumentarian disciplines of the hive, its “extended + chilling effects,” and Big Other’s relentless greed. In the context of + government surveillance, the practices of “hiding” have been called + “privacy protests” and are well-known for drawing the suspicion of + law-enforcement agencies.33 Now, hiding is also invoked by Big Other + and its market masters, whose reach is far and deep as they install + themselves in our walls, our bodies, and on our streets, claiming our + faces, our feelings, and our fears of exclusion. I have suggested + + [...] + + Equally more poignant is the way in which a new generation of + activists, artists, and inventors feels itself called to create the art + and science of hiding.34 The intolerable conditions of glass life + compel these + + [...] + + Chicago artist Leo Selvaggio produces 3-D–printed resin prosthetic + masks to confound facial recognition. He calls his effort “an organized + artistic intervention.”35 Perhaps most poignant is the Backslash Tool + Kit: “a series of functional devices designed for protests and riots of + the future, + + [...] + + New Museum for Contemporary Art in Manhattan, and you pass a display of + its bestseller: table-top mirrors whose reflecting surface is covered + with the bright-orange message “Today’s Selfie Is Tomorrow’s Biometric + Profile.” This “Think Privacy Selfie Mirror” is a project of the young + Berlin-based artist Adam Harvey, whose work is aimed at the problem of + surveillance and + + [...] + + Trevor Paglen’s richly orchestrated performance art combines music, + photography, satellite imagery, and artificial intelligence to reveal + Big Other’s omnipresent knowing and doing. “It’s trying to look inside + the software that is running an AI… to look into the architectures of + different computer vision + + [...] + + greatest danger is that we come to feel at home in glass life or in the + prospect of hiding from it. Both alternatives rob us of the + life-sustaining inwardness + + [...] + + Glass life is intolerable, but so is fitting our faces with masks and + draping our bodies in digitally resistant fabrics to thwart the + ubiquitous lawless machines. Like every counter-declaration, hiding + risks becomes an adaptation when it should be a rallying point for + outrage. These conditions are unacceptable. Tunnels under this wall are + not enough. This wall must come down. + + [...] + + Surveillance capitalists are no different from other capitalists in + demanding freedom from any sort of constraint. They insist upon the + “freedom to” launch every novel practice while aggressively asserting + the necessity of their “freedom from” law and regulation. This classic + pattern reflects two bedrock assumptions about capitalism made by its + own theorists: The first is that markets are intrinsically unknowable. + The second is that the ignorance produced by this lack of knowledge + requires wide-ranging freedom of action for market actors + + [...] + + Adam Smith’s famous metaphor of the “invisible hand” drew on these + enduring realities of human life. Each individual, Smith reasoned, + employs his capital locally in pursuit of immediate comforts and + necessities. Each one attends to “his own security… his own gain… led + by an invisible hand to promote an end which was no part of his + intention.” That end is the efficient employ of capital in the broader + market: the wealth of nations. The individual actions that produce + efficient markets add up to a staggeringly complex pattern, a mystery + that no one person or entity could hope to know or understand, let + alone to direct + + [...] + + Adam Smith,” Hayek wrote, “was the first to perceive that we have + stumbled upon methods of ordering human economic cooperation that + exceed the limits of our knowledge and perception. His ‘invisible hand’ + had perhaps better have been described as an invisible or unsurveyable + pattern + + [...] + + As with Planck, Meyer, and Skinner, both Hayek and Smith unequivocally + link freedom and ignorance. In Hayek’s framing, the mystery of the + market is that a great many people can behave effectively while + remaining ignorant of the whole. Individuals not only can choose + freely, but they must freely choose their own pursuits because there is + no alternative, no source of total knowledge or conscious control to + guide them. “Human design” is impossible, Hayek says, because the + relevant information flows are “beyond the span of the control of any + one mind + + [...] + + However, Big Other and the steady application of instrumentarian power + challenge the classic quid pro quo of freedom for ignorance. When it + comes to surveillance capitalist operations, the “market” is no longer + invisible + + [...] + + Hayek chose the market over democracy, arguing that the market system + enabled not only the division of labor but also “the coordinated + utilization of resources based on equally divided knowledge.” This + system, he argued, is the only one compatible with freedom. Perhaps + some other kind of civilization might have been devised, he reckoned, + “like the ‘state’ of the termite ants,” but it would not be compatible + with human freedom.4 + + [...] + + More astonishing still is that surveillance capital derives from the + dispossession of human experience, operationalized in its unilateral + and pervasive programs of rendition: our lives are scraped and sold to + fund their freedom and our subjugation, their knowledge and our + ignorance about what they know + + [...] + + One conclusion of our investigations is that surveillance capitalism’s + command and control of the division of learning in society are the + signature feature that breaks with the old justifications of the + invisible hand and its entitlements. The combination of knowledge and + freedom works to accelerate the asymmetry of power between surveillance + capitalists and the societies in which they operate. This cycle will be + broken only when we acknowledge as citizens, as societies, and indeed + as a civilization that surveillance capitalists know too much to + qualify for freedom + + [...] + + The surveillance capitalists that operate at hyperscale or outsource to + hyperscale operations dramatically diminish any reliance on their + societies as sources of employees, and the few for whom they do + compete, as we have seen, are drawn from the most-rarefied strata of + data science + + [...] + + The absence of organic reciprocities with people as either sources of + consumers or employees is a matter of exceptional importance in light + of the historical relationship between market capitalism and democracy + + [...] + + ’s dependency on the “masses” and their contribution to the prosperity + necessitated by the new organization of production.23 The rise of + volume production and its wage-earning labor force established British + workers’ economic power and led to a growing appreciation of their + political legitimacy and power. This produced a new sense of + interdependence between ordinary people and elites. Acemoglu and + Robinson conclude that the “dynamic positive feedback” between + “inclusive economic institutions” (i.e., industrial firms defined by + employment reciprocities) and political institutions was critical to + Britain’s substantial and nonviolent democratic reforms. Inclusive + economic institutions, they argue, “level the playing field,” + especially when it comes to the fight for power, making it more + difficult for elites to “crush the masses” rather than accede to their + demands. Reciprocities in employment produced and sustained + reciprocities in politics + + [...] + + sharp contrast to the pragmatic concessions of Britain’s early + industrial capitalists, surveillance capitalists’ extreme structural + independence from people breeds exclusion rather than inclusion and + lays the foundation for the unique approach that we have called + “radical indifference + + [...] + + significant result of the systematic application of radical + indifference is that the public-facing “first text” is vulnerable to + corruption with content that would normally be perceived as repugnant: + lies, systematic disinformation, fraud, violence, hate speech, and so + on. As long as content contributes to “growth tactics,” Facebook + “wins.” This vulnerability can be an explosive problem on + + [...] + + guiding principles of radical indifference are reflected in the + operations of Facebook’s hidden low-wage labor force charged with + limiting the perversion of the first text. Nowhere is surveillance + capitalism’s outsized influence over the division of learning in + society more concretely displayed than in this outcast function of + “content moderation,” and nowhere is the nexus of economic imperatives + and + + [...] + + The larger point of the exercise is to find the point of equilibrium + between the ability to pull users and their surplus into + + [...] + + site and the risk of repelling them. This is a calculation of radical + indifference that has nothing to do with assessing the truthfulness of + content or respecting reciprocities with users.36 This tension helps to + explain why disinformation is not a priority. One investigative report + quotes a Facebook insider: “They absolutely have the tools to shut down + fake news. + + [...] + + radical indifference is a permanent invitation to the corruption of the + first text + + [...] + + It is obvious that the rogue forces of disinformation grasp this fact + more crisply than do Facebook’s or Google’s genuine users and customers + as those forces learn to exploit the blind eye of radical indifference + and escalate the perversion of learning in an open society + + [...] + + Surveillance capitalism’s antidemocratic and antiegalitarian juggernaut + is best described as a market-driven coup from above. It is not a coup + d’état in the classic sense but rather a coup de gens: an overthrow of + the people concealed as the technological Trojan horse that is Big + Other. On the strength of its annexation of human experience, this coup + achieves exclusive concentrations of knowledge and power that sustain + privileged influence over the division of learning in society: the + privatization of the central principle of social ordering in the + twenty-first century. Like the adelantados and their silent + incantations of the Requirimiento, surveillance capitalism operates in + the declarative form and imposes the social relations of a premodern + absolutist authority. It is a form of tyranny that feeds on people but + is not of the people. In a surreal paradox, this coup is celebrated as + “personalization,” although it defiles, ignores, overrides, and + displaces everything about you and me that is personal + + [...] + + Tyranny” is not a word that I choose lightly. Like the instrumentarian + hive, tyranny is the obliteration of politics. It is founded on its own + strain of radical indifference in which every person, except the + tyrant, is understood as an organism among organisms in an equivalency + of Other-Ones. Hannah Arendt observed that tyranny is a perversion of + egalitarianism because it treats all others as equally insignificant: + “The tyrant rules in accordance with his own will and interest… the + ruler who rules one against all, and the ‘all’ he oppresses are all + equal, namely equally powerless.” Arendt notes that classical political + theory regarded the tyrant as “out of mankind altogether… a wolf in + human shape.…”55Surveillance capitalism rules by instrumentarian power + through its materialization in Big Other, which, like the ancient + tyrant, exists out of mankind while paradoxically assuming human shape + + [...] + + Polanyi’s lens, we see that surveillance capitalism annexes human + experience to the market dynamic so that it is reborn as behavior: the + fourth “fictional commodity.” Polanyi’s first three fictional + commodities—land, labor, and money—were subjected to law. Although + these laws have been imperfect, the institutions of labor law, + environmental law, and banking law are regulatory frameworks intended + to defend society (and nature, life, and exchange) from the worst + excesses of raw capitalism’s destructive power. Surveillance + capitalism’s expropriation of human experience has faced no such + impediments. + + [...] + + will be Facebook, he says, that will address problems that are + civilizational in scale and scope, building “the long-term + infrastructure to bring humanity together” and keeping people safe with + “artificial intelligence” that quickly understands “what is happening + across our community.”56 Like Pentland, Zuckerberg imagines machine + intelligence that can “identify risks that nobody would have flagged at + all, including terrorists planning attacks using private channels, + people bullying someone too afraid to report it themselves, and other + issues both local and global.”57 When asked about his responsibility to + shareholders, Zuckerberg told CNN, “That’s why it helps to have control + of the company + + [...] + + industrial civilization aimed to exert control over nature for the sake + of human betterment. Machines were our means of extending and + overcoming the limits of the animal body so that we could accomplish + this aim of domination. Only later did we begin to fathom the + consequences + + [...] + + Years later, in his moving 1966 essay “Education after Auschwitz,” + social theorist Theodor Adorno attributed the success of German fascism + to the ways in which the quest for effective life had become an + overwhelming burden for too many people: “One must accept that fascism + and the terror it caused are connected with the fact that the old + established authorities… decayed and were toppled, while the people + psychologically were not yet ready for self-determination. They proved + to be unequal to the freedom that fell into their laps.”67 + + [...] + + We can now see that surveillance capitalism takes an even more + expansive turn toward domination than its neoliberal source code would + predict, claiming its right to freedom and knowledge, while setting its + sights on a collectivist vision that claims the totality of society. + Though still sounding like Hayek, and even Smith, its antidemocratic + collectivist ambitions reveal it as an insatiable child devouring its + aging fathers + + [...] + + The critical role of public opinion explains why even the most + destructive “ages” do not last forever. I echo here what Edison said a + century ago: that capitalism is “all wrong, out of gear.” The + instability of Edison’s day threatened every promise of industrial + civilization. It had to give way, he insisted, to a new synthesis that + reunited capitalism and its populations. Edison was prophetic. + Capitalism has survived the longue durée less because of any specific + capability and more because of its plasticity. It survives and thrives + by periodically renewing its roots in the social, finding new ways to + generate new wealth by meeting new needs. Its evolution has been marked + by a convergence of basic principles—private property, the profit + motive, and + + [...] + + It is not OK for every move, emotion, utterance, and desire to be + catalogued, manipulated, and then used to surreptitiously herd us + through the future tense for the sake of someone else’s profit. “These + things are brand-new,” I tell them. “They are unprecedented. You should + not take them for granted because they are not OK + + [...] + + Burnham’s cowardice is a cautionary tale. We are living in a moment + when surveillance capitalism and its instrumentarian power appear to be + invincible. Orwell’s courage demands that we refuse to cede the future + to illegitimate power. He asks us to break the spell of enthrallment, + helplessness, resignation, and numbing. We answer his call when we bend + ourselves toward friction, rejecting the smooth flows of coercive + confluence. Orwell’s courage sets us against the relentless tides of + dispossession that demean all human experience. Friction, courage, and + bearings are the resources we require to begin the shared work of + synthetic declarations that claim the digital future as a human place, + demand that digital capitalism operate as an inclusive force bound to + the people it must serve, and defend the division of learning in + society as a source of genuine democratic renewal -- cgit v1.2.3