1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
2244
2245
2246
2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316
2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
2389
2390
2391
2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
2404
2405
2406
2407
2408
2409
2410
2411
2412
2413
2414
2415
2416
2417
2418
2419
2420
2421
2422
2423
2424
2425
2426
2427
2428
2429
2430
2431
2432
2433
2434
2435
2436
2437
2438
2439
2440
2441
2442
2443
2444
2445
2446
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
2463
2464
2465
2466
2467
2468
2469
2470
2471
2472
2473
2474
2475
2476
2477
2478
2479
2480
2481
2482
2483
2484
2485
2486
2487
2488
2489
2490
2491
2492
2493
2494
2495
2496
2497
2498
2499
2500
2501
2502
2503
2504
2505
2506
2507
2508
2509
2510
2511
2512
2513
2514
2515
2516
2517
2518
2519
2520
2521
2522
2523
2524
2525
2526
2527
2528
2529
2530
2531
2532
2533
2534
2535
2536
2537
2538
2539
2540
2541
2542
2543
2544
2545
2546
2547
2548
2549
2550
2551
2552
2553
2554
2555
2556
2557
2558
2559
2560
2561
2562
2563
2564
2565
2566
2567
2568
2569
2570
2571
2572
2573
2574
2575
2576
2577
2578
2579
2580
2581
2582
2583
2584
2585
2586
2587
2588
2589
2590
2591
2592
2593
2594
2595
2596
2597
2598
2599
2600
2601
2602
2603
2604
2605
2606
2607
2608
2609
2610
2611
2612
2613
2614
2615
2616
2617
2618
2619
2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644
2645
2646
2647
2648
2649
2650
2651
2652
2653
2654
2655
2656
2657
2658
2659
2660
2661
2662
2663
2664
2665
2666
2667
2668
2669
2670
2671
2672
2673
2674
2675
2676
2677
2678
2679
2680
2681
2682
2683
2684
2685
2686
2687
2688
2689
2690
2691
2692
2693
2694
2695
2696
2697
2698
2699
2700
2701
2702
2703
2704
2705
2706
2707
2708
2709
2710
2711
2712
2713
2714
2715
2716
2717
2718
2719
2720
2721
2722
2723
2724
2725
2726
2727
2728
2729
2730
2731
2732
2733
2734
2735
2736
2737
2738
2739
2740
2741
2742
2743
2744
2745
2746
2747
2748
2749
2750
2751
2752
2753
2754
2755
2756
2757
2758
2759
2760
2761
2762
2763
2764
2765
2766
2767
2768
2769
2770
2771
2772
2773
2774
2775
2776
2777
2778
2779
2780
2781
2782
2783
2784
2785
2786
2787
2788
2789
2790
2791
2792
2793
2794
2795
2796
2797
2798
2799
2800
2801
2802
2803
2804
2805
2806
2807
2808
2809
2810
2811
2812
2813
2814
2815
2816
2817
2818
2819
2820
2821
2822
2823
2824
2825
2826
2827
2828
2829
2830
2831
2832
2833
2834
2835
2836
2837
2838
2839
2840
2841
2842
2843
2844
2845
2846
2847
2848
2849
2850
2851
2852
2853
2854
2855
2856
2857
2858
2859
2860
2861
2862
2863
2864
2865
2866
2867
2868
2869
2870
2871
2872
2873
2874
2875
2876
2877
2878
2879
2880
2881
2882
2883
2884
2885
2886
2887
2888
2889
2890
2891
2892
2893
2894
2895
2896
2897
2898
2899
2900
2901
2902
2903
2904
2905
2906
2907
2908
2909
2910
2911
2912
2913
2914
2915
2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
2927
2928
2929
2930
2931
2932
2933
2934
2935
2936
2937
2938
2939
2940
2941
2942
2943
2944
2945
2946
2947
2948
2949
2950
2951
2952
2953
2954
2955
2956
2957
2958
2959
2960
2961
2962
2963
2964
2965
2966
2967
2968
2969
2970
2971
2972
2973
2974
2975
2976
2977
2978
2979
2980
2981
2982
2983
2984
2985
2986
2987
2988
2989
2990
2991
2992
2993
2994
2995
2996
2997
2998
2999
3000
3001
3002
3003
3004
3005
3006
3007
3008
3009
3010
3011
3012
3013
3014
3015
3016
3017
3018
3019
3020
3021
3022
3023
3024
3025
3026
3027
3028
3029
3030
3031
3032
3033
3034
3035
3036
3037
3038
3039
3040
3041
3042
3043
3044
3045
3046
3047
3048
3049
3050
3051
3052
3053
3054
3055
3056
3057
3058
3059
3060
3061
3062
3063
3064
3065
3066
3067
3068
3069
3070
3071
3072
3073
3074
3075
3076
3077
3078
3079
3080
3081
3082
3083
3084
3085
3086
3087
3088
3089
3090
3091
3092
3093
3094
3095
3096
3097
3098
3099
3100
3101
3102
3103
3104
3105
3106
3107
3108
3109
3110
3111
3112
3113
3114
3115
3116
3117
3118
3119
3120
3121
3122
3123
3124
3125
3126
3127
3128
3129
3130
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
3145
3146
3147
3148
3149
3150
3151
3152
3153
3154
3155
3156
3157
3158
3159
3160
3161
3162
3163
3164
3165
3166
3167
3168
3169
3170
3171
3172
3173
3174
3175
3176
3177
3178
3179
3180
3181
3182
3183
3184
3185
3186
3187
3188
3189
3190
3191
3192
3193
3194
3195
3196
3197
3198
3199
3200
3201
3202
3203
3204
3205
3206
3207
3208
3209
3210
3211
3212
3213
3214
3215
3216
3217
3218
3219
3220
3221
3222
3223
3224
3225
3226
3227
3228
3229
3230
3231
3232
3233
3234
3235
3236
3237
3238
3239
3240
3241
3242
3243
3244
3245
3246
3247
3248
3249
3250
3251
3252
3253
3254
3255
3256
3257
3258
3259
3260
3261
3262
3263
3264
3265
3266
3267
3268
3269
3270
3271
3272
3273
3274
3275
3276
3277
3278
3279
3280
3281
3282
3283
3284
3285
3286
3287
3288
3289
3290
3291
3292
3293
3294
3295
3296
3297
3298
3299
3300
3301
3302
3303
3304
3305
3306
3307
3308
3309
3310
3311
3312
3313
3314
3315
3316
3317
3318
3319
3320
3321
3322
3323
3324
3325
3326
3327
3328
3329
3330
3331
3332
3333
3334
3335
3336
3337
3338
3339
3340
3341
3342
3343
3344
3345
3346
3347
3348
3349
3350
3351
3352
3353
3354
3355
3356
3357
3358
3359
3360
3361
3362
3363
3364
3365
3366
3367
3368
3369
3370
3371
3372
3373
3374
3375
3376
3377
3378
3379
3380
3381
3382
3383
3384
3385
3386
3387
3388
3389
3390
3391
3392
3393
3394
3395
3396
3397
3398
3399
3400
3401
3402
3403
3404
3405
3406
3407
3408
3409
3410
3411
3412
3413
3414
3415
3416
3417
3418
3419
3420
3421
3422
3423
3424
3425
3426
3427
3428
3429
3430
3431
3432
3433
3434
3435
3436
3437
3438
3439
3440
3441
3442
3443
3444
3445
3446
3447
3448
3449
3450
3451
3452
3453
3454
3455
3456
3457
3458
3459
3460
3461
3462
3463
3464
3465
3466
3467
3468
3469
3470
3471
3472
3473
3474
3475
3476
3477
3478
3479
3480
3481
3482
3483
3484
3485
3486
3487
3488
3489
3490
3491
3492
3493
3494
3495
3496
3497
3498
3499
3500
3501
3502
3503
3504
3505
3506
3507
3508
3509
3510
3511
|
[[!meta title="The Age of Surveillance Capitalism"]]
## Topics
* Second-modernity individuals.
* Dispossession Cycle: incursion, adaptation, habituation, redirection.
* Division of learning: who knows? Who decides? Who decides who decides?
* Shadow text.
## Excerpts
Just a moment ago, it still seemed reasonable to focus our concerns on the
challenges of an information workplace or an information society. Now the
oldest questions must be addressed to the widest possible frame, which is best
defined as “civilization” or, more specifically, information civilization. Will
this emerging civilization be a place that we can call home?
[...]
The sense of home slipping away provokes an unbearable yearning. The
Portuguese have a name for this feeling: saudade, a word said to capture the
homesickness and longing of separation from the homeland among emigrants across
the centuries. Now the disruptions of the twenty-first century have turned
these exquisite anxieties and longings of dislocation into a universal story
that engulfs each one of us.3
[...]
Although the saying tells us “If it’s free, then you are the product,” that
is also incorrect. We are the sources of surveillance capitalism’s crucial
surplus: the objects of a technologically advanced and increasingly inescapable
raw-material-extraction operation. Surveillance capitalism’s actual customers
are the enterprises that trade in its markets for future behavior.
[...]
Although the saying tells us “If it’s free, then you are the product,” that
is also incorrect. We are the sources of surveillance capitalism’s crucial
surplus: the objects of a technologically advanced and increasingly inescapable
raw-material-extraction operation. Surveillance capitalism’s actual customers
are the enterprises that trade in its markets for future behavior.
[...]
Surveillance capitalism operates through unprecedented asymmetries in
knowledge and the power that accrues to knowledge. Surveillance capitalists
know everything about us, whereas their operations are designed to be
unknowable to us. They accumulate vast domains of new knowledge from us, but
not for us. They predict our futures for the sake of others’ gain, not ours. As
long as surveillance capitalism and its behavioral futures markets are allowed
to thrive, ownership of the new means of behavioral modification eclipses
ownership of the means of production as the fountainhead of capitalist wealth
and power in the twenty-first century. These facts and their consequences for
our individual lives, our societies, our democracies, and our emerging
information civilization are examined in detail in the coming chapters. The
evidence and reasoning employed here suggest that surveillance capitalism is a
rogue force driven by novel economic imperatives that disregard social norms
and nullify the elemental rights associated with individual autonomy that are
essential to the very possibility of a democratic society. Just as industrial
civilization flourished at the expense of nature and now threatens to cost us
the Earth, an information civilization shaped by surveillance capitalism and
its new instrumentarian power will thrive at the expense of human nature and
will threaten to cost us our humanity. The industrial legacy of climate chaos
fills us with dismay, remorse, and fear. As surveillance capitalism becomes the
dominant form of information capitalism in our time, what fresh legacy of
damage and regret will be mourned by future generations?
[...]
For now, suffice to say that despite all the futuristic sophistication of
digital innovation, the message of the surveillance capitalist companies barely
differs from the themes once glorified in the motto of the 1933 Chicago World’s
Fair: “Science Finds—Industry Applies—Man Conforms.”
[...]
In order to challenge such claims of technological inevitability, we must
establish our bearings. We cannot evaluate the current trajectory of
information civilization without a clear appreciation that technology is not
and never can be a thing in itself, isolated from economics and society. This
means that technological inevitability does not exist. Technologies are always
economic means, not ends in themselves: in modern times, technology’s DNA comes
already patterned by what the sociologist Max Weber called the “economic
orientation.” Economic ends, Weber observed, are always intrinsic to
technology’s development and deployment. “Economic action” determines
objectives, whereas technology provides “appropriate means.” In Weber’s
framing, “The fact that what is called the technological development of modern
times has been so largely oriented economically to profit-making is one of the
fundamental facts of the history of technology.”15 In a modern capitalist
society, technology was, is, and always will be an expression of the economic
objectives that direct it into action. A worthwhile exercise would be to delete
the word “technology” from our vocabularies in order to see how quickly
capitalism’s objectives are exposed.
[...]
Surveillance capitalism employs many technologies, but it cannot be equated
with any technology. Its operations may employ platforms, but these operations
are not the same as platforms. It employs machine intelligence, but it cannot
be reduced to those machines. It produces and relies on algorithms, but it is
not the same as algorithms. Surveillance capitalism’s unique economic
imperatives are the puppet masters that hide behind the curtain orienting the
machines and summoning them to action. These imperatives, to indulge another
metaphor, are like the body’s soft tissues that cannot be seen in an X-ray but
do the real work of binding muscle and bone. We are not alone in falling prey
to the technology illusion. It is an enduring theme of social thought, as old
as the Trojan horse. Despite this, each generation stumbles into the quicksand
of forgetting that technology is an expression of other interests. In modern
times this means the interests of capital, and in our time it is surveillance
capital that commands the digital milieu and directs our trajectory toward the
future. Our aim in this book is to discern the laws of surveillance capitalism
that animate today’s exotic Trojan horses, returning us to age-old questions as
they bear down on our lives, our societies, and our civilization.
[...]
We have stood at this kind of precipice before. “We’ve stumbled along for a
while, trying to run a new civilization in old ways, but we’ve got to start to
make this world over.” It was 1912 when Thomas Edison laid out his vision for a
new industrial civilization in a letter to Henry Ford. Edison worried that
industrialism’s potential to serve the progress of humanity would be thwarted
by the stubborn power of the robber barons and the monopolist economics that
ruled their kingdoms. He decried the “wastefulness” and “cruelty” of US
capitalism: “Our production, our factory laws, our charities, our relations
between capital and labor, our distribution—all wrong, out of gear.” Both
Edison and Ford understood that the modern industrial civilization for which
they harbored such hope was careening toward a darkness marked by misery for
the many and prosperity for the few.
[...]
Most important for our conversation, Edison and Ford understood that the
moral life of industrial civilization would be shaped by the practices of
capitalism that rose to dominance in their time. They believed that America,
and eventually the world, would have to fashion a new, more rational capitalism
in order to avert a future of misery and conflict. Everything, as Edison
suggested, would have to be reinvented: new technologies, yes, but these would
have to reflect new ways of understanding and fulfilling people’s needs; a new
economic model that could turn those new practices into profit; and a new
social contract that could sustain it all. A new century had dawned, but the
evolution of capitalism, like the churning of civilizations, did not obey the
calendar or the clock. It was 1912, and still the nineteenth century refused to
relinquish its claim on the twentieth.
[...]
I describe the “collision” between the centuries-old historical processes
of individualization that shape our experience as self-determining individuals
and the harsh social habitat produced by a decades-old regime of neoliberal
market economics in which our sense of self-worth and needs for
self-determination are routinely thwarted. The pain and frustration of this
contradiction are the condition that sent us careening toward the internet for
sustenance and ultimately bent us to surveillance capitalism’s draconian quid
pro quo.
[...]
The youngest members of our societies already experience many of these
destructive dynamics in their attachment to social media, the first global
experiment in the human hive. I consider the implications of these developments
for a second elemental right: the right to sanctuary. The human need for a
space of inviolable refuge has persisted in civilized societies from ancient
times but is now under attack as surveillance capital creates a world of “no
exit” with profound implications for the human future at this new frontier of
power.
[...]
The Apple inversion depended on a few key elements. Digitalization made it
possible to rescue valued assets—in this case, songs—from the institutional
spaces in which they were trapped. The costly institutional procedures that
Sloan had described were eliminated in favor of a direct route to listeners. In
the case of the CD, for example, Apple bypassed the physical production of the
product along with its packaging, inventory, storage, marketing,
transportation, distribution, and physical retailing. The combination of the
iTunes platform and the iPod device made it possible for listeners to
continuously reconfigure their songs at will. No two iPods were the same, and
an iPod one week was different from the same iPod another week, as listeners
decided and re-decided the dynamic pattern. It was an excruciating development
for the music industry and its satellites—retailers, marketers, etc.—but it was
exactly what the new listeners wanted.
[...]
The implication is that new market forms are most productive when they are
shaped by an allegiance to the actual demands and mentalities of people. The
great sociologist Emile Durkheim made this point at the dawn of the twentieth
century, and his insight will be a touchstone for us throughout this book.
Observing the dramatic upheavals of industrialization in his time—factories,
specialization, the complex division of labor—Durkheim understood that although
economists could describe these developments, they could not grasp their cause.
He argued that these sweeping changes were “caused” by the changing needs of
people and that economists were (and remain) systematically blind to these
social facts: The division of labor appears to us otherwise than it does to
economists. For them, it essentially consists in greater production. For us,
this greater productivity is only a necessary consequence, a repercussion of
the phenomenon. If we specialize, it is not to produce more, but it is to
enable us to live in the new conditions of existence that have been made for
us.7
[...]
The sociologist identified the perennial human quest to live effectively in
our “conditions of existence” as the invisible causal power that summons the
division of labor, technologies, work organization, capitalism, and ultimately
civilization itself. Each is forged in the same crucible of human need that is
produced by what Durkheim called the always intensifying “violence of the
struggle” for effective life: “If work becomes more divided,” it is because the
“struggle for existence is more acute.”
[...]
What are these modernities and how do they matter to our story? The advent
of the individual as the locus of moral agency and choice initially occurred in
the West, where the conditions for this emergence first took hold. First let’s
establish that the concept of “individualization” should not be confused with
the neoliberal ideology of “individualism” that shifts all responsibility for
success or failure to a mythical, atomized, isolated individual, doomed to a
life of perpetual competition and disconnected from relationships, community,
and society. Neither does it refer to the psychological process of
“individuation” that is associated with the lifelong exploration of
self-development. Instead, individualization is a consequence of long-term
processes of modernization.10
[...]
The Spanish poet Antonio Machado captured the exhilaration and daring of
these first-modernity individuals in his famous song: “Traveler, there is no
road; the road is made as you go.” This is what “search” has meant: a journey
of exploration and self-creation, not an instant swipe to already composed
answers.
[...]
Socialization and adaptation were the materials of a psychology and
sociology that regarded the nuclear family as the “factory” for the “production
of personalities” ready-made for conformity to the social norms of mass
society.12 Those “factories” also produced a great deal of pain: the feminine
mystique, closeted homosexuals, church-going atheists, and back-alley
abortions. Eventually, though, they even produced people like you and me.
[...]
The free-market creed originated in Europe as a sweeping defense against
the threat of totalitarian and communist collectivist ideologies. It aimed to
revive acceptance of a self-regulating market as a natural force of such
complexity and perfection that it demanded radical freedom from all forms of
state oversight. Hayek explained the necessity of absolute individual and
collective submission to the exacting disciplines of the market as an
unknowable “extended order” that supersedes the legitimate political authority
vested in the state: “Modern economics explains how such an extended order…
constitutes an information-gathering process… that no central planning agency,
let alone any individual, could know as a whole, possess, or control.…”22 Hayek
and his ideological brethren insisted on a capitalism stripped down to its raw
core, unimpeded by any other force and impervious to any external authority.
Inequality of wealth and rights was accepted and even celebrated as a necessary
feature of a successful market system and as a force for progress.23 Hayek’s
ideology provided the intellectual superstructure and legitimation for a new
theory of the firm that became another crucial antecedent to the surveillance
capitalist corporation: its structure, moral content, and relationship to
society.
[...]
In 1976 Jensen and Meckling published a landmark article in which they
reinterpreted the manager as a sort of parasite feeding off the host of
ownership: unavoidable, perhaps, but nonetheless an obstacle to shareholder
wealth.
[...]
In the “crisis of democracy” zeitgeist, the neoliberal vision and its
reversion to market metrics was deeply attractive to politicians and policy
makers, both as the means to evade political ownership of tough economic
choices and because it promised to impose a new kind of order where disorder
was feared.25 The absolute authority of market forces would be enshrined as the
ultimate source of imperative control, displacing democratic contest and
deliberation with an ideology of atomized individuals sentenced to perpetual
competition for scarce resources. The disciplines of competitive markets
promised to quiet unruly individuals and even transform them back into subjects
too preoccupied with survival to complain.
[...]
In Capital in the Twenty-First Century, the French economist Thomas
Piketty integrated years of income data to derive a general law of
accumulation: the rate of return on capital tends to exceed the rate of
economic growth. This tendency, summarized as r > g, is a dynamic that
produces ever-more-extreme income divergence and with it a range of
antidemocratic social consequences long predicted as harbingers of an
eventual crisis of capitalism.
[...]
Many scholars have taken to describing these new conditions as
neofeudalism, marked by the consolidation of elite wealth and power far
beyond the control of ordinary people and the mechanisms of democratic
consent.55 Piketty calls it a return to “patrimonial capitalism,” a
reversion to a premodern society in which one’s life chances depend
upon inherited wealth rather than meritocratic achievement.56
[...]
We now have the tools to grasp the collision in all of its destructive
complexity: what is unbearable is that economic and social inequalities
have reverted to the preindustrial “feudal” pattern but that we, the
people, have not. We are not illiterate peasants, serfs, or slaves.
Whether “middle class” or “marginalized,” we share
[...]
Nevertheless, Occupy revealed a similar conflict between inequality’s
facts and inequality’s feelings, expressed in a creatively
individualized political culture that insisted on “direct democracy”
and “horizontal leadership.”60 Some analysts concluded that it was this
conflict that ultimately crippled the movement, with its “inner core”
of leaders unwilling to compromise their highly individualized approach
in favor of the strategies and tactics required for a durable mass
movement.61 However
[...]
This is the existential contradiction of the second modernity that
defines our conditions of existence: we want to exercise control over
our own lives, but everywhere that control is thwarted.
Individualization has sent each one of us on the prowl for the
resources we need to ensure effective life, but at each turn we are
forced to do battle with an economics and politics from whose vantage
point we are but ciphers. We live in the knowledge that our lives have
unique value, but we are treated as invisible
[...]
The deepest contradiction of our time, the social philosopher Zygmunt
Bauman wrote, is “the yawning gap between the right of self-assertion
and the capacity to control the social settings which render such
self-assertion feasible. It is from that abysmal gap that the most
poisonous effluvia contaminating the lives of contemporary individuals
emanate.”
[...]
When it comes to genuine economic mutation, there is always a tension
between the new features of the form and its mother ship. A combination
of old and new is reconfigured in an unprecedented pattern.
Occasionally, the elements of a mutation find the right environment in
which to be “selected” for propagation. This is when the new form
stands a chance of becoming fully institutionalized and establishes its
unique migratory path toward the future. But it’s even more likely that
potential mutations meet their fate in “transition failure,” drawn back
by the gravitational pull of established practices.63
[...]
Among the many violations of advocacy expectations, ubiquitous
“terms-of-service agreements” were among the most pernicious.67 Legal
experts call these “contracts of adhesion” because they impose
take-it-or-leave-it conditions on users that stick to them whether they
like it or not.
[...]
These “contracts” impose an unwinnable infinite regress upon the user
that law professor Nancy Kim describes as “sadistic.”
[...]
The digital milieu has been essential to these degradations. Kim points
out that paper documents once imposed natural restraints on contracting
behavior simply by virtue of their cost to produce, distribute, and
archive. Paper contracts require a physical signature, limiting the
burden a firm is likely to impose on a customer by requiring her to
read multiple pages of fine print. Digital terms, in contrast, are
“weightless.
[...]
Radin calls this “private eminent domain,” a unilateral seizure of
rights without consent. She
[...]
Once firms understood that the courts were disposed to validate their
click-wrap and browse-wrap agreements, there was nothing to stop them
from expanding the reach of these degraded contracts “to extract from
consumers additional benefits unrelated to the transaction.”73 This
coincided with the discovery of behavioral surplus that we examine in
Chapter 3, as
[...]
2008 two Carnegie Mellon professors calculated that a reasonable
reading of all the privacy policies that one encounters in a year would
require 76 full workdays at a national opportunity cost of $781
billion.75 The numbers are much higher today. Still, most
[...]
These developments reflect the simple truth that genuine economic
reformation takes time and that the internet world, its investors and
shareholders, were and are in a hurry. The credo of digital innovation
quickly turned to the language of disruption and an obsession with
speed, its campaigns conducted under the flag of “creative
destruction.” That famous, fateful phrase coined by evolutionary
economist Joseph Schumpeter was seized upon as a way to legitimate what
Silicon Valley euphemistically calls “permissionless innovation.”77
Destruction rhetoric promoted what I think of as a “boys and their
toys” theory of history, as if the winning hand in capitalism is about
blowing things up with new technologies. Schumpeter’s analysis was, in
fact, far more nuanced and complex than modern destruction rhetoric
suggests.
[...]
was as if a shark had been silently circling the depths all along, just
below the surface of the action, only to occasionally leap glistening
from the water in pursuit of a fresh bite of flesh
[...]
Over time, the shark revealed itself as a rapidly multiplying,
systemic, internally consistent new variant of information capitalism
that had set its sights on domination. An unprecedented formulation of
capitalism was elbowing its way into history: surveillance capitalism.
[...]
As we shall explore in detail throughout the coming chapters, thanks to
surveillance capitalism the resources for effective life that we seek
in the digital realm now come encumbered with a new breed of menace.
Under this new regime, the precise moment at which our needs are met is
also the precise moment at which our lives are plundered for behavioral
data, and all for the sake of others’ gain. The result is a perverse
amalgam of empowerment inextricably layered with diminishment. In the
absence of a decisive societal response that constrains or outlaws this
logic of accumulation, surveillance capitalism appears poised to become
the dominant form of capitalism in our time.
[...]
This left us wholly unprepared to defend ourselves from new companies
with imaginative names run by young geniuses that seemed able to
provide us with exactly what we yearn for at little or no cost. This
new regime’s most poignant harms, now and later, have been difficult to
grasp or theorize, blurred by extreme velocity and camouflaged by
expensive and illegible machine operations, secretive corporate
practices, masterful rhetorical misdirection, and purposeful cultural
misappropriation. On this road, terms whose meanings we take to be
positive or at least banal—“the open internet,” “interoperability,” and
“connectivity”—have been quietly harnessed to a market process in which
individuals are definitively cast as the means to others’ market ends.
[...]
The new harms we face entail challenges to the sanctity of the
individual, and chief among these challenges I count the elemental
rights that bear on individual sovereignty, including the right to the
future tense and the right to sanctuary. Each of these rights invokes
claims to individual agency and personal autonomy as essential
prerequisites to freedom of will and to the very concept of democratic
order.
[...]
The Spanish Data Protection Agency recognized that not all information
is worthy of immortality
[...]
As for the Spanish people, their Data Protection Agency, and the
European Court of Justice, the passage of time is likely to reveal
their achievements as a stirring early chapter in the longer story of
our fight for a third modern that is first and foremost a human future,
rooted in an inclusive democracy and committed to the individual’s
right to effective life. Their message is carefully inscribed for our
children to ponder: technological inevitability is as light as
democracy is heavy, as temporary as the scent of rose petals and the
taste of honey are enduring.
[...]
The point for us is that every successful vaccine begins with a close
understanding of the enemy disease. The mental models, vocabularies,
and tools distilled from past catastrophes obstruct progress. We smell
smoke and rush to close doors to rooms that are already fated to
vanish. The result is like hurling snowballs at a smooth marble wall
only to watch them slide down its facade, leaving nothing but a wet
smear: a fine paid here, an operational detour there, a new encryption
package there.
[...]
It is the habitat for progress “at the speed of dreams,” as one Google
engineer vividly describes it.100 My aim here is to slow down the
action in order to enlarge the space for such debate and unmask the
tendencies of these new creations as they amplify inequality, intensify
social hierarchy, exacerbate exclusion
[...]
Peter Drucker’s field studies for his seminal Concept of the
Corporation, the 1946 book that codified the practices of the twentieth
[...]
The closest thing we have to a Buck Weaver or James Couzens codifying
Google’s practices and objectives is the company’s longtime chief
economist, Hal Varian, who aids the cause of understanding with
scholarly articles that explore important themes. Varian has been
described as “the Adam Smith of the discipline of Googlenomics” and the
“godfather” of its advertising model.6 It is in
[...]
Nowadays there is a computer in the middle of virtually every
transaction… now that they are available these computers have several
other uses.”8 He then identifies four such new uses: “data extraction
and analysis,” “new contractual forms due to better monitoring,”
“personalization and customization,” and “continuous
experiments.”Varian’s discussions of
[...]
Data extraction and analysis,” Varian writes, “is what everyone is
talking about when they talk about big data.” “Data” are the raw
material necessary for surveillance capitalism’s novel manufacturing
processes. “Extraction” describes the social relations and material
infrastructure with which the firm asserts authority over those raw
materials to achieve economies of scale in its raw-material supply
operations. “Analysis” refers to the complex of highly specialized
computational systems that I will generally refer to in these chapters
as “machine intelligence.” I like this umbrella phrase because it
trains us on the forest rather than the trees, helping us decenter from
technology to its objectives. But in choosing this phrase I also follow
Google’s lead. The company describes itself “at the forefront of
innovation in machine intelligence,” a term in which it includes
machine learning as well as “classical” algorithmic production, along
with many computational operations that
[...]
Amit Patel, a young Stanford graduate student with a special interest
in “data mining,” is frequently credited with the groundbreaking
insight into the significance of Google’s accidental data caches. His
[...]
Google’s engineers soon grasped that the continuous flows of collateral
behavioral data could turn the search engine into a recursive learning
system that constantly improved search results and spurred product
innovations such as spell check,
[...]
was Google that recognized the gold dust in the detritus of its
interactions with its users and took the trouble to collect it up.…
Google exploits information that is a by-product of user interactions,
or data exhaust, which is automatically recycled
[...]
that early stage of Google’s development, the feedback loops involved
in improving its Search functions produced a balance of power: Search
needed people to learn from, and people needed Search
[...]
Hal Varian, who joined Google as its chief economist in 2002, would
note, “Every action a user performs is considered a signal to be
analyzed and fed back into the system.”16 The Page Rank algorithm,
named after its founder
[...]
The key point for us rests on a critical distinction. During this early
period, behavioral data were put to work entirely on the user’s behalf.
User data provided value at no cost, and that value was reinvested in
the user experience in the form of improved services: enhancements that
were also offered at no cost to users. Users provided the raw material
in the form of behavioral data, and those data were harvested to
improve speed, accuracy, and relevance and to help build ancillary
products such as translation. I call this the behavioral value
reinvestment cycle, in which all behavioral data are reinvested in the
improvement of the product or service (see Figure 1).
[...]
cycle was similarly oriented toward the individual as its subject, but
without a physical product to sell, it floated outside the marketplace,
an interaction with “users” rather than a market transaction with
customers.
[...]
Users are not paid for their labor, nor do they operate the means of
production, as we’ll discuss in more depth later in this chapter.
Finally, people often say that the user is the “product.” This is also
misleading, and it is a point that we will revisit more than once. For
now let’s say that users are not products, but rather we are the
sources of raw-material supply. As we shall see, surveillance
capitalism
[...]
impatient money
[...]
These behavioral data available for uses beyond service improvement
constituted a surplus, and it was on the strength of this behavioral
surplus that the young company would find its way to the “sustained and
exponential profits” that would be necessary for survival. Thanks to a
perceived
[...]
the New York Times reported, “The precision of the Carol Brady data was
eye-opening for some.” Even Brin was stunned by the clarity of Search’s
predictive power, revealing events and trends before they “hit the
radar” of traditional media. As he told the Times, “It was like trying
an electron microscope for the first time. It was like a
moment-by-moment barometer
[...]
Google maximizes the revenue it gets from that precious real estate by
giving its best position to the advertiser who is likely to pay Google
the most in total, based on the price per click multiplied by Google’s
estimate of the likelihood that someone will actually click on the
ad.”42 That pivotal multiplier was the result of Google’s advanced
computational capabilities trained on its most significant and secret
discovery: behavioral surplus
[...]
competitors, in which ads were targeted to keywords or content, were
unable to identify relevant ads “for a particular user.” Now the
inventors offered a scientific solution that exceeded the
most-ambitious dreams of any advertising executive
[...]
This new Google assures its actual customers that it will do whatever
it takes to transform the natural obscurity of human desire into
scientific fact. This Google is the superpower that establishes its own
values and pursues its own purposes above and beyond the social
contracts to which others are bound.
[...]
Google’s unique auction methods and capabilities earned a great deal of
attention, which distracted observers from reflecting on exactly what
was being auctioned: derivatives of behavioral surplus. Click-through
metrics institutionalized “customer” demand for these prediction
products and thus established the central importance of economies of
scale in surplus supply operations. Surplus capture would have to
become automatic and ubiquitous if the new logic was to succeed, as
measured by the successful trading of behavioral futures.
[...]
kind of commerce that depended upon online surveillance at scale.
Insiders referred to Google’s new science of behavioral
[...]
here was an unprecedented and lucrative brew: behavioral surplus, data
science, material infrastructure, computational power, algorithmic
systems, and automated platforms. This convergence produced
unprecedented “relevance” and billions of auctions. Click-through rates
skyrocketed. Work on AdWords and AdSense became just as important as
work on Search
[...]
their community effectively declared a “state of exception” in which it
was judged necessary to suspend the values and principles that had
guided Google’s founding and early practices.
[...]
Google’s inventions, their origins in emergency, and the 180-degree
turn from serving users to surveilling them. Most of all, he credited
the discovery of behavioral surplus as the game-changing asset that
turned Google into a fortune-telling giant, pinpointing Google’s
breakthrough transformation of the Overture model, when the young
company first applied its analytics of behavioral surplus to predict
the likelihood of a click:
[...]
Google loosed a new incarnation of capitalism upon the world, a
Pandora’s box whose contents we are only beginning
[...]
On the strength of Google’s inventions, discoveries, and strategies, it
became the mother ship and ideal type of a new economic logic based on
fortune-telling and selling—an ancient and eternally lucrative craft
that has fed on humanity’s confrontation with uncertainty from the
beginning of the human story.
[...]
The scientific and material complexity that supported the capture and
analysis of behavioral surplus also enabled the hiding strategy, an
invisibility cloak over the whole operation. “Managing search at our
scale is a very serious barrier to entry,” Schmidt warned would-be
competitors.79 To be sure, there are always sound business
[...]
public were told that Google’s magic derived from its exclusive
capabilities in unilateral surveillance of online behavior and its
methods specifically designed to override individual decision rights?
Google policies had to enforce secrecy in order to protect operations
that were designed to be undetectable because they took things from
users without asking and employed those unilaterally claimed resources
to work in the service of others’ purposes.
[...]
George Orwell once observed that euphemisms are used in politics, war,
and business as instruments that “make lies sound truthful and murder
respectable.”81 Google has been careful to camouflage the significance
of
[...]
Google discovered this necessary element of the new logic of
accumulation: it must assert the rights to take the information upon
which its success depends.
[...]
signing on with Facebook, the talented Sandberg became the “Typhoid
Mary” of surveillance capitalism as she led Facebook’s transformation
from a social networking site to an advertising behemoth. Sandberg
understood that Facebook’s social graph represented an awe-inspiring
source of behavioral surplus: the extractor’s equivalent of a
nineteenth-century prospector stumbling into a valley that sheltered
the largest diamond mine and the deepest gold mine ever to be
discovered. “We have better information than anyone else. We know
gender, age, location, and it’s real data as opposed to the stuff other
people infer,” Sandberg
[...]
Sandberg understood that through the artful manipulation of Facebook’s
culture of intimacy and sharing, it would be possible to use behavioral
surplus not only to satisfy demand but also to create demand. For
starters, that meant inserting advertisers into the fabric of
Facebook’s online culture, where they could
[...]
This new market form declares that serving the genuine needs of people
is less lucrative, and therefore less important, than selling
predictions of their behavior. Google discovered that we are less
valuable than others’ bets on our future behavior. This changed
everything.
[...]
VIII. Summarizing the Logic and Operations of Surveillance Capitalism
[...]
is obscene to suppose that this harm can be reduced to the obvious fact
that users receive no fee for the raw material they supply. That
critique is a feat of misdirection that would use a
[...]
remarkable questions here concern the facts that our lives are rendered
as behavioral data in the first place; that ignorance is a condition of
this ubiquitous rendition; that decision rights vanish before one even
knows that there is a decision to make; that there are consequences to
this diminishment of rights that we can neither see nor foretell; that
there is no exit, no voice, and no loyalty, only helplessness,
resignation, and psychic numbing; and that encryption is the only
positive action left to discuss when we sit around the dinner table and
casually ponder how to hide from the forces that hide from us.
[...]
Social theorist David Harvey builds on Arendt’s insight with his notion
of “accumulation by dispossession”: “What accumulation by dispossession
does is to release a set of assets… at very low (and in some instances
zero) cost. Overaccumulated capital can seize hold of such assets and
immediately turn them to profitable use.” He adds that entrepreneurs
who are determined to “join the system” and enjoy “the benefits of
capital accumulation” are often the ones who drive this
[...]
Even when knowledge derived from our behavior is fed back to us as a
quid pro quo for participation, as in the case of so-called
“personalization,” parallel secret operations pursue the conversion of
surplus into sales that point far beyond our interests. We have no
formal control because we are not essential to this market action. In
this future we are exiles from our own behavior, denied access to or
control over knowledge derived from its dispossession by others for
others. Knowledge
[...]
When asked about government regulation, Schmidt said that technology
moves so fast that governments really shouldn’t try to regulate it
because it will change too fast, and any problem will be solved by
technology. ‘We’ll move much faster than any government.’”26 Both Brin
and Page are even more candid in their contempt
[...]
Economic historians describe the dedication to lawlessness among the
Gilded Age “robber barons” for whom Herbert Spencer’s social Darwinism
played the same role that Hayek, Jensen, and even Ayn Rand play for
today’s digital barons. In the same way that surveillance capitalists
excuse their corporations’ unprecedented
[...]
There was no need for law, they argued, when one had the “law of
evolution,” the “laws of capital,” and the “laws of industrial
society.” John Rockefeller insisted that his outsized oil fortune was
the result of “the natural law of trade development.” Jay Gould, when
questioned by Congress on the need for federal regulation of railroad
rates, replied that rates were already regulated by “the laws of supply
and demand, production and consumption.”31 The millionaires mobilized
in 1896 to defeat the populist Democrat William Jennings Bryan, who had
vowed to tether economic policy to the political realm, including
regulating the railroads and protecting the people from “robbery
[...]
Surveillance After September 11, surveillance scholar David Lyon
[...]
After several decades in which data-protection officials, privacy
watchdogs, civil rights groups, and others have tried to mitigate
negative social effects of surveillance, we are witnessing a sharp tilt
toward more exclusionary and intrusive surveillance practices.”56 This
abrupt refocusing of governmental power and policy after the 9/11
attacks in New York City and Washington, DC
[...]
With the attacks of September 11, 2001, everything changed. The new
focus was overwhelmingly on security rather than privacy.”61 The
privacy provisions debated just months earlier vanished from the
conversation more or less overnight. In both the US Congress and across
the EU, legislation was
[...]
including Germany (a country that had been highly sensitized to
surveillance under the hammer of both Nazi and Stalinist
totalitarianism), the UK, and France.62 In the US the failure to
“connect the dots” on the terrorist attack was a source of shame and
dismay that overwhelmed other concerns. Policy guidelines shifted from
“need to know” to “need to share” as agencies were urged to tear down
walls and blend databases for comprehensive information and analysis.63
In a parallel development, privacy scholar Chris Jay
[...]
The elective affinity between public and private missions was evident
as early as 2002, when former NSA Chief Admiral John Poindexter
proposed his Total Information Awareness (TIA) program with a vision
that reads like an early guide to the foundational mechanisms of
behavioral surplus capture and analysis:
[...]
secret public-private intelligence collaborations that tend to be
“orchestrated around handshakes rather than legal formalities, such as
search warrants, and may be arranged this way to evade oversight and,
at times, to defy the law.”88 He observed that intelligence agencies
are irresistibly drawn to “and in some respects dependent upon” firms’
privately held data resources.89
[...]
former NSA Director Mike McConnell offered another glimpse into the
elective affinities between Google and the intelligence community.
Writing in the Washington Post, McConnell made clear that Google’s
surveillance-based operations in data capture, extraction, and analysis
were both taken for granted and coveted. Here the boundaries of private
and public melt in the intense heat of new threats and their
high-velocity demands that must be met in “milliseconds.” In
McConnell’s future there is one “seamless” surveillance empire in which
the requirements of self-preservation leave no opportunity for the
amenities of
[...]
Once again, history offers us no control groups
[...]
1) the demonstration of Google’s unique capabilities as a source of
competitive advantage in electoral politics; (2) a deliberate blurring
of public and private interests through relationships and aggressive
lobbying activities; (3) a revolving door of personnel who migrated
between Google and the Obama administration, united by elective
affinities during Google’s crucial growth years of 2009–2016; and (4)
Google’s intentional campaign of influence over academic work and the
larger cultural conversation so vital to policy
[...]
Obama used his proximity to Schmidt to cement his own identity as the
innovation candidate poised to disrupt business as usual in
Washington.98 Once elected, Schmidt joined the Transition Economic
Advisory Board and appeared next to Obama at
[...]
Political correspondent Jim Rutenberg’s New York Times account of the
data scientists’ seminal role in the 2012 Obama victory offers a vivid
picture of the capture and analysis of behavioral surplus as a
political methodology. The campaign knew “every single wavering voter
in the country that it needed to persuade to vote for Obama, by name,
address, race, sex, and income,” and it had figured out how to target
its television ads to these individuals. One breakthrough was the
“persuasion score” that identified
[...]
According to the Center for Media and Democracy’s investigatory
research report, “The Googlization of the Far Right,” the corporation’s
2012 list of grantees featured a new
[...]
Meanwhile, a list of Google Policy Fellows for 2014 included
individuals from a range of nonprofit organizations whom one would
expect to be leading the fight against that corporation’s
concentrations of information and power, including the Center for
Democracy and Technology, the Electronic Frontier Foundation, the
Future of Privacy Forum, the National Consumers League, the Citizen
Lab, and the Asociación por los Derechos Civiles.116 In July 2017 the
Wall Street Journal reported that
[...]
That summer, one of the New America Foundation’s most highly regarded
scholars and a specialist in digital monopolies, Barry Lynn, posted a
statement praising the EU’s historic decision to levy a $2.7 billion
fine on Google as the result of a multiyear antitrust investigation.
According to the New York Times and Lynn’s own account, New America’s
director bent to pressure from Schmidt, firing Lynn and his Open
Markets team of ten researchers. “Google is very aggressive in throwing
its money around Washington and Brussels, and then pulling strings,”
Lynn told the New York Times. “People are so afraid of Google now.” The
reporters cite Google
[...]
Google in the lead, surveillance capitalism vastly expanded the market
dynamic as it learned to expropriate human experience and translate it
into coveted behavioral predictions. Google and this larger
surveillance project have been birthed, sheltered, and nurtured to
success by the historical conditions of their era—second-modernity
needs, the neoliberal inheritance, and the realpolitik of surveillance
exceptionalism—as well as by their own purpose-built fortifications
designed to protect supply chain operations from scrutiny through
political and cultural capture.
[...]
the capture of behavioral surplus and the acquisition of decision
rights. Like a river running to the sea, if one route is blocked
[...]
increasingly ruthless cycle of kidnapping human experience, cornering
surplus supplies, and competing in new behavioral futures markets
[...]
The extraction imperative demands that everything be possessed. In this
new context, goods and services are merely surveillance-bound supply
routes. It’s not the car; it’s the behavioral data from driving the
car. It’s not the map; it’s the behavioral data from interacting with
the map. The ideal here is continuously expanding borders that
eventually describe the world and everything in it, all the time.
[...]
Traditionally, monopolies on goods and services disfigure markets by
unfairly eliminating competition in order to raise prices at will.
Under surveillance capitalism, however, many of the practices defined
as monopolistic actually function as means of cornering user-derived
raw-material supplies. There is no monetary price for the user to pay,
only an opportunity
[...]
The corporation unfairly impedes competitors in Search in order to
protect the dominance of its most important supply route, not primarily
to fix prices. These cornering operations are not abstractions,
[...]
products such as Android are valued more for supply than for sales.
Disconnect, Inc., founded in 2011 by two former Google engineers and a
privacy-rights attorney, developed
[...]
Google executive, noting that if other manufacturers switched to
Skyhook, it “would be awful for Google, because it will cut off our
ability to continue collecting data” for the company’s Wi-Fi location
database. Court documents from Skyhook’s eventual lawsuit against
Motorola (and Samsung) include an e-mail from Google’s senior vice
president of Mobile
[...]
Finally, extraordinary research from the French nonprofit Exodus
Privacy and the Yale Privacy Lab in 2017 documented the exponential
proliferation of tracking software. Exodus identified 44 trackers in
more than 300 apps for Google’s Android platform, some
[...]
For example, the ad tracker FidZup developed “communication between a
sonic emitter and a mobile phone.…” It can detect the presence of
mobile phones and therefore their owners by diffusing a tone, inaudible
to the human ear, inside a building: “Users installing
[...]
a pattern foreshadowed by the Google patent that we examined in Chapter
3 and that we shall see repeatedly in the coming chapters, the research
findings emphasize that the always-on tracking is impervious to the
Android “permissions system,” despite its promises of user control.17
[...]
Disconnect software was banned from Google Play’s vast catalog of
mobile apps, leading to Disconnect’s lawsuit against Google in 2015.
The startup’s complaint explains that “advertising companies including
Google use these invisible
[...]
dispossession operations reveal a predictable sequence of stages that
must be crafted and orchestrated in great detail in order to achieve
their ultimate destination as a system of facts through which surplus
extraction is normalized.The four stages of the cycle are incursion,
habituation, adaptation, and redirection. Taken together, these stages
constitute a “theory of change” that describes and predicts
dispossession as a political and cultural
[...]
with Google’s wider practice: it’s great to empower people, but not too
much, lest they notice the pilfering of their decision rights and try
to reclaim them. The firm wants to enable people to make
[...]
Google’s ideal society is a population of distant users, not a
citizenry. It idealizes people who are informed, but only in the ways
that the corporation chooses. It means for us to be docile, harmonious,
and, above all, grateful.
[...]
Within days, an independent analysis by German security experts proved
decisively that Street View’s cars were extracting unencrypted personal
information from homes. Google was forced to concede that it had
intercepted and stored “payload data,” personal information grabbed
from unencrypted Wi-Fi transmissions. As its apologetic blog post
noted,
[...]
Google’s “Spy-Fi” scandal filled headlines around the world. Many
believed that the Street View revelations would inflict irreparable
[...]
April 2012 FCC report is heart wrenching in its way, a melancholic
depiction of democracy’s vulnerability in the face-off with a wealthy,
determined, and audacious surveillance capitalist opponent. In November
2010 the FCC sent Google a letter of inquiry
[...]
The second point is that in retrospect, one sees that the very idea of
a single rogue engineer was designed and elaborated as a brilliant
piece of misdirection, a classic scapegoating ploy. It directed
attention away from the ambitious and controversial agenda of the
extraction imperative toward a different narrative of a single infected
cell excised from the flesh of an enormous but innocent organism. All
that was left was to excise the infected flesh and let the organism
declare itself cured of its privacy kleptomania. Then—a return to the
streets
[...]
This is to say that her job was a logical impossibility. That she may
have nevertheless taken it seriously is suggested by
[...]
Street View’s redirection and elaboration announced a critical shift in
the orientation and ambition of the surveillance program: it would no
longer be only about routes, but about routing
[...]
For now, suffice to say that Street View and the larger project of
Google Maps illustrate the new and even more ambitious goals toward
which this cycle of dispossession would soon point: the migration from
an online data source to a real-world monitor to an advisor to an
active shepherd—from knowledge to influence to control. Ultimately,
Street View’s elaborate data would become the basis for another complex
of spectacular Google incursions: the self-driving car and “Google
City,” which we learn more about in Chapter 7. Those programs aim to
take surplus capture to new levels while opening up substantial new
frontiers for the establishment of behavioral futures markets in the
real world of goods and services. It is important to understand that
each level of innovation builds on the one before and that all are
united in one aim, the extraction of behavioral surplus at scale.In
this progression, Google perceives an opportunity
[...]
Google discovered by chance or intention the source of every mapmaker’s
power
[...]
The first US rectangular land survey captured this language perfectly
in its slogan: “Order upon the Land.”72 The cartographer is the
instrument of power as the author of that order, reducing reality to
only two conditions: the map and oblivion. The cartographer’s truth
crystallizes the message that Google and all surveillance capitalists
must impress upon all humans: if you are not on our map, you do not
exist
[...]
Google has done incrementally and furtively what would plainly be
illegal
[...]
done all at once.”98
[...]
Nevertheless, the company made a canny decision not to disclose the
true extent of Cortana’s knowledge to its users. It wants to know
everything about you, but it does not want you to know how much it
knows or that its operations are entirely geared to continuously
learning more. Instead, the “bot” is programmed to ask for permission
and confirmation. The idea is to avoid spooking the public by
presenting Cortana’s intelligence as “progressive” rather than
“autonomous,” according to the project’s group program manager, who
noted that people do not want to be surprised by how much their phones
are starting to take over: “We made an explicit decision to be a little
less ‘magical’ and little
[...]
The Siren Song of Surveillance Revenues
[...]
PrecisionID
[...]
ID is then broadcast to every “unencrypted website a Verizon customer
visits from a mobile device. It allows third-party advertisers and
websites to assemble a deep, permanent profile of visitors’ web
browsing habits without their consent.”126 Alarmed by the threat of
fresh competition, Google, posing as a privacy advocate, launched a
campaign for a new internet protocol that would prevent “header
injections” such as Verizon’s PrecisionID.127 Privacy expert and
journalist Julia Angwin and
[...]
UIDH [unique identifier header], and expect that to be available soon.”
The New
[...]
capitalism gene therapy. As Verizon’s president of Operations told
investors, “For
[...]
The companies understood, and they persuaded Republican senators, that
the principle of consent would strike a serious blow to the
foundational mechanisms of the new capitalism: the legitimacy of
unilateral surplus dispossession, ownership rights to surplus, decision
rights over surplus, and the right to lawless space for the prosecution
of these activities.146 To this end the resolution also prevented
[...]
another trend, surveillance in the interest of behavioral surplus
capture and sale has become a service in its own right. Such companies
are often referred to as “software-as-a-service” or SaaS, but they are
more accurately termed “surveillance as a service,” or “SVaaS.” For
example, a new app-based approach to lending instantly establishes
creditworthiness based on detailed
[...]
You’re able to get in and really understand the daily life of these
customers,” explained the CEO of one lending company that analyzes
10,000 signals per customer.151 Such methods were originally
[...]
Surveillance capitalism was born digital, but as we shall see in
following chapters, it is no longer confined to born-digital companies.
This logic for translating investment into revenue is highly adaptive
and exceptionally lucrative as long as raw-material supplies are free
and law is kept at bay. The rapid migration to surveillance revenues
that is now underway recalls the late-twentieth-century shift from
revenues derived from goods and services to revenues derived from
mastering the speculative and shareholder-value-maximizing
[...]
Who knows? Who decides? Who decides who decides
[...]
According to the philosopher of language John Searle, a declaration is
a particular way of speaking and acting that establishes facts out of
thin air, creating a new reality where there was nothing. Here is how
it works: sometimes we speak to simply describe the world—“you have
brown eyes”—or to change it—“Shut the door.” A declaration combines
both, asserting a new reality by describing the world as if a desired
change were already true: “All humans are created equal.” “They are
yours to command.” As Searle writes, “We
[...]
Searle concludes, “All of institutional reality, and therefore… all of
human civilization is created by… declarations
[...]
Instead, Durkheim trained his sights on the social transformation
already gathering around him, observing that “specialization” was
gaining “influence” in politics, administration, the judiciary,
science, and the arts. He concluded that the division of labor was no
longer quarantined in the industrial workplace. Instead, it had burst
through those factory walls to becoming the critical organizing
principle of industrial society. This is also an example of Edison’s
insight: that the principles of capitalism initially aimed at
production eventually shape the wider social and moral milieu.
“Whatever opinion one has about the division of labor,” Durkheim wrote,
“everyone knows that it exists, and is more and more becoming one of
the fundamental bases of the social order.”17 Economic imperatives
predictably mandated the division of labor in production, but what was
the purpose of the division of labor in society? This was the question
that motivated Durkheim’s analysis, and his century
[...]
What would hold society together in the absence of the rules and
rituals of clan and kin? Durkheim’s answer was the division of labor.
People’s needs for a coherent new source of meaning and structure were
the cause, and the effect was an ordering
[...]
conclusions are still relevant for us now. He argued that the division
of labor accounts for the interdependencies and reciprocities that link
the many diverse members of a modern industrial society in a larger
prospect of solidarity. Reciprocities breed mutual need, engagement,
and respect, all of which imbue this new ordering principle with moral
force
[...]
Britain, university administrators are already talking about a “missing
generation” of data scientists. The huge salaries of the tech firms
have lured so many professionals that there is no one left to teach the
next generation of students. As one scholar described it, “The real
problem is these people are not dispersed through society. The
intellect and expertise is concentrated in a small number of
companies.”32
[...]
Under the regime of surveillance capitalism, the corporation’s
scientists are not recruited to solve world hunger or eliminate
carbon-based fuels. Instead, their genius is meant to storm the gates
of human experience, transforming it into data and translating it into
a new market colossus that creates wealth by predicting, influencing
[...]
under scrutiny, those long-awaited delivery trucks look more like
automated vehicles of invasion and conquest: more Mad Max than Red
Cross, more Black Sails than Carnival Cruise. The wizards behind their
steering wheels careen across every hill and hollow, learning how to
scrape and stockpile our behavior
[...]
Schmidt was, in fact, merely paraphrasing computer scientist Mark
Weiser’s seminal 1991 article, “The Computer for the 21st Century,”
which has framed Silicon Valley’s technology objectives for nearly
three decades. Weiser introduced what he called “ubiquitous computing”
with two legendary sentences: “The most profound technologies are those
that disappear. They weave themselves into the fabric of everyday life
until they are indistinguishable from it.” He described a new way of
thinking “that allows the computers themselves to vanish into the
background.… Machines that fit the human environment instead of forcing
humans to enter theirs will make using a computer as refreshing as
taking a walk in the woods.”2
[...]
new phase, supply operations were enlarged and intensified to
accommodate economies of scope and economies of action. What does this
entail? The shift toward economies of scope defines a new set of aims:
behavioral surplus must be vast, but it must also be varied. These
variations are developed along two dimensions. The first is the
extension of extraction operations from the virtual world into the
“real
[...]
call economies of action. In order to achieve these economies, machine
processes are configured to intervene in the state of play in the real
world among real people and things. These interventions are designed to
enhance certainty by doing things: they nudge, tune, herd, manipulate,
and modify behavior in specific directions by executing actions as
subtle as inserting a specific phrase into your Facebook news feed,
timing the appearance of a BUY button on your phone, or shutting down
your car engine when an insurance payment is late.
[...]
means of behavioral modification.” The aim of this undertaking is not
to impose behavioral norms, such as conformity or obedience, but rather
to produce behavior that reliably, definitively, and certainly leads to
desired commercial results. The research director of Gartner, the
well-respected business advisory and research firm, makes the point
unambiguously when he observes
[...]
shadow text.4As the prediction imperative gathers force, it gradually
becomes clear that extraction was the first phase of a
far-more-ambitious project. Economies of action mean that real-world
machine architectures must be able to know as well as to do. Extraction
is not enough; now it must be twinned with execution. The extraction
architecture is combined with a new execution architecture, through
which hidden economic objectives are imposed upon the vast and varied
field of behavior.5 Gradually, as surveillance capitalism’s imperatives
[...]
is an extraordinary statement because there can be no such guarantees
in the absence of the power to make it so. This wider complex that we
refer to as the “means of behavioral modification” is the expression of
this gathering power. The prospect of guaranteed outcomes alerts us to
the force of the prediction imperative, which demands that surveillance
capitalists make the future for the sake of predicting it. Under this
regime, ubiquitous computing is not just a knowing machine; it is an
actuating machine designed to produce more certainty about us and for
them.
[...]
Finally, I want to underscore that although it may be possible to
imagine something like the “internet of things” without surveillance
capitalism, it is impossible to imagine surveillance capitalism without
something like the “internet of things.” Every command arising from the
prediction imperative requires this pervasive real-world material
“knowing and doing” presence. The new apparatus is the material
expression of the prediction imperative, and it represents a new kind
of power animated by the economic compulsion toward certainty. Two
vectors converge in this fact: the early ideals of ubiquitous computing
and the economic imperatives of surveillance capitalism. This
convergence signals the metamorphosis of the digital infrastructure
from a thing that we have to a thing that has us.
[...]
was coaxed to life nearly sixty years ago under the warm equatorial sun
of the Galapagos Islands, when a giant tortoise stirred from her torpor
to swallow a succulent chunk of cactus into which a dedicated scientist
had wedged a small machine. It was a time when scientists reckoned with
the obstinacy of free-roaming animals and concluded that surveillance
was the necessary price of knowledge. Locking these creatures in a zoo
would only eliminate the very behavior that scientists wanted to study,
but how were they to be surveilled? The solutions once concocted by
scholars of elk herds, sea turtles, and geese have been refurbished by
surveillance capitalists and presented as an inevitable feature of
twenty-first-century life on Earth. All that has changed is that now we
are the animals
[...]
If you’re not in the system, you don’t exist
[...]
dark data
[...]
This means that surplus must be both plentiful (economies of scale) and
varied (economies of scope) in both range and depth.
[...]
gamification
[...]
all else fails, insurers are advised to induce a sense of inevitability
and helplessness in their customers. Deloitte counsels companies to
emphasize “the multitude of other technologies already in play to
monitor driving” and that “enhanced surveillance and/or geo-location
capabilities are part of the world we live in now, for better or
worse.”46
[...]
Behavioral data drawn from their experience are processed, and the
results flow in two directions. First, they return to the drivers,
executing procedures to interrupt and shape behavior in order to
enhance the certainty, and therefore profitability, of predictions
(economies of action). Second, prediction products that rank and sort
driver behavior flow into newly convened behavioral futures markets in
which third parties lay bets on what drivers will do now, soon, and
later: Will
[...]
uncontract
[...]
This is not the automation of society, as some might think, but rather
the replacement of society with machine action dictated by economic
imperatives. The uncontract is not
[...]
Despite its pervasiveness both in Silicon Valley and in the wider
culture of data scientists and technology developers, inevitabilism is
rarely discussed or critically evaluated. Paradiso
[...]
Paradiso imagines a society in which it falls to each individual to
protect herself from the omniscient ubiquitous sensate computational
systems of the new apparatus. Rather than paradise, it seems a recipe
for a new breed of madness. Yet this is precisely the world that is now
under construction around us, and this madness appears to be a happy
feature of the plan.
[...]
Cisco Kinetic gets the right data to the right applications at the
right time… while executing policies to enforce data ownership,
privacy, security and even data sovereignty laws.”73 But, as is so
often the case, the most audacious effort to transform the urban
commons into the surveillance capitalist’s equivalent of Paradiso’s
250-acre marsh comes from Google, which has introduced and legitimated
the concept of the “for-profit city.” Just as MacKay had counseled and
Weiser proselytized, the computer would be operational everywhere and
detectable nowhere, always beyond the edge of individual awareness. In
2015, shortly after Google reorganized
[...]
We fund it all… through a very novel advertising model.… We can
actually then target ads to people in proximity, and then obviously
over time track them through things like beacons and location services
as well as their browsing activity
[...]
As we are shorn of alternatives, we are forced to purchase products
that we can never own while our payments fund our own surveillance and
coercion. Adding insult to injury, data rendered by this wave of things
[...]
life pattern marketing” based on techniques derived from military
intelligence known as “patterns of life analysis
[...]
It allows you to tap into people’s compulsive nature by encouraging
impulse buys with the notifications you send out.… It also allows you
to gain insight on your current customers by reading what they’re
saying on Yelp and Facebook.…”24 Another mobile marketing firm
recommends
[...]
November 2017 Quartz investigative reporters discovered that since
early 2017, Android phones had been collecting location information by
triangulating the nearest cell towers, even when location services were
disabled, no apps were running, and no carrier SIM card was installed
in the phone. The information was used to manage Google’s “push”
notifications and messages sent to users on their Android phones,
enabling the company to track “whether an individual with
[...]
databases of ruin.”34
[...]
The company built an “employment index” for the national economy as
well as a “consumption index.” It also touted its ability to generate
quite-specific predictions such
[...]
The agencies’ well-meaning guidelines overlook the inconvenient truth
that transparency and privacy represent friction for surveillance
capitalists in much the same way that improving working conditions,
rejecting child labor, or shortening the working day represented
friction for the early industrial capitalists. It took targeted laws to
change working conditions back
[...]
The only real protection is when an app randomly but regularly
generates a new MAC address for your phone, but of the nine trackers,
only Apple’s performed this operation. The report also identifies a
general pattern of careless
[...]
other words, privacy policies are more aptly referred to as
surveillance policies, and that is what I suggest we call them. There
are many new territories of body rendition
[...]
Nobody reckoned with the fact that the prediction imperative makes
individual ignorance the preferred condition for rendition operations,
just as Arendt had observed and Mackay had prescribed for animals in
the wild. Original sin prefers the dark. The talks continued without
the advocates, and
[...]
1767 the political economist Nathaniel Forster worried that
“fashionable luxury” was spreading “like a contagion,” and he
complained of the “perpetual restless ambition in each of the inferior
ranks to raise themselves to the level of those immediately above
them.”4 Adam Smith wrote insightfully on this social process, noting
that upper-class luxuries can in time be recast as “necessaries.” This
occurs as “the established rules of decency” change to reflect new
customs introduced by elites, triggering lower-cost production methods
that transform what was once unattainable into newly affordable goods
and services.5 Ford’s Model T
[...]
Conversation” stands alone in its promise to dominate raw-material
supply, and the rewards to the One Voice would be astronomical. Casual
talk helps to blur the boundaries between “it”—the apparatus saturated
with commercial agents—and us. In conversation we imagine friendship.
The more we fancy the apparatus as our confidante, nanny, governess,
and support system—a disembodied, pervasive “Mrs. Doubtfire” for each
person—the more experience we allow it to render, and the richer its
supply operations grow. Communication is the first human joy, and a
conversational interface is prized for the frictionless ease in which a
mere utterance can trigger action
[...]
2018, Amazon had inked deals with home builders, installing its Dot
speakers directly into ceilings throughout the house as well as Echo
devices and
[...]
Mark Zuckerberg’s unilateral upending of established privacy norms in
2010, when he famously announced that Facebook users no longer have an
expectation of privacy. Zuckerberg had described the corporation’s
decision to unilaterally release users’ personal information,
declaring, “We decided that these would be the social norms now, and we
just went for it.”55 Despite
[...]
The personalization project descends deeper toward the ocean floor with
these new tools, where they lay claim to yet a new frontier of
rendition trained not only on your personality but also on your
emotional life. If this project of surplus from the depths is to
succeed, then your unconscious—where feelings form before there are
words to express them—must be recast as simply one more source of
raw-material supply for machine rendition and analysis, all of it for
the sake of more-perfect prediction. As a market research report on
affective computing explains, “Knowing the real-time emotional state
can help businesses to sell their product and thereby increase
revenue.”88 Emotion analytics products such as SEWA use
[...]
Conditioning” is a well-known approach to inducing behavior change,
primarily associated with the famous Harvard behaviorist B. F. Skinner.
He argued that behavior modification should mimic the evolutionary
process, in which naturally occurring behaviors are “selected” for
success by environmental conditions. Instead of the earlier, more
simplistic model of stimulus/response, associated with behaviorists
such as Watson and Pavlov, Skinner interpolated a third variable:
“reinforcement.” In his laboratory work with mice and pigeons, Skinner
learned how to observe a range of naturally occurring behaviors in the
experimental animal and then reinforce the specific action, or
“operant,” that he wanted the animal to reproduce. Ultimately, he
mastered intricate designs or “schedules” of reinforcement that could
reliably shape precise behavioral routines. Skinner called the
application
[...]
Conditioning at scale is essential to the new science of massively
engineered human behavior.” He believes that smartphones, wearable
devices, and the larger
[...]
Varian endorsed and celebrated this self-authorizing experimental role,
warning that all the data in the world “can only measure correlation,
not causality.”3 Data tell what happened but not why it happened
[...]
element in the construction of high-quality prediction products—i.e.,
those that approximate guaranteed outcomes—depends upon causal
knowledge. As Varian says, “If you really want to understand causality,
you have to run experiments. And if you run experiments continuously,
you can continuously improve your system.”4
[...]
Psychologists have found that the more a person can project himself or
herself into the feelings of another and take the other’s perspective,
the more likely he or she is to be influenced by subliminal cues,
including hypnosis. Empathy orients people toward other people. It
allows one to get absorbed in emotional experience and to resonate with
others’ experiences
[...]
Facebook’s persistence warns us again of the dispossession cycle’s
stubborn march. Facebook had publicly acknowledged and apologized for
its overt experimental incursions into behavior modification and
emotional manipulation, and it promised adaptations to curb or mitigate
these practices. Meanwhile, a new threshold of intimate life had been
breached. Facebook’s potential mastery of emotional manipulation became
discussable and even taken for granted as habituation set in. From
Princeton’s Fiske to critic Grimmelmann and supporter Meyer, the
experts believed that if Facebook’s activities were to be forced into a
new regulatory regime, the corporation would merely continue in secret
[...]
Individual awareness is the
[...]
The evasion of individual and group awareness was critical to
Facebook’s behavior-modification success, just as MacKay had
stipulated. The first paragraph of the research article on emotional
contagion celebrates this evasion: “Emotional states can be transferred
to others via emotional contagion, leading people to experience the
same emotions without their awareness.” Nor do the young adults of
Australia’s great cities suspect that the precise measure of their
fears and fantasies is exploited for commercial result at the hour and
moment of their greatest vulnerability.
[...]
enemy of telestimulation because it is the necessary condition for the
mobilization of cognitive and existential resources. There is no
autonomous judgment without awareness. Agreement and disagreement,
participation and withdrawal, resistance or collaboration: none of
these self-regulating choices can exist without awareness.
[...]
Indeed, some theorists have suggested that the primary purpose of self
awareness is to enable self-regulation.” Every threat to human autonomy
begins with an assault on awareness, “tearing down our capacity to
regulate our thoughts, emotions, and desires.”22
[...]
salience of self-awareness as a bulwark against self-regulatory failure
is also underscored in the work of two Cambridge University researchers
who developed a scale to measure a person’s “susceptibility to
persuasion.” They found that the single most important determinant of
one’s ability to resist persuasion is what they call “the ability to
premeditate.”23 This means that people who harness self-awareness to
think through the consequences of their actions are more disposed to
chart their own course and are significantly less vulnerable to
persuasion techniques. Self-awareness also figures in the
second-highest-ranking factor on their scale: commitment. People who
are consciously committed to a course of action or set of principles
[...]
We have seen already that democracy threatens surveillance revenues.
Facebook’s practices suggest an equally disturbing conclusion: human
consciousness itself is a threat to surveillance revenues, as awareness
endangers the larger project of behavior modification. Philosophers
recognize “self-regulation,” “self-determination,” and “autonomy” as
“freedom of will.” The word autonomy derives from the Greek and
literally means “regulation by the self.” It stands in contrast to
heteronomy, which means “regulation by others.” The competitive
necessity of economies of action means that surveillance capitalists
must use all means available to supplant autonomous action with
heteronomous action.
[...]
However, it would be dangerous to nurse the notion that today’s
surveillance capitalists simply represent more of the same. This
structural requirement of economies of action turns the means of
behavioral modification into an engine of growth. At no other time in
history have private corporations of unprecedented wealth and power
enjoyed the free exercise of economies of action supported by a
pervasive global architecture of ubiquitous computational knowledge and
control constructed and maintained by all the advanced scientific
know-how that money can buy.
[...]
Most research on games concludes that these structures can be effective
at motivating action, and researchers generally predict that games will
increasingly be used as the methodology of choice to change individual
behavior.34 In practice, this has meant that the power of games to
change behavior is shamelessly instrumentalized as gamification spreads
to thousands of situations in which a company merely wants to tune,
herd, and condition the behavior of its customers or employees toward
its own objectives
[...]
One analyst compiled a survey of more than ninety such “gamification
cases,” complete with return-on-investment statistics.35 Ian Bogost, a
professor of interactive computing at Georgia Tech and a digital
culture observer, insists that these systems should be called
“exploitationware” rather than games because their sole aim is behavior
manipulation and modification.36
[...]
The zeal for Pokémon Go gradually diminished, but the impact of Hanke’s
accomplishments is indelible. “We’ve only just scratched the surface,”
Hanke told a crowd of fans.46 The game had demonstrated that it was
possible to achieve economies of action on a global scale while
simultaneously directing specific individual actions toward precise
local market opportunities where high bidders enjoy an ever-closer
approximation of guaranteed outcomes. Niantic’s distinctive
accomplishment
[...]
TechCrunch noted the game’s “precise location tracking” and “ability to
perform audio fingerprinting” through its access to your camera and
microphone, concluding, “So it’s prudent to expect some of your
location data to end up in Google’s hands.”48 The Electronic Privacy
Information Center noted in a letter of complaint to the Federal
[...]
However, it does not acknowledge that its services operate on two
levels: game services for players and prediction services for Niantic’s
customers. The company concedes that it uses third-party services,
including Google’s, to “collect and interpret data,” but it is careful
to sidestep the aims of those analyses.51 The seven-page letter
mentions “sponsored
[...]
The genius of Pokémon Go was to transform the game you see into a
higher-order game of surveillance capitalism, a game about a game
[...]
the end we recognize that the probe was designed to explore the next
frontier: the means of behavioral modification. The game about the game
is, in fact, an experimental facsimile of surveillance capitalism’s
design for our future
[...]
Thus began a morbidly fascinating and often bizarre chapter in the
history of American spy craft.55 Much of the new work was conducted in
the context of the CIA’s highly classified MKUltra project, which was
tasked with “research and development of chemical, biological, and
radiological materials capable for employment in clandestine operations
to control human behavior
[...]
Another factor was the 1971 publication of B. F. Skinner’s incendiary
social meditation Beyond Freedom & Dignity. Skinner prescribed a future
based on behavioral control, rejecting the very idea of freedom (as
well as every tenet of a liberal society) and cast the notion of human
dignity as an accident of self-serving narcissism
[...]
First Amendment, the subcommittee argued, “must equally protect the
individual’s right to generate ideas,” and the right to privacy should
protect citizens from intrusions into their thoughts, behavior,
personality, and identity lest these concepts “become meaningless.” It
was in this context that Skinnerian behavioral engineering was singled
out for critical examination
[...]
Where is the hammer of democracy now, when the threat comes from your
phone, your digital assistant, your Facebook login? Who will stand for
freedom now, when Facebook threatens to retreat into the shadows if we
dare to be the friction that disrupts economies of action that have
been carefully, elaborately, and expensively constructed to exploit our
natural empathy, elude our awareness, and circumvent our prospects for
self-determination? If we fail to take notice now, how long before we
are numb to this incursion and to all the incursions? How long until we
notice nothing at all? How long before we forget who we were before
they owned us, bent over the old texts of self-determination in the dim
[...]
Now we know that surveillance capitalists’ ability to evade our
awareness is an essential condition for knowledge production. We are
excluded because we are friction that impedes
[...]
The commodification of behavior under the conditions of surveillance
capitalism pivots us toward a societal future in which an exclusive
division of learning is protected by secrecy, indecipherability, and
expertise. Even when knowledge derived from your behavior is fed back
to you in the first text as a quid pro quo for participation, the
parallel secret operations of the shadow text capture surplus for
crafting into prediction products destined for other marketplaces that
are about you rather than for you. These markets do not depend upon you
except first as a source of raw material from which surplus is derived,
and then as a target for guaranteed outcomes
[...]
this future we are exiles from our own behavior, denied access to or
control over knowledge derived from our experience. Knowledge,
authority, and power rest with surveillance capital, for which we are
merely “human natural resources
[...]
Centuries of debate have been levied on the notion of free will, but
too often their effect has been to silence our own declarations of
will, as if we are embarrassed to assert this most fundamental human
fact. I recognize my direct experience of freedom as an inviolate truth
that cannot be reduced to the behaviorists’ formulations of life as
necessarily accidental and random, shaped by external stimuli beyond my
knowledge or
[...]
influence and haunted by irrational and untrustworthy mental processes
that I can neither discern nor avoid
[...]
American philosopher John Searle, whose work on the “declaration” we
discussed in Chapter 6, comes to a similar conclusion in his
examination of “free will.” He points to the “causal gap” between the
reasons for our actions and their enactment. We may have good reasons
to do something, he observes, but that does not necessarily mean it
will be done. “The traditional name of this gap in philosophy is ‘the
freedom of the will.’” In response to the “sordid history” of this
concept, he reasons, “even if the gap is an illusion it is one we
cannot shake off.… The notion of making and keeping promises
presupposes the gap.… [It] requires consciousness and a sense of
freedom on the part of the promise-making and promise-keeping agent
[...]
Our freedom flourishes only as we steadily will ourselves to close the
gap between making promises and keeping them. Implicit in this action
is an assertion that through my will I can influence the future. It
does not imply total authority over the future, of course, only over my
piece
[...]
should an experience as elemental as this claim on the future tense be
cast as a human right? The short answer is that it is only necessary
now because it is imperiled. Searle argues that such elemental
“features of human life” rights are crystallized as formal human rights
only at that moment in history when they come under systematic threat.
So, for example, the ability to speak is elemental. The concept of
“freedom of speech” as a formal right emerged only when society evolved
to a degree of political complexity that the freedom to speak came
under threat. The philosopher observes that speech is not more
elemental to human life than breathing or being able to move one’s
body. No one has declared a “right to breathe” or a “right to bodily
movement” because these elemental rights have not come under attack and
therefore do not require formal protection. What counts as a basic
right, Searle argues, is both “historically contingent” and “pragmatic
[...]
Most simply put, there is no freedom without uncertainty; it is the
medium in which human will is expressed in promises. Of course, we do
not only make promises to ourselves; we also make promises to one
another. When we join our wills and our promises, we create the
possibility of collective action toward a shared future, linked in
determination to make our vision real in the world. This is the origin
of the institution we call “contract,” beginning with the ancient
Romans.6 Contracts originated as shared “islands of predictability”
intended to mitigate uncertainty for the human community, and they
still retain this meaning. “The simplest way
[...]
The uncontract aims instead for a condition that the economist Oliver
Williamson describes as “contract utopia”: a state of perfect
information known to perfectly rational people who always perform
exactly as promised.12 The problem is, as Williamson writes, “All
complex contracts are unavoidably incomplete
[...]
you have ever seen a house built according to architectural plans, then
you have a good idea of what Williamson means. There is no blueprint
that sufficiently details everything needed to convert drawings and
specifications into an actual house. No plan anticipates every problem
that might arise, and most do not come close. The builders’ skills are
a function of how they collaborate to invent the actions that fulfill
the intention of the drawings as they solve the unexpected but
inevitable complications that arise along the way. They work together
to construct a reality from the uncertainty of the plan
[...]
Were “contract utopia” to exist, Williamson says, it would best be
described as a “plan” that, like other “utopian modes,” requires “deep
commitment to collective purposes” and “personal subordination.”
Subordination to what? To the plan. Contract in this context of perfect
rationality is what Williamson describes as “a world of planning.” Such
planning was the basic institution of socialist economics, where the
“new man” was idealized as possessing “a high level of cognitive
competence” and therefore, it was espoused, could design highly
effective plans.14 Varian deftly swaps out socialism’s “new man” and
installs instead a market defined by surveillance capitalism’s economic
imperatives, expressed through a ubiquitous computational architecture,
the machine intelligence capabilities to which data are continuously
supplied, the analytics that discern patterns, and the algorithms that
convert them into rules. This is the essence of the uncontract, which
[...]
Uncertainty is not
[...]
chaos but rather the necessary habitat of the present tense. We choose
the fallibility of shared promises and problem solving over the certain
tyranny imposed by a dominant power or plan because this is the price
we pay for the freedom to will, which founds our right to the future
tense. In the absence of this freedom, the future collapses into an
infinite present of mere behavior, in which there can be no subjects
and no projects: only objects. In the future
[...]
Life inclines us to take action and to make commitments even when the
future is unknown. Anyone who has brought a child into the world or
[...]
the real world of human endeavor, there is no perfect information and
no perfect rationality
[...]
improve their approximation to guaranteed outcomes. Just as industrial
capitalism was driven to the continuous intensification of the means of
production, so surveillance capitalists are now locked in a cycle of
continuous intensification of the means of behavioral modification
[...]
Surveillance capitalists’ interests have shifted from using automated
machine processes to know about your behavior to using machine
processes to shape your behavior according to their interests. In other
words, this decade-and-a-half trajectory has taken us from automating
information flows about you to automating you. Given the conditions of
increasing ubiquity, it has become difficult if not impossible to
escape this audacious, implacable web
[...]
In order to reestablish our bearings, I have asked for a rebirth of
astonishment and outrage. Most of all, I have asked that we reject the
Faustian pact of participation for dispossession that requires our
submission to the means of behavioral modification built on the
foundation of the Google declarations. I am also mindful, though, that
when we ask How did they get away with it? there are many compelling
reasons to consider, no one of which stands alone
[...]
need laws that reject the fundamental legitimacy of surveillance
capitalism’s declarations and interrupt its most basic operations,
including the illegitimate rendition of human experience as behavioral
data; the use of behavioral surplus as free raw material; extreme
concentrations of the new means of production
[...]
shock and awe
[...]
withdrawal of agreement takes two broad forms, a distinction that will
be useful as we move into Part III. The first is what I call the
counter-declaration. These are defensive measures such as encryption
and other privacy tools, or arguments for “data ownership.” Such
measures may be effective in discrete situations
[...]
turn to the history of the Berlin Wall as an illustration of these two
forms of disagreement
[...]
industrial capitalism dangerously disrupted nature, what havoc might
surveillance capitalism wreak on human nature? The answer to this
question requires a return to imperatives
[...]
Industrial capitalism brought us to the brink of epic peril, but not as
a consequence of an evil lust for destruction or runaway technology.
Rather, this result was ineluctably driven by its own inner logic of
accumulation, with its imperatives of profit maximization, competition,
the relentless drive for labor productivity through the technological
elaboration of production, and growth funded by the continuous
reinvestment of
[...]
Similarly, the meaning of Polanyi’s prophecy for us now can be grasped
only through the lens of surveillance capitalism’s economic imperatives
as they frame its claim to human experience. If we are to rediscover
our sense of astonishment, then let it be here: if industrial
civilization flourished at the expense of nature and now threatens to
cost us the Earth, an information civilization shaped by surveillance
capitalism will thrive at the expense of human nature and threatens to
cost us our humanity
[...]
The idea from the start was that naming and taming are inextricable,
that fresh and careful naming can better equip us to intercept these
mechanisms of dispossession, reverse their action, produce urgently
needed friction, challenge the pathological division of learning, and
ultimately synthesize new forms of information capitalism that
genuinely meet our needs for effective life
[...]
the heart of Gentile’s political philosophy is the concept of the
“total.”3 The state was to be understood as an inclusive organic unity
that transcends individual
[...]
secret plans executed by secret police, the silent complicities and
hidden atrocities, the ceaseless transformation of who or what was up
or down, the intentional torsion of facts into anti-facts accompanied
by a perpetual deluge of propaganda, misinformation, euphemism, and
mendacity. The authoritative leader, or “egocrat,” to use the French
philosopher Claude Lefort’s term, displaces the rule of law and
“common” sense to become the quixotic judge of what is just or unjust,
truth or lie, at each moment.9
[...]
Great Terror
[...]
murders of whole sectors of the Soviet population, from poets to
diplomats, generals to political loyalists. According to Soviet
historian Robert Conquest, that two-year period saw seven million
arrests, one million executions, two million deaths in labor camps, one
million people imprisoned, and another seven million people still in
camps by the end of 1938.11 Despite the immediacy of catastrophic
[...]
Until the rise of surveillance capitalism, the prospect of
instrumentarian power was relegated to a gauzy world of dream and
delusion. This new species of power follows the logic of Planck, Meyer,
and Skinner in the forfeit of freedom for knowledge, but those
scientists each failed to anticipate the actual terms of this
surrender. The knowledge that now displaces our freedom is proprietary.
The knowledge is theirs, but the lost freedom belongs solely to us.
With this origin story in
[...]
Instrumentarian power cultivates an unusual “way of knowing” that
combines the “formal indifference” of the neoliberal worldview with the
observational perspective of radical behaviorism
[...]
Forget the cliché that if it’s free, “You are the product.” You are not
the product; you are the abandoned carcass. The “product” derives from
the surplus that is ripped from your life.Big Other finally enables the
universal technology of behavior that, as Skinner, Stuart MacKay, Mark
Weiser, and Joe Paradiso each insisted, accomplishes its aims quietly
and persistently, using methods that intentionally bypass our
awareness, disappearing into the background of all things. Recall that
Alphabet/Google’s Eric Schmidt provoked uproar in 2015 when in response
to a question on the future of the web, he said, “The internet will
disappear.” What he really meant was that “The internet will disappear
into Big Other.”
[...]
We may confuse Big Other with the behaviorist god of the vortex, but
only because it effectively conceals the machinations of surveillance
capital that are the wizard behind the digital curtain
[...]
Under the regime of instrumentarian power, the mental agency and
self-possession of the right to the future tense are gradually
submerged beneath a new kind of automaticity: a lived experience of
stimulus-response-reinforcement aggregated as the comings and goings of
mere organisms. Our conformity is irrelevant to instrumentarianism’s
success. There is no need for mass submission to social norms, no loss
of self to the collective induced by terror and compulsion, no offers
of acceptance
[...]
Take one wrong step, one deviation from the path of seamless
frictionless predictability, and that same voice turns acid in an
instant as it instructs “the vehicular monitoring system not to allow
the car to be started.”
[...]
belonging as a reward for bending to the group. All of that is
superseded by a digital order that thrives within things and bodies,
transforming volition into reinforcement and action into conditioned
response. In this way instrumentarian power produces endlessly accruing
knowledge for surveillance capitalists and endlessly diminishing
freedom for us as it continuously renews surveillance capitalism’s
domination of the division of learning in society. False consciousness
is no longer produced by the hidden facts of class and their relation
to production but rather by the hidden facts of instrumentarian power’s
command over the division of learning in society as it usurps the
rights to answer the essential questions: Who knows? Who decides? Who
decides who decides? Power was once identified with the ownership of
the means of production, but it is now identified with ownership of the
means of behavioral modification that is Big Other.
[...]
The last stage of the laboring society, the society of jobholders,
demands of its members a sheer automatic functioning, as though
individual life had actually been submerged in the over-all life
process of the species and the only active decision still required of
the individual were to let go, so to speak, to abandon his
individuality, the still individually sensed pain and trouble of
living, and acquiesce in a dazed
[...]
tranquilized,” functional type of behavior. The trouble with modern
theories of behaviorism is not that they are wrong but that they could
become true, that they actually are the best possible conceptualization
of certain obvious trends in modern society. It is quite conceivable
that the modern age—which began with such an unprecedented and
promising outburst of human activity—may end in the deadliest, most
sterile passivity history has ever known.5 Is this to be our home
[...]
Now imagine, decades hence, another thinker meditating on the
“disturbing relevance” of instrumentarian power, observing that “the
true problems of our time cannot be understood, let alone solved,
without acknowledgement that instrumentarianism became this century’s
curse only because it so terrifyingly took care of its problems.” What
problems? I have
[...]
In the age of surveillance capitalism it is instrumentarian power that
fills the void, substituting machines for social relations, which
amounts to the substitution of certainty for society. In this imagined
collective life, freedom is forfeit to others’ knowledge, an
achievement that is only possible with the resources of the shadow text
[...]
private institutions of capital led the way in this ambitious
reformation of collective life and individual experience, but they
found necessary support from public institutions, especially as the
declaration of a “war on terror” legitimated every inclination to
enshrine machine-produced certainty as the ultimate solution to
societal uncertainty. These mutual affinities assured that
instrumentarian power would not be a stepchild but rather an equal
partner or even, with increasing regularity, the lord and master upon
whom the state depends in its quest for “total awareness.” That
instrumentarian power is regarded as the certain
[...]
White House briefing memo encouraged the companies to develop a
“radicalism algorithm” that would digest social media and other sources
of surplus to produce something comparable to a credit score, but aimed
at evaluating the “radicalness” of online content.14 The turn to
instrumentarian
[...]
Global Internet Forum to Counter Terrorism. The objective was to
tighten the net of instrumentarian power through
[...]
One startup, Geofeedia, specializes in detailed location tracking of
activists and protesters, such as Greenpeace members or union
organizers, and the computation of individualized
[...]
ACLU attorney countered that the government is using tech companies “to
build massive dossiers on people” based on nothing more than their
constitutionally protected speech.26 Another, more prominent
surveillance-as-a-service company, Palantir, once touted by Bloomberg
Businessweek as “the war on terror’s secret weapon,” was found to be in
a secret collaboration with the New Orleans Police Department to test
its “predictive policing” technology. Palantir’s software not only
identified gang members but also “traced people’s ties to other gang
members, outlined
[...]
thrust their scores into an inexorable downward spiral: “First your
score drops. Then your friends hear you are on the blacklist and,
fearful that their scores might be affected, quietly drop you as a
contact. The algorithm notices, and your
[...]
places less value on privacy than does Western culture and that most
Chinese have accommodated to the certain knowledge of online government
surveillance and censorship. The most common word for privacy, yinsi,
didn’t even appear in popular Chinese dictionaries until the
mid-1990s.42 Chinese citizens have accepted national ID cards with
biometric chips, “birth permits,” and now social credit rankings
because their society has been saturated with surveillance and
profiling for decades. For example, the “dang’an” is a wide-ranging
personal dossier compiled on hundreds of millions of urban residents
from childhood and maintained throughout life. This “Mao-era system for
recording the most intimate details of life” is updated by teachers,
Communist Party officials, and employers. Citizens have no rights to
see its contents, let alone contest them. The dossier is only one
feature of long-institutionalized
[...]
government urges the tech companies to train their algorithms for a
“radicalism” score. Indeed, the work of the shadow text is to evaluate,
categorize, and predict our behavior in millions of ways that we can
neither know nor combat—these are our digital dossiers. When it comes
to credit scoring, US and UK banks and
[...]
the Chinese context, the state will run the show and own it, not as a
market project but as a political one, a machine solution that shapes a
new society of automated behavior for guaranteed political and social
outcomes: certainty without terror. All the pipes from all the supply
chains will carry behavioral surplus to this new, complex means of
behavioral modification. The state will assume the role of the
behaviorist god, owning the shadow text and determining the schedule of
reinforcements and the behavioral routines that it will shape. Freedom
will be forfeit to knowledge, but it will be the state’s knowledge that
it exercises, not for the sake of revenue but for the sake of its own
perpetuation.
[...]
Joe” Stalin
[...]
The road from Shenzhen to an American or European airport also leads to
the Roomba vacuum cleaner mapping your living room and your breakfast
with Alexa
[...]
one direction lies the possibility of a synthetic declaration for a
third modernity based on the strengthening of democratic institutions
and the creative construction of a double movement for our time. On
this road we harness the digital to forms of information capitalism
that reunite supply and demand in ways that are both genuinely
productive of effective life and compatible with a flourishing
democratic social order. The first step down this road begins with
naming, establishing our bearings, reawakening our astonishment, and
sharing a sense of righteous indignity.
[...]
They aim to fashion a new society that emulates machine learning in
much the same way that industrial society was patterned on the
disciplines and methods of factory production. In their vision,
instrumentarian power replaces social trust, Big Other substitutes
certainty for social relations, and society as we know it shades into
obsolescence
[...]
Citing Abraham Lincoln, Facebook’s founder located his company’s
mission in the evolutionary time line of civilization, during which
humanity organized itself first in tribes, then cities, then nations.
The next phase of social evolution would be “global community,” and
Facebook was to lead the way, constructing the means and overseeing the
ends.14 Speaking at Facebook’s 2017 developers’ conference, Zuckerberg
linked his assertion of the company’s historic role in establishing a
“global community” to the standard myth of the modern utopia, assuring
his followers, “In the future, technology is going to… free us up to
spend more time on the things we all care about, like enjoying and
interacting with each other and expressing ourselves in new ways.… A
lot more of us are gonna do what today is considered the arts, and
that’s gonna form the basis of a lot of our communities
[...]
The “societal goal” articulated by the leading surveillance capitalists
fits snugly into the notion of limitless technological progress that
dominated utopian thought from the late eighteenth century through the
late nineteenth century, culminating with Marx. Indeed, surveillance
capitalists such as Nadella, Page, and Zuckerberg conform to five of
the six elements with which the great scholars of utopian thought,
Frank and Fritzie Manuel, define the classic profile of the most
ambitious modern utopianists: (1) a tendency toward highly focused
tunnel vision that simplifies the utopian challenge, (2) an earlier and
more trenchant grasp of a “new state of being” than other
contemporaries, (3) the obsessive pursuit and defense of an idée fixe,
(4) an unshakable belief in the inevitability of one’s ideas coming to
fruition, and (5) the drive for total reformation at the level of the
species and the entire world system
[...]
Often a utopian foresees the later evolution and consequences of
technological development already present in an embryonic state; he may
have antennae sensitive to the future. His gadgets, however, rarely go
beyond the mechanical potentialities of his age. Try as he may to
invent something wholly new, he cannot make a world out of nothing.”18
In our time, however, surveillance capitalists can and do make such a
world—a genuinely historic deviation from the norm. Individually and
collectively, the
[...]
The only way to grasp the theory advanced in their applied utopistics
is to reverse engineer their operations and scrutinize their meaning,
as we have done throughout these chapters.
[...]
Microsoft’s instrumentarian society, the factories and workplaces are
like Skinner’s labs, and the machines replace his pigeons and rats.
These are the settings where the architecture and velocities of
instrumentarian power are readied for translation to society in a
digital-age iteration of Walden Two in which machine relations are the
model for social relations. Nadella’s construction site exemplifies the
grand confluence in which machines and humans are united as objects in
the cloud, all instrumented and orchestrated in accordance with the
“policies.” The magnificence of “policies” lies precisely in the fact
[...]
result is that “policies” are functionally equivalent to plans, as Big
Other directs human and machine action. It ensures that doors will be
locked or unlocked, car engines will shut down or come to life, the
jackhammer will scream “no” in suicidal self-sacrifice, the worker will
adhere to norms, the group will swarm to defeat anomalies. We will all
be safe as each organism hums in harmony with every other organism,
less a society than a population that ebbs and flows in perfect
frictionless confluence, shaped by the means of behavioral modification
that elude our awareness and thus can neither be mourned nor resisted.
[...]
the twentieth century the critical success factors of industrial
capitalism—efficiency, productivity, standardization,
interchangeability, the minute division of labor, discipline,
attention, scheduling, conformity, hierarchical administration, the
separation of knowing and doing, and so forth—were discovered and
crafted in the workplace and then transposed to society, where they
were institutionalized in schools, hospitals, family life, and
personality. As generations of scholars have documented, society became
more factory-like so that we might train and socialize the youngest
among us to fit the new requirements of a mass production order.
[...]
With conspicuously thin theory complemented by thick practice, the
patented device is designed to monitor user behavior in order to
preemptively detect “any deviation from normal or acceptable behavior
that is likely to affect the
[...]
Alternatively, the behavior could be assessed in relation to a “feature
distribution representing normal and/or acceptable behavior for an
average member of a population
[...]
user’s mental state
[...]
the circle widens as the patent specifications unfold. The scientists
note the utility of alerts for health care providers, insurance
companies, and law-enforcement personnel. Here is a new
surveillance-as-a-service opportunity geared to preempt whatever
behavior clients choose. Microsoft’s patent returns us to Planck,
[...]
In each case, corporate objectives define the “policies” toward which
confluent behavior harmoniously streams.
[...]
The machine hive—the confluent mind created by machine learning—is the
material means to the final elimination of the chaotic elements that
interfere with guaranteed outcomes
[...]
Instead of the typical assurances that machines can be designed to be
more like human beings and therefore less threatening, Schmidt and
Thrun argue just the opposite: it is necessary for people to become
more machine-like.
[...]
In this world the “correct” outcomes are known in advance and
guaranteed in action. The same ubiquitous instrumentation and
transparency that define the machine system must also define the social
system, which in the end is simply another way of describing the ground
truth of instrumentarian society.
[...]
this human hive, individual freedom is forfeit to collective knowledge
and action. Nonharmonious elements are preemptively targeted with high
doses of tuning, herding, and conditioning, including the full
seductive force of social persuasion and influence. We march in
certainty, like the smart machines. We learn to sacrifice our freedom
to collective knowledge imposed by others and for the sake of their
guaranteed outcomes. This is the signature of the third modernity
offered up by surveillance capital as its answer to our quest for
effective life together
[...]
Pentland is often referred to as the “godfather of wearables,”
especially Google Glass. In 1998 he predicted that wearables “can
extend one’s senses, improve memory, aid the wearer’s social life and
even help him or her stay calm and collected
[...]
Most noteworthy is that Pentland “completes” Skinner, fulfilling his
social vision with big data, ubiquitous digital instrumentation,
advanced mathematics, sweeping theory, numerous esteemed coauthors,
institutional legitimacy, lavish funding, and corporate friends in high
places without having attracted the worldwide backlash, moral
revulsion, and naked vitriol once heaped on Harvard’s outspoken
behaviorist. This fact alone suggests the depth of psychic numbing to
which we have succumbed and the loss of our collective bearings.
[...]
’s like watching beavers from outer space, like Jane Goodall watching
gorillas. You observe from a distance.”7 (This is a slur on Goodall, of
course, whose seminal genius was her ability to understand the gorillas
she studied not as “other ones” but rather as “one of us.”)
[...]
The team saw that it would be possible to exploit the increasingly
“ubiquitous infrastructure” of mobile phones and combine those data
with new streams of information from their wearable behavioral
monitors. The result was a radical new solution that Pentland and Eagle
called “reality mining
[...]
Pentland argued that information gathered by his
sociometers—“unobtrusive wearable sensors” measuring communication,
voice tones, and body language—“could help managers understand who is
working with whom and infer the relationships between colleagues” and
“would be an efficient way to find people who might work well
together.”20
[...]
people analytics
[...]
Pentland appeared in 2016 at a conference organized by Singularity
University, a Silicon Valley hub of instrumentarian ideology funded in
part by Larry Page. An interviewer tasked to write about Pentland
explains, “Though people are one of the most valuable assets in an
organization, many companies are still approaching management with a
20th century mentality.… Pentland saw the factor that was always
messing things up was—the people.”29 Like Nadella, Pentland described
his aims as developing the social systems that would work along the
same lines as the machine systems, using behavioral data flows to judge
the “correctness” of action patterns and to intervene when it is
necessary to change “bad” action to “correct” action. “If people aren’t
interacting correctly and information isn’t spreading correctly,”
Pentland warns, “people
[...]
Pentland articulated his ambitions for the capabilities and objectives
of this new milieu in a series of papers, published primarily between
2011 and 2014, but one remarkable 2011 essay of which he is the sole
author stands out: “Society’s Nervous System: Building Effective
Government, Energy, and Public Health Systems.”31
[...]
The initial premise is reasonable enough: industrial-age technology
once revolutionized the world with reliable systems for water, food,
waste, energy, transportation, police, health care, education, and so
forth, but these systems are now hopelessly “old,” “centralized,”
“obsolete,” and “unsustainable.” New digital systems are required that
must be “integrated,” “holistic,” “responsive,” “dynamic,” and
“self-regulating”: “We need a radical rethinking of societies’ systems.
We must create a nervous system for humanity that maintains the
stability of our societies’ systems
[...]
What is missing… are the dynamic models of demand and reaction,” along
with an architecture that guarantees “safety, stability, and
efficiency.… The models required must describe human
[...]
Regarding incentives, Pentland outlines a principle of “social
efficiency,” which means that participation must provide value to the
individual but also to the system as a whole.37 For the sake of this
wholeness, it is believed, each of us will surrender to a totally
measured life of instrumentarian order
[...]
Skinner advocated, via Frazier, that the virtue of a “planned society”
is “to keep intelligence on the right track, for the good of society
rather than of the intelligent individual.… It does this by making sure
that the individual will not forget his
[...]
Pentland says that “continuous streams of data about human behavior”
mean that everything from traffic, to energy use, to disease, to street
crime will be accurately forecast, enabling a “world without war or
financial crashes, in which infectious disease is quickly detected and
stopped, in which energy, water, and other resources are no longer
wasted, and in which governments are part of the solution rather than
part of the problem.”48 This new “collective intelligence” operates to
serve the greater good as we learn to act “in a coordinated manner”
based on “social universals.” “Great leaps in health care,
transportation
[...]
The main barriers are privacy concerns and the fact that we don’t yet
have any consensus around the trade-offs between personal and social
values.” Like Skinner, he is emphatic that these attachments to a
bygone era of imperfect knowledge threaten to undermine the prospect of
a perfectly engineered future society: “We cannot ignore the public
goods that such a nervous system could provide.…”49 Pentland avoids the
question “Whose greater good?” How is the greater good determined when
surveillance capitalism owns the machines and the means of behavioral
modification? “Goodness” arrives already oriented toward the interests
of the owners of the means of behavioral modification and the clients
whose guaranteed outcomes they seek to achieve. The greater good is
someone’s, but it may not be ours
[...]
Capitalism and socialism are equally tainted by their shared emphasis
on economic growth, which breeds overconsumption and pollution. Skinner
is intrigued by the Chinese system but rejects it on the grounds of the
bloody revolution that any effort to convert Westerners would entail.
“Fortunately,” Skinner concludes in the preface to Walden Two, “there
is another possibility.” This option is Skinner’s version of a
behaviorist society that provides a way in which “political action is
to be avoided.” In Walden Two a “plan” replaces politics, overseen by a
“noncompetitive” group of “Planners” who eschew power in favor of the
dispassionate administration of the schedules of reinforcement aimed at
the greater good.52 Planners exercise unique control over society but
“only because that control is necessary for the proper functioning of
the community
[...]
Pentland worries that our political-economic constructs such as
“market” and “class” hail from an old, slow world of the eighteenth and
nineteenth centuries. The new, “light-speed hyperconnected world”
leaves no time for the kind of rational deliberation and face-to-face
negotiation and compromise that characterized the social milieu in
which such political concepts originated
[...]
There is no room for politics in this instrumentarian society because
politics means establishing and asserting our bearings. Individual
moral and political bearings are a source of friction that wastes
precious time and diverts behavior from confluence
[...]
Computation thus replaces the political life of the community as the
basis for governance. The depth and breadth of instrumentation make it
possible, Pentland says, to calculate idea flow, social network
structure, the degree of social influence between people, and even
“individual susceptibilities to new ideas.” Most important,
instrumentation makes it possible for those with the God view to modify
others’ behavior. The data provide a “reliable prediction of how
changing any of these variables will change the performance of all the
people
[...]
Frazier acknowledges that you cannot coerce people into doing the right
thing. The solution is far more subtle and sophisticated, based upon
scientifically calibrated schedules of reinforcement: “Instead you have
to set up certain behavioral processes which will lead the individual
to design his own ‘good’ conduct.… We call that sort of thing
‘self-control.’ But don’t be misled, the control always rests in the
last analysis in the hands of society
[...]
Pentland’s idea is comparable: “The social physics approach to getting
everyone to cooperate” is “social network incentives,” his version of
“reinforcement.” With such incentives, he explains, “we focus on
changing the connections between people rather than focusing on getting
people individually to change their behavior.… We can leverage those
exchanges to generate social pressure for change.”60 Social media is
critical to establishing these tuning capabilities, Pentland believes,
because this is the environment in which social pressure can best be
controlled, directed, manipulated, and scaled
[...]
Pentland ignores the role of empathy in emulation because empathy is a
felt experience that is not subject to the observable metrics required
for computational governance. Instead, Pentland subscribes to the label
Homo imitans to convey that it is mimicry, not empathy, and certainly
not politics, which defines human existence
[...]
stream of ideas as a swarm or collective intelligence, flowing through
time, with all the humans in it learning from each other’s experiences
in order to jointly discover the patterns of preferences and habits of
action that best
[...]
What is being abolished is autonomous man—the inner man, the
homunculus, the possessing demon, the man defended by the literatures
of freedom and dignity
[...]
One important study of Bitcoin, the cryptocurrency that relies on
blockchain, suggests that such machine solutions both express and
contribute to the general erosion of the social fabric in ways that are
both consistent with instrumentarianism and further pave the way for
its success. Information scholars Primavera De Filippi and Benjamin
Loveluck conclude that contrary to popular belief, “Bitcoin is neither
anonymous nor privacy-friendly
[...]
We can begin by asking our children. Without knowing it, we sent the
least formed and most vulnerable among us to scout the hive and settle
its wilderness. Now their messages are filtering in from the frontier
[...]
Indeed, Facebook’s early advantage in this work arose in no small
measure from the simple fact that its founders and original designers
were themselves adolescents and emerging adults. They designed
practices for an imagined universe of adolescent users and college
[...]
contrary to Pentland’s belief that “class” divisions would disappear,
life in the hive produces new cleavages and forms of stratification:
not only tune or be tuned but also pressure or be pressured
[...]
Schüll learned that addictive players seek neither entertainment nor
the mythical jackpot of cash. Instead, they chase what Harvard Medical
School addiction researcher Howard Shaffer calls “the capacity of the
drug or gamble to shift subjective experience,” pursuing an
experiential state that Schüll calls the “machine zone,” a state of
self-forgetting in which one is carried along by an irresistible
momentum that feels like one is “played by the machine.”12 The machine
zone achieves a sense of complete immersion
[...]
Addiction by Design
[...]
Shaffer, the addiction researcher, has identified five elements that
characterize this state of compulsion: frequency of use, duration of
action, potency, route of administration, and player attributes
[...]
Perhaps the most difficult quality to capture is that in this period
that precedes the hard bargaining, an “inner” sense of “self” simply
does not yet exist. It is a time when “I” am whatever the “others”
think of me, and how “I” feel is a function of how the “others” treat
me. Instead of a stable sense of identity, there is only a chameleon
that reinvents itself depending upon the social mirror into which it is
drawn. In this condition, the “others” are not individuals but the
audience for whom I perform. Who “I” am depends upon the audience. This
state
[...]
Research shows that these big leaps in self-construction are stimulated
by experiences such as structured reflection, conflict, dissonance,
crisis, and failure. The people who help trigger this new inward
connection refuse to act as our mirrors. They reject fusion in favor of
genuine reciprocity
[...]
What are the consequences of the failure to win a healthy balance
between inner and outer, self and relationship? Clinical studies
identify specific patterns associated with this developmental
stagnation. Not surprisingly, these include an inability to tolerate
solitude, the feeling of being merged with others, an unstable sense of
self, and even an excessive need to control others as a way of keeping
the mirror close. Loss of the mirror is the felt equivalent of
extinction
[...]
The cultivation of inner resources is thus critical to the capacity for
intimacy and relationship, challenges that have become more
time-consuming with each new phase of the modern era. And while young
people are bound as ever to the enduring existential task of
self-making, our story suggests three critical ways in which this task
now converges with history and the unique conditions of existence in
our time
[...]
For example, Evil by Design author Chris Nodder, a user-experience
consultant, explains that evil design aims to exploit human weakness by
creating interfaces that “make users emotionally involved in doing
something that benefits the designer more than them.” He coaches his
readers in psychic numbing, urging them to accept the fact that such
practices have become the standard suggesting that consumers and
designers find ways to “turn them to your advantage
[...]
Facebook’s precocious mastery of “social proof”: “Much of our behavior
is determined by our impressions of what is the correct thing to do…
based on what we observe others doing.… This influence is known as
social proof
[...]
Most critical is that the more the need for the “others” is fed, the
less able one is to engage the work of self-construction. So
devastating is the failure to attain that positive equilibrium between
inner and outer life that Lapsley and Woodbury say it is “at the heart”
of most adult personality disorders
[...]
more that a user “liked,” the more that she informed Facebook about the
precise shape and composition of her “hand,” thus allowing the company
to continuously tighten the glove and increase the predictive value of
her signals.
[...]
On the demand side, Facebook’s “likes” were quickly coveted and craved,
morphing into a universal reward system or what one young app designer
called “our generation’s crack cocaine.” “Likes” became those variably
timed dopamine shots, driving users to double down on their bets “every
time they
[...]
News Feed is also the fulcrum of the social mirror. In the years
between revulsion and reverence, News Feed became Facebook’s most
intensely scrutinized object of data science and the subject of
extensive organizational innovation, all of it undertaken at a level of
sophistication and capital intensity that one might more naturally
associate with the drive to solve world hunger, cure cancer, or avert
climate destruction
[...]
laugh, cry, smile, click, like, share, or comment.”40 The glove
tightens around the hand with closed feedback loops enabled by the God
view, which favors posts from people with whom you have already
interacted, posts that have drawn high levels of engagement from
others, and posts that are like the ones with which you have already
engaged
[...]
According to the 302 most significant quantitative research studies on
the relationships between social media use and mental health (most of
them produced since 2013), the psychological process that most defines
the Facebook experience is what psychologists call “social
comparison.”45 It is usually considered a natural and virtually
[...]
One study found an increase in criminal larceny as television diffused
across society, awakening an awareness of and desire for consumer
goods. A related issue was that increased exposure to television
programs depicting affluence led to “the overestimation of others’
wealth and more dissatisfaction with one’s own life
[...]
Both television and social media deprive us of real-life encounters, in
which we sense the other’s inwardness and share something of our own,
thus establishing some threads of communality. Unlike
[...]
consequence of the new density of social comparison triggers and their
negative feedback loops is a psychological condition known as FOMO
(“fear of missing out”). It
[...]
Profile inflation triggers more negative self-evaluation among
individuals as people compare themselves to others, which then leads to
more profile inflation, especially among larger networks that include
more “distant friends.” As one study concluded, “Expanding one’s social
network by adding a number of distant friends through Facebook may be
detrimental by stimulating negative emotions for users
[...]
This compulsive behavior is intended to produce relief in the form of
social reassurance, but it predictably breeds more anxiety and more
searching.52 Social comparison
[...]
When considered from the vantage point of the self-other balance,
positive social comparisons are just as pernicious as negative
comparisons. Both are substitutes for the “hard bargain” of carving out
a self that is capable of reciprocity rather than fusion
[...]
Facebook use does not promote well-being.… Individual social media
users might do well to curtail their use of social media and focus
instead on real-world relationships
[...]
This is the world of Pentland’s “social learning,” his theory of
“tuning” little more than the systematic manipulation of the rewards
and punishments of inclusion and exclusion. It succeeds through the
natural human inclination to avoid psychological pain
[...]
confluence,” in which harmonies are achieved at the expense of the
psychological integrity of participants
[...]
This synthetic hive is a devilish pact for a young person. In terms of
sheer everyday effectiveness—contact, logistics, transactions,
communications—turn away, and you are lost. And if you simply crave the
fusion juice that is proof of life at a certain age and stage—turn
away, and you are extinguished
[...]
Just as Pentland stipulated, these closed loops are imposed outside the
realm of politics and individual volition. They move in stealth,
crafting their effects at the level of automatic psychological
responses and tipping the self-other balance toward the
pseudo-harmonies of the hive mind. In this process, the inwardness that
is the necessary source of autonomous action and moral judgment suffers
and suffocates. These are the preparatory steps toward the death of
individuality that Pentland advocates. In fact, this
[...]
the eighteenth century’s political ideal of the individual as the
repository of inalienable dignity, rights, and obligations; (2) the
early twentieth century’s individualized
[...]
human being called into existence by history, embarking on Machado’s
road because she must, destined to create “a life of one’s own” in a
world of ever-intensifying social complexity and receding traditions;
and (3) the late twentieth century’s psychologically autonomous
individual whose inner resources and capacity for moral judgment rise
to the challenges of self-authorship that history demands and act as a
bulwark against the predations of power. The self-authorship toward
which young people strive
[...]
post-political societal processes that bind the hive rely on social
comparison and social pressure for their durability and predictive
certainty, eliminating the need for trust
[...]
the closing lines of Jean-Paul Sartre’s existential drama No Exit, the
character Garcin arrives at his famous realization, “Hell is other
people.” This was not intended as a statement of misanthropy but rather
a recognition that the self-other balance can never be adequately
struck as long as the “others” are constantly “watching.” Another
mid-century social psychologist, Erving Goffman, took up these themes
in his immortal The Presentation of Self in Everyday Life. Goffman
developed the idea of the “backstage” as the region in which the self
retreats from the performative demands of social life.
[...]
work as in life, “control of the backstage” allows individuals “to
buffer themselves from the deterministic demands that surround them.”
Backstage, the language is one of reciprocity, familiarity, intimacy,
humor. It offers the seclusion in which one can surrender to the
“uncomposed” face in sleep, defecation, sex, “whistling, chewing,
nibbling, belching, and flatulence.” Perhaps most of all, it is an
opportunity for “regression,” in which we don’t have to be “nice”: “The
surest sign of backstage solidarity is to feel that it is safe to lapse
into an asociable mood of sullen, silent irritability.” In the absence
of such respite where a “real” self can incubate and grow, Sartre’s
idea of hell begins to make sense.62
[...]
Milgram identified three key themes in the subway experiment as he and
his students debriefed their experiences. The first was a new sense of
gravitas toward “the enormous inhibitory anxiety that ordinarily
prevents us from breaching social norms.” Second was that the reactions
of the “breacher” are not an expression of individual personality but
rather are “a compelled playing out of the logic of social relations
[...]
Embarrassment and the fear of violating apparently trivial norms often
lock us into intolerable predicaments.… These are not minor regulatory
forces in social life, but basic ones.” Finally, Milgram understood
that any confrontation of social norms crucially depends upon the
ability to escape. It was not an adolescent who boarded the subway that
day. Milgram was an erudite adult and an expert on human behavior,
especially the mechanisms entailed in obedience to authority, social
influence, and conformity. The subway was just an ordinary slice of
life, not a capital-intensive architecture of surveillance and behavior
modification, not a “personalized reward device.” Still, Milgram could
not fight off the anxiety of the situation. The only thing that made it
tolerable was the possibility of an exit. Unlike Milgram, we face an
intolerable situation
[...]
are meant to fuse with the system and play to extinction: not the
extinction of our funds but rather the extinction of our selves.
Extinction is a design feature formalized in the conditions of no exit.
The aim of the tuners is to contain us within “the power of immediate
circumstances” as we are compelled by the “logic of social relations”
in the hive to bow to social pressure exerted in calculated patterns
that exploit our natural empathy. Continuously tightening feedback
loops cut off the means of exit, creating impossible levels of anxiety
that further drive the loops toward confluence. What is to be killed
here is the inner impulse toward autonomy and the arduous, exciting
elaboration of the autonomous self as a source of moral judgment and
authority capable of asking for a subway seat or standing against rogue
power.
[...]
To exit means to enter the place where a self can be birthed and
nurtured. History has a name for that kind of place: sanctuary
[...]
We know that nothing guarantees safety and certainty in this world, but
we are comforted by the serenity of this home and its layered silences.
The days unfurl now
[...]
In the march of institutional interests intent on implementing Big
Other, the very first citadel to fall is the most ancient: the
principle of sanctuary. The sanctuary privilege has stood as an
antidote to power since the beginning of the human story. Even in
ancient societies where tyranny prevailed, the right of sanctuary stood
as a fail-safe. There was an exit from totalizing power, and that exit
was the entrance to a sanctuary in the form of a city, a community, or
a temple.4 By the time of the Greeks, sanctuaries were sacred sites
built across the ancient Greek world and consecrated to the purposes of
asylum and religious sacrifice. The Greek word asylon means
“unplunderable” and founds the notion
[...]
sanctuary as an inviolable space.5 The right of asylum survived into
the eighteenth century in many parts of Europe, attached to holy sites,
churches, and monasteries. The demise of the sanctuary privilege was
not a repudiation but rather a reflection of social evolution and the
firm establishment of the rule of law. One historian summarized this
transformation: “justice as sanctuary.”6 In the modern
[...]
empirical study makes the point. In “Psychological Functions of
Privacy,” Darhl Pedersen defines privacy as a “boundary control
process” that invokes the decision rights associated with “restricting
and seeking interaction
[...]
The same themes appear from the perspective of psychology. Those who
would eviscerate sanctuary are keen to take the offensive, putting us
off guard with the guilt-inducing question “What have you got to hide?”
But as we have seen, the crucial developmental challenges of the
self-other balance cannot be negotiated adequately without the sanctity
of “disconnected” time and space for the ripening of inward awareness
and the possibility of reflexivity: reflection on and by oneself. The
real psychological truth is this: If you’ve got nothing to hide, you
are nothing.
[...]
six categories of privacy behaviors: solitude, isolation
[...]
proper realm of inaccessibility or secrecy with respect to the world at
large as well as a recognition of the important social dimension of
such protected inner space.…”7
[...]
contemplation, autonomy, rejuvenation, confiding, freedom, creativity,
recovery, catharsis, and concealment
[...]
anonymity, reserve, intimacy with friends, and intimacy with family
[...]
billions of sensors filled with personal data fall outside of Fourth
Amendment protections, a large-scale surveillance network will exist
without constitutional limits
[...]
This theme is illustrated in the odyssey of Belgian mathematician and
data protection activist Paul-Olivier Dehaye, who in December 2016
initiated a request for his personal data collected through Facebook’s
Custom Audiences and tracking Pixel tools, which would reveal the web
pages where Facebook had tracked him. Dehaye probably knew more about
the rogue data operations of Cambridge Analytica than anyone in the
world, outside of its own staff and masterminds
[...]
of the right to contest “automatic decision making.” If the algorithms
are to be contestable in any meaningful way, it will require new
countervailing authority and power, including machine resources and
expertise to reach into the core disciplines of machine intelligence
and construct new approaches that are available for inspection, debate,
and combat. Indeed, one expert has already proposed the creation of a
government agency—an “FDA for algorithms
[...]
is already possible to see a new awakening to empowering collective
action, at least in the privacy domain. One example is None of Your
Business (NOYB), a nonprofit organization led by privacy activist Max
Schrems. After many years of legal contest, Schrems made history in
2015 when his challenge to Facebook’s data-collection and
data-retention practices—which he asserted were in violation of EU
privacy law—led the Court of Justice of the European Union to
invalidate the Safe Harbor agreement that governed data transfers
[...]
the absence of synthetic declarations that secure the road to a human
future, the intolerability of glass life turns us toward a societal
arms race of counter-declarations in which we search for and embrace
increasingly complex ways to hide in our own lives, seeking respite
from lawless machines and their masters. We do this to satisfy our
enduring need for sanctuary and as an act of resistance with which to
reject the instrumentarian disciplines of the hive, its “extended
chilling effects,” and Big Other’s relentless greed. In the context of
government surveillance, the practices of “hiding” have been called
“privacy protests” and are well-known for drawing the suspicion of
law-enforcement agencies.33 Now, hiding is also invoked by Big Other
and its market masters, whose reach is far and deep as they install
themselves in our walls, our bodies, and on our streets, claiming our
faces, our feelings, and our fears of exclusion. I have suggested
[...]
Equally more poignant is the way in which a new generation of
activists, artists, and inventors feels itself called to create the art
and science of hiding.34 The intolerable conditions of glass life
compel these
[...]
Chicago artist Leo Selvaggio produces 3-D–printed resin prosthetic
masks to confound facial recognition. He calls his effort “an organized
artistic intervention.”35 Perhaps most poignant is the Backslash Tool
Kit: “a series of functional devices designed for protests and riots of
the future,
[...]
New Museum for Contemporary Art in Manhattan, and you pass a display of
its bestseller: table-top mirrors whose reflecting surface is covered
with the bright-orange message “Today’s Selfie Is Tomorrow’s Biometric
Profile.” This “Think Privacy Selfie Mirror” is a project of the young
Berlin-based artist Adam Harvey, whose work is aimed at the problem of
surveillance and
[...]
Trevor Paglen’s richly orchestrated performance art combines music,
photography, satellite imagery, and artificial intelligence to reveal
Big Other’s omnipresent knowing and doing. “It’s trying to look inside
the software that is running an AI… to look into the architectures of
different computer vision
[...]
greatest danger is that we come to feel at home in glass life or in the
prospect of hiding from it. Both alternatives rob us of the
life-sustaining inwardness
[...]
Glass life is intolerable, but so is fitting our faces with masks and
draping our bodies in digitally resistant fabrics to thwart the
ubiquitous lawless machines. Like every counter-declaration, hiding
risks becomes an adaptation when it should be a rallying point for
outrage. These conditions are unacceptable. Tunnels under this wall are
not enough. This wall must come down.
[...]
Surveillance capitalists are no different from other capitalists in
demanding freedom from any sort of constraint. They insist upon the
“freedom to” launch every novel practice while aggressively asserting
the necessity of their “freedom from” law and regulation. This classic
pattern reflects two bedrock assumptions about capitalism made by its
own theorists: The first is that markets are intrinsically unknowable.
The second is that the ignorance produced by this lack of knowledge
requires wide-ranging freedom of action for market actors
[...]
Adam Smith’s famous metaphor of the “invisible hand” drew on these
enduring realities of human life. Each individual, Smith reasoned,
employs his capital locally in pursuit of immediate comforts and
necessities. Each one attends to “his own security… his own gain… led
by an invisible hand to promote an end which was no part of his
intention.” That end is the efficient employ of capital in the broader
market: the wealth of nations. The individual actions that produce
efficient markets add up to a staggeringly complex pattern, a mystery
that no one person or entity could hope to know or understand, let
alone to direct
[...]
Adam Smith,” Hayek wrote, “was the first to perceive that we have
stumbled upon methods of ordering human economic cooperation that
exceed the limits of our knowledge and perception. His ‘invisible hand’
had perhaps better have been described as an invisible or unsurveyable
pattern
[...]
As with Planck, Meyer, and Skinner, both Hayek and Smith unequivocally
link freedom and ignorance. In Hayek’s framing, the mystery of the
market is that a great many people can behave effectively while
remaining ignorant of the whole. Individuals not only can choose
freely, but they must freely choose their own pursuits because there is
no alternative, no source of total knowledge or conscious control to
guide them. “Human design” is impossible, Hayek says, because the
relevant information flows are “beyond the span of the control of any
one mind
[...]
However, Big Other and the steady application of instrumentarian power
challenge the classic quid pro quo of freedom for ignorance. When it
comes to surveillance capitalist operations, the “market” is no longer
invisible
[...]
Hayek chose the market over democracy, arguing that the market system
enabled not only the division of labor but also “the coordinated
utilization of resources based on equally divided knowledge.” This
system, he argued, is the only one compatible with freedom. Perhaps
some other kind of civilization might have been devised, he reckoned,
“like the ‘state’ of the termite ants,” but it would not be compatible
with human freedom.4
[...]
More astonishing still is that surveillance capital derives from the
dispossession of human experience, operationalized in its unilateral
and pervasive programs of rendition: our lives are scraped and sold to
fund their freedom and our subjugation, their knowledge and our
ignorance about what they know
[...]
One conclusion of our investigations is that surveillance capitalism’s
command and control of the division of learning in society are the
signature feature that breaks with the old justifications of the
invisible hand and its entitlements. The combination of knowledge and
freedom works to accelerate the asymmetry of power between surveillance
capitalists and the societies in which they operate. This cycle will be
broken only when we acknowledge as citizens, as societies, and indeed
as a civilization that surveillance capitalists know too much to
qualify for freedom
[...]
The surveillance capitalists that operate at hyperscale or outsource to
hyperscale operations dramatically diminish any reliance on their
societies as sources of employees, and the few for whom they do
compete, as we have seen, are drawn from the most-rarefied strata of
data science
[...]
The absence of organic reciprocities with people as either sources of
consumers or employees is a matter of exceptional importance in light
of the historical relationship between market capitalism and democracy
[...]
’s dependency on the “masses” and their contribution to the prosperity
necessitated by the new organization of production.23 The rise of
volume production and its wage-earning labor force established British
workers’ economic power and led to a growing appreciation of their
political legitimacy and power. This produced a new sense of
interdependence between ordinary people and elites. Acemoglu and
Robinson conclude that the “dynamic positive feedback” between
“inclusive economic institutions” (i.e., industrial firms defined by
employment reciprocities) and political institutions was critical to
Britain’s substantial and nonviolent democratic reforms. Inclusive
economic institutions, they argue, “level the playing field,”
especially when it comes to the fight for power, making it more
difficult for elites to “crush the masses” rather than accede to their
demands. Reciprocities in employment produced and sustained
reciprocities in politics
[...]
sharp contrast to the pragmatic concessions of Britain’s early
industrial capitalists, surveillance capitalists’ extreme structural
independence from people breeds exclusion rather than inclusion and
lays the foundation for the unique approach that we have called
“radical indifference
[...]
significant result of the systematic application of radical
indifference is that the public-facing “first text” is vulnerable to
corruption with content that would normally be perceived as repugnant:
lies, systematic disinformation, fraud, violence, hate speech, and so
on. As long as content contributes to “growth tactics,” Facebook
“wins.” This vulnerability can be an explosive problem on
[...]
guiding principles of radical indifference are reflected in the
operations of Facebook’s hidden low-wage labor force charged with
limiting the perversion of the first text. Nowhere is surveillance
capitalism’s outsized influence over the division of learning in
society more concretely displayed than in this outcast function of
“content moderation,” and nowhere is the nexus of economic imperatives
and
[...]
The larger point of the exercise is to find the point of equilibrium
between the ability to pull users and their surplus into
[...]
site and the risk of repelling them. This is a calculation of radical
indifference that has nothing to do with assessing the truthfulness of
content or respecting reciprocities with users.36 This tension helps to
explain why disinformation is not a priority. One investigative report
quotes a Facebook insider: “They absolutely have the tools to shut down
fake news.
[...]
radical indifference is a permanent invitation to the corruption of the
first text
[...]
It is obvious that the rogue forces of disinformation grasp this fact
more crisply than do Facebook’s or Google’s genuine users and customers
as those forces learn to exploit the blind eye of radical indifference
and escalate the perversion of learning in an open society
[...]
Surveillance capitalism’s antidemocratic and antiegalitarian juggernaut
is best described as a market-driven coup from above. It is not a coup
d’état in the classic sense but rather a coup de gens: an overthrow of
the people concealed as the technological Trojan horse that is Big
Other. On the strength of its annexation of human experience, this coup
achieves exclusive concentrations of knowledge and power that sustain
privileged influence over the division of learning in society: the
privatization of the central principle of social ordering in the
twenty-first century. Like the adelantados and their silent
incantations of the Requirimiento, surveillance capitalism operates in
the declarative form and imposes the social relations of a premodern
absolutist authority. It is a form of tyranny that feeds on people but
is not of the people. In a surreal paradox, this coup is celebrated as
“personalization,” although it defiles, ignores, overrides, and
displaces everything about you and me that is personal
[...]
Tyranny” is not a word that I choose lightly. Like the instrumentarian
hive, tyranny is the obliteration of politics. It is founded on its own
strain of radical indifference in which every person, except the
tyrant, is understood as an organism among organisms in an equivalency
of Other-Ones. Hannah Arendt observed that tyranny is a perversion of
egalitarianism because it treats all others as equally insignificant:
“The tyrant rules in accordance with his own will and interest… the
ruler who rules one against all, and the ‘all’ he oppresses are all
equal, namely equally powerless.” Arendt notes that classical political
theory regarded the tyrant as “out of mankind altogether… a wolf in
human shape.…”55Surveillance capitalism rules by instrumentarian power
through its materialization in Big Other, which, like the ancient
tyrant, exists out of mankind while paradoxically assuming human shape
[...]
Polanyi’s lens, we see that surveillance capitalism annexes human
experience to the market dynamic so that it is reborn as behavior: the
fourth “fictional commodity.” Polanyi’s first three fictional
commodities—land, labor, and money—were subjected to law. Although
these laws have been imperfect, the institutions of labor law,
environmental law, and banking law are regulatory frameworks intended
to defend society (and nature, life, and exchange) from the worst
excesses of raw capitalism’s destructive power. Surveillance
capitalism’s expropriation of human experience has faced no such
impediments.
[...]
will be Facebook, he says, that will address problems that are
civilizational in scale and scope, building “the long-term
infrastructure to bring humanity together” and keeping people safe with
“artificial intelligence” that quickly understands “what is happening
across our community.”56 Like Pentland, Zuckerberg imagines machine
intelligence that can “identify risks that nobody would have flagged at
all, including terrorists planning attacks using private channels,
people bullying someone too afraid to report it themselves, and other
issues both local and global.”57 When asked about his responsibility to
shareholders, Zuckerberg told CNN, “That’s why it helps to have control
of the company
[...]
industrial civilization aimed to exert control over nature for the sake
of human betterment. Machines were our means of extending and
overcoming the limits of the animal body so that we could accomplish
this aim of domination. Only later did we begin to fathom the
consequences
[...]
Years later, in his moving 1966 essay “Education after Auschwitz,”
social theorist Theodor Adorno attributed the success of German fascism
to the ways in which the quest for effective life had become an
overwhelming burden for too many people: “One must accept that fascism
and the terror it caused are connected with the fact that the old
established authorities… decayed and were toppled, while the people
psychologically were not yet ready for self-determination. They proved
to be unequal to the freedom that fell into their laps.”67
[...]
We can now see that surveillance capitalism takes an even more
expansive turn toward domination than its neoliberal source code would
predict, claiming its right to freedom and knowledge, while setting its
sights on a collectivist vision that claims the totality of society.
Though still sounding like Hayek, and even Smith, its antidemocratic
collectivist ambitions reveal it as an insatiable child devouring its
aging fathers
[...]
The critical role of public opinion explains why even the most
destructive “ages” do not last forever. I echo here what Edison said a
century ago: that capitalism is “all wrong, out of gear.” The
instability of Edison’s day threatened every promise of industrial
civilization. It had to give way, he insisted, to a new synthesis that
reunited capitalism and its populations. Edison was prophetic.
Capitalism has survived the longue durée less because of any specific
capability and more because of its plasticity. It survives and thrives
by periodically renewing its roots in the social, finding new ways to
generate new wealth by meeting new needs. Its evolution has been marked
by a convergence of basic principles—private property, the profit
motive, and
[...]
It is not OK for every move, emotion, utterance, and desire to be
catalogued, manipulated, and then used to surreptitiously herd us
through the future tense for the sake of someone else’s profit. “These
things are brand-new,” I tell them. “They are unprecedented. You should
not take them for granted because they are not OK
[...]
Burnham’s cowardice is a cautionary tale. We are living in a moment
when surveillance capitalism and its instrumentarian power appear to be
invincible. Orwell’s courage demands that we refuse to cede the future
to illegitimate power. He asks us to break the spell of enthrallment,
helplessness, resignation, and numbing. We answer his call when we bend
ourselves toward friction, rejecting the smooth flows of coercive
confluence. Orwell’s courage sets us against the relentless tides of
dispossession that demean all human experience. Friction, courage, and
bearings are the resources we require to begin the shared work of
synthetic declarations that claim the digital future as a human place,
demand that digital capitalism operate as an inclusive force bound to
the people it must serve, and defend the division of learning in
society as a source of genuine democratic renewal
|