path: root/books/sociedade/who-owns-the-future.md
diff options
authorSilvio Rhatto <rhatto@riseup.net>2017-09-30 14:06:22 -0300
committerSilvio Rhatto <rhatto@riseup.net>2017-09-30 14:06:22 -0300
commit23ac9f57b9b4c761cb8edc5bfa0c0de77ec89326 (patch)
tree3dab0ec66d67cd62b7e815fea4d62da481042b7b /books/sociedade/who-owns-the-future.md
parent9c21d35c535a4956960851d3c438d58af5f67d92 (diff)
Change extension to .md
Diffstat (limited to 'books/sociedade/who-owns-the-future.md')
1 files changed, 673 insertions, 0 deletions
diff --git a/books/sociedade/who-owns-the-future.md b/books/sociedade/who-owns-the-future.md
new file mode 100644
index 0000000..e53d275
--- /dev/null
+++ b/books/sociedade/who-owns-the-future.md
@@ -0,0 +1,673 @@
+[[!meta title="Who owns the future?"]]
+* Author: Jaron Lanier
+* Year: 2013
+* Publisher: Simon & Schuster
+## Index
+* Star system versus the bell curve as network designs.
+* Siren Servers: narcissism, hyperamplified risk aversion, and extreme information asymmetry.
+* Siren Servers and Maxwell’s Demon.
+* Disruptive innovation as the tedious scheme to shrink markets.
+* Science isn't automatic.
+* Nine dismal humors of futurism, and a hopeful one.
+* Marx as one of the first technology writers (when discussing Luddites).
+* Human obsolescence is avoidable.
+* Keynes Considered as a Big Data Pioneer.
+* Amazon's Mechanical Turk.
+* Humanistic information economics.
+* What is experience? If personal experience were missing from the universe, how would things be different?
+* Gurus and New Age at the Sillicon Valley: Gurdjieff, Steve Jobs.
+## Prelude
+ Instagram isn’t worth a billion dollars just because those thirteen employees
+ are extraordinary. Instead, its value comes from the millions of users who
+ contribute to the network without being paid for it. Networks need a great
+ number of people to participate in them to generate significant value. But when
+ they have them, only a small number of people get paid. That has the net effect
+ of centralizing wealth and limiting overall economic growth.
+ [...]
+ By “digital networking” I mean not only the Internet and the Web, but also
+ other networks operated by outfits like financial institutions and intelligence
+ agencies. In all these cases, we see the phenomenon of power and money becoming
+ concentrated around the people who operate the most central computers in a
+ network, undervaluing everyone else. That is the pattern we have come to
+ expect, but it is not the only way things can go.
+## The Price of Heaven
+ Utopians presume the advent of abundance not because it will be affordable, but
+ because it will be free, provided we accept surveillance.
+ Starting back in the early 1980s, an initially tiny stratum of gifted
+ technologists conceived new interpretations of concepts like privacy, liberty,
+ and power. I was an early participant in the process and helped to formulate
+ many of the ideas I am criticizing in this book. What was once a tiny
+ subculture has blossomed into the dominant interpretation of computation and
+ software-mediated society.
+ One strain of what might be called “hacker culture” held that liberty means
+ absolute privacy through the use of cryptography. I remember the thrill of
+ using military-grade stealth just to argue about who should pay for a pizza at
+ MIT in 1983 or so.
+ On the other hand, some of my friends from that era, who consumed that pizza,
+ eventually became very rich building giant cross-referenced dossiers on masses
+ of people, which were put to use by financiers, advertisers, insurers, or other
+ concerns nurturing fantasies of operating the world by remote control.
+ It is typical of human nature to ignore hypocrisy. The greater a hypocrisy, the
+ more invisible it typically becomes, but we technical folk are inclined to seek
+ an airtight whole of ideas. Here is one such synthesis—of cryptography for
+ techies and massive spying on others—which I continue to hear fairly often:
+ Privacy for ordinary people can be forfeited in the near term because it will
+ become moot anyway.
+ Surveillance by the technical few on the less technical many can be tolerated
+ for now because of hopes for an endgame in which everything will become
+ transparent to everyone. Network entrepreneurs and cyber-activists alike seem
+ to imagine that today’s elite network servers in positions of information
+ supremacy will eventually become eternally benign, or just dissolve.
+ Bizarrely, the endgame utopias of even the most ardent high-tech libertarians
+ always seem to take socialist turns. The joys of life will be too cheap to
+ meter, we imagine. So abundance will go ambient.
+ This is what diverse cyber-enlightened business concerns and political groups
+ all share in common, from Facebook to WikiLeaks. Eventually, they imagine,
+ there will be no more secrets, no more barriers to access; all the world will
+ be opened up as if the planet were transformed into a crystal ball. In the
+ meantime, those true believers encrypt their servers even as they seek to
+ gather the rest of the world’s information and find the best way to leverage
+ it.
+ It is all too easy to forget that “free” inevitably means that someone else
+ will be deciding how you live.
+## Just Blurt the Idea Out
+ So we begin with the simple question of how to design digital networks to
+ deliver more help than harm in aligning human intention to meet great
+ challenges. A starting point for an answer can be summarized: “Digital
+ information is really just people in disguise.”
+### Aristotle frets
+ Aristotle directly addressed the role of people in a hypothetical high-tech
+ world: If every instrument could accomplish its own work, obeying or
+ anticipating the will of others, like the statues of Daedalus, or the tripods
+ of Hephaestus, which, says the poet, of their own accord entered the assembly
+ of the Gods; if, in like manner, the shuttle would weave and the plectrum touch
+ the lyre without a hand to guide them, chief workmen would not want servants,
+ nor masters slaves.1
+ At this ancient date, a number of possibilities were at least slightly visible
+ to Aristotle’s imagination. One was that the human condition was in part a
+ function of what machines could not do. Another was that it was possible to
+ imagine, at least hypothetically, that machines could do more. The synthesis
+ was also conceived: Better machines could free and elevate people, even slaves.
+ If we could show Aristotle the technology of our times, I wonder what he would
+ make of the problem of unemployment. Would he take Marx’s position that better
+ machines create an obligation (to be carried out by political bodies) to
+ provide care and dignity to people who no longer need to work? Or would
+ Aristotle say, “Kick the unneeded ones out of town. The polis is only for the
+ people who own the machines, or do what machines still cannot do.” Would he
+ stand by idly as Athens was eventually depopulated?
+ I’d like to think the best of Aristotle, and assume he would realize that both
+ choices are bogus; machine autonomy is nothing but theater. Information needn’t
+ be thought of as a freestanding thing, but rather as a human product. It is
+ entirely legitimate to understand that people are still needed and valuable
+ even when the loom can run without human muscle power. It is still running on
+ human thought.
+ [...]
+ Note: How prescient that Aristotle chose musical instruments and looms as his
+ examples for machines that might one day operate automatically! These two types
+ of machines did indeed turn out to be central to the prehistory of computation.
+ The Jacquard programmable loom helped inspire calculating engines, while music
+ theory and notation helped further the concept of abstract computation, as when
+ Mozart wrote algorithmic, nondeterministic music incorporating dice throws.
+ Both developments occurred around the turn of the 19th century.
+ [...]
+ Aristotle seems to want to escape the burden of accommodating lesser people.
+ His quote about self-operating lutes and looms could be interpreted as a
+ daydream that better technology will free us to some degree from having to deal
+ with one another.
+ It’s not as if everyone wanted to be closer to all of humanity when cities
+ first formed. Athens was a necessity first, and a luxury second. No one wants
+ to accommodate the diversity of strangers. People deal with each other
+ politically because the material advantages are compelling. We find relative
+ safety and sustenance in numbers. Agriculture and armies happened to work
+ better as those enterprises got bigger, and cities built walls.
+ But in Aristotle’s words you get a taste of what a nuisance it can be to
+ accommodate others. Something was lost with the advent of the polis, and we
+ still dream of getting it back.
+ [...]
+ The reward for a Roman general, upon retiring after years of combat, was a plot
+ of land he could farm for himself. To be left alone, to be able to live off the
+ land with the illusion of no polis to bug you, that was the dream. The American
+ West offered that dream again, and still loathes giving it up. Justice Louis
+ Brandeis famously defined privacy as the “right to be left alone.”
+ In every case, however, abundance without politics was an illusion that could
+ only be sustained in temporary bubbles, supported by armies. The ghosts of the
+ losers haunt every acre of easy abundance. The greatest beneficiaries of
+ civilization use all their power to create a temporary illusion of freedom from
+ politics. The rich live behind gates, not just to protect themselves, but to
+ pretend to not need anyone else, if only for a moment. In Aristotle’s quote, we
+ find the earliest glimmer of the hope that technological advancement could
+ replace territorial conquest as a way of implementing an insulating bubble
+ around a person.
+ [...]
+ People naturally seek the benefits of society, meaning the accommodation of
+ strangers, while avoiding direct vulnerabilities to specific others as much as
+ possible. This is a clichéd criticism of the online culture of the moment.
+ People have thousands of “friends” and yet stare at a little screen when in the
+ proximity of other people. As it was in Athens, so it is online.
+## Money
+ Money might have begun as a mnemonic counter for assets you couldn’t keep under
+ direct observation, like wandering sheep. A stone per sheep, so the shepherd
+ would be confident all had been reunited after a day at pasture. In other
+ words, artifacts took on information storage duties.
+ [...]
+ Ancient money was information storage that represented events in the past. To
+ the ears of many a financier, at this early stage “money” had not been born
+ yet, only accounting. That kind of money can be called “past-oriented money.”
+## Noise and luck
+ Consider the problem of noise, or what is known as luck in human affairs.
+ [...]
+ And yet the rewards of winning and losing are vastly different. While some
+ critics might have aesthetic or ethical objections to winner-take-all outcomes,
+ a mathematical problem with them is that noise is amplified. Therefore, if a
+ societal system depends too much on winner-take-all contests, then the acuity
+ of that system will suffer. It will become less reality-based.
+ When a bell curve distribution is appreciated as a bell curve instead of as a
+ winner-take-all distribution, then noise, luck, and conceptual ambiguity aren’t
+ amplified. It makes statistical sense to talk about average intelligence or
+ high intelligence, but not to identify the single most intelligent person.
+## Letting Bell Curves Be Bell Curves
+ In a star system, the top players are rewarded tremendously, while almost
+ everyone else—facing in our era an ever-larger, more global body of competitive
+ peers—is driven toward poverty (because of competition or perhaps automation).
+## Absolutism
+ Being an absolutist is a certain way to become a failed technologist.
+ Markets are an information technology. A technology is useless if it can’t be
+ tweaked. If market technology can’t be fully automatic and needs some
+ “buttons,” then there’s no use in trying to pretend otherwise. You don’t stay
+ attached to poorly performing quests for perfection. You fix bugs.
+## The Taste of Politics
+ Despite my favorable regard for organized labor, for the purposes of this book
+ I have to focus somewhat on certain failings. The problems of interest to me
+ are not really with the labor movement, but with the nature of levees. What
+ might be called “upper-class levees,” like exclusive investment funds, have
+ been known to blur into Ponzi schemes or other criminal enterprises, and the
+ same pattern exists for levees at all levels.
+ Levees are more human than algorithmic, and that is not an entirely good thing.
+ Whether for the rich or the middle class, levees are inevitably a little
+ conspiratorial, and conspiracy naturally attracts corruption. Criminals easily
+ exploited certain classic middle-class levees; the mob famously infiltrated
+ unions and repurposed music royalties as a money-laundering scheme.
+ Levees are a rejection of unbridled algorithm and an insertion of human will
+ into the flow of capital. Inevitably, human oversight brings with it all the
+ flaws of humans. And yet despite their rough and troubled nature, antenimbosian
+ levees worked well enough to preserve middle classes despite the floods,
+ storms, twisters, and droughts of a world contoured by finance. Without our
+ system of levees, rising like a glimmering bell-curved mountain of rice
+ paddies, capitalism would probably have decayed into Marx’s “attractor
+ nightmare” in which markets decay into plutocracy.
+## A First Pass at a Definition
+ A Siren Server, as I will refer to such a thing, is an elite computer,
+ or coordinated collection of computers, on a network. It is
+ characterized by narcissism, hyperamplified risk aversion, and extreme
+ information asymmetry. It is the winner of an all-or-nothing contest,
+ and it inflicts smaller all-or-nothing contests on those who interact
+ with it.
+ Siren Servers gather data from the network, often without having to pay
+ for it. The data is analyzed using the most powerful available
+ computers, run by the very best available technical people. The results
+ of the analysis are kept secret, but are used to manipulate the rest of
+ the world to advantage.
+ That plan will always eventually backfire, because the rest of the world
+ cannot indefinitely absorb the increased risk, cost, and waste dispersed
+ by a Siren Server. Homer sternly warned sailors to not succumb to the
+ call of the sirens, and yet was entirely complacent about Hephaestus’s
+ golden female robots. But Sirens might be even more dangerous in
+ inorganic form, because it is then that we are really most looking at
+ ourselves in disguise. It is not the siren who harms the sailor, but the
+ sailor’s inability to think straight. So it is with us and our machines.
+ Siren Servers are fated by their nature to sow illusions. They are
+ cousins to another seductive literary creature, star of the famous
+ thought experiment known as Maxwell’s Demon, after the great 19th
+ century physicist James Clerk Maxwell. The demon is an imaginary
+ creature that, if it could only exist, would be able to implement a
+ perpetual motion machine and perform other supernatural tricks.
+ Maxwell’s Demon might be stationed at a tiny door separating two
+ chambers filled with water or air. It would only allow hot molecules to
+ pass one way, and cold molecules to pass in the opposite direction.
+ After a while, one side would be hot and the other cold, and you could
+ let them mix again, rushing together so quickly that the stream could
+ run a generator. In that way, the tiny act of discriminating between hot
+ and cold would produce infinite energy, because you could repeat the
+ process forever.
+ The reason Maxwell’s Demon cannot exist is that it does take resources
+ to perform an act of discrimination. We imagine computation is free, but
+ it never is. The very act of choosing which particle is cold or hot
+ itself becomes an energy drain and a source of waste heat. The principle
+ is also known as “no free lunch.”
+ We do our best to implement Maxwell’s Demon whenever we manipulate
+ reality with our technologies, but we can never do so perfectly; we
+ certainly can’t get ahead of the game, which is known as entropy. All
+ the air conditioners in a city emit heat that makes the city hotter
+ overall. While you can implement what seems to be a Maxwell’s Demon if
+ you don’t look too far or too closely, in the big picture you always
+ lose more than you gain.
+ Every bit in a computer is a wannabe Maxwell’s Demon, separating the
+ state of “one” from the state of “zero” for a while, at a cost. A
+ computer on a network can also act like a wannabe demon if it tries to
+ sort data from networked people into one or the other side of some
+ imaginary door, while pretending there is no cost or risk involved. For
+ instance, a Siren Server might allow only those who would be cheap to
+ insure through a doorway (to become insured) in order to make a
+ supernaturally ideal, low-risk insurance company. Such a scheme would
+ let high-risk people pass one way, and low-risk ones pass the other way,
+ in order to implement a phony perpetual motion machine out of a human
+ society. However, the uninsured would not cease to exist; rather, they
+ would instead add to the cost of the whole system, which includes the
+ people who run the Siren Server. A short-term illusion of risk reduction
+ would actually lead to increased risk in the longer term.
+## Candy
+ The primary business of digital networking has come to be the creation of
+ ultrasecret mega-dossiers about what others are doing, and using this
+ information to concentrate money and power. It doesn’t matter whether the
+ concentration is called a social network, an insurance company, a derivatives
+ fund, a search engine, or an online store. It’s all fundamentally the same.
+ Whatever the intent might have been, the result is a wielding of digital
+ technology against the future of the middle class.
+ [...]
+ We loved the crazy cheap easy mortgages, motivated by crazed overleveraging. We
+ love the free music, enabled by crazed copying. We love cheap online prices,
+ offered by what would have once seemed like national intelligence agencies.
+ These newer spy services do not struggle on behalf of our security, but instead
+ figure out just how little payment everyone in the chain can be made to accept.
+ We are not benefiting from the benevolence of some artificial intelligence
+ superbeing. We are exploiting each other off the books while those
+ concentrating our information remain on the books. We love our treats but will
+ eventually discover we are depleting our own value.
+ That’s how we can have economic troubles despite there being so much wealth in
+ the system, and during a period of increasing efficiencies. Great fortunes are
+ being made on shrinking the economy instead of growing it. It’s not a result of
+ some evil scheme, but a side effect of an idiotic elevation of the fantasy that
+ technology is getting smart and standing on its own, without people.
+## From Autocollate to Autocollude
+ It seems as though online services are bringing bargains to everyone, and yet
+ wealth disparity is increasing while social mobility is decreasing. If everyone
+ were getting better options, wouldn’t everyone be doing better as well?
+## From the Customer’s Point of View
+ Wal-Mart confronted the ordinary shopper with two interesting pieces of news.
+ One was that stuff they wanted to buy got cheaper, which of course was great.
+ This news was delivered first, and caused cheering.
+ But there was another piece of news that emerged more gradually. It has often
+ been claimed that Wal-Mart plays a role in the reduction of employment
+ prospects for the very people who tend to be its customers.1 Wal-Mart has
+ certainly made the world more efficient in a certain sense. It moved
+ manufacturing to any spot in the world that could accomplish it at the very
+ lowest cost; it rewarded vendors willing to cut corners to the maximum degree.
+ [...]
+ All Siren Servers deliver dual messages similar to the pair pioneered by
+ Wal-Mart. On the one hand, “Good news! Treats await! Information systems have
+ made the world more efficient for you.”
+ On the other hand, a little later: “It turns out you, your needs, and your
+ expectations are not maximally efficient from the lofty point of view of our
+ server. Therefore, we are reshaping the world so that in the long term, your
+ prospects are being reduced.”
+ The initial benefits don’t remotely balance the long-term degradations.
+ Initially you made some money day trading or getting an insanely easy loan, or
+ saved some money couch-surfing or by using coupons from an Internet site, but
+ then came the pink slip, the eviction notice, and the halving of your savings
+ when the market drooped. Or you loved getting music for free, but then realized
+ that you couldn’t pursue a music career yourself because there were hardly any
+ middle-class, secure jobs left in what was once the music industry. Maybe you
+ loved the supercheap prices at your favorite store, but then noticed that the
+ factory you might have worked for closed up for good.
+## Financial Siren Servers
+ The schemes were remarkably similar to Silicon Valley designs. A few of them
+ took as input everything they possibly could scrape from the Internet as well
+ as other, proprietary networks. As in Google’s data centers, stupendous
+ correlative algorithms would crunch on the whole ’net’s data overnight, looking
+ for correlations. Maybe a sudden increase in comments about mosquito bites
+ would cause an automatic, instant investment in a company that sold lotions.
+ Actually, that’s an artificially sensible example. The real examples made no
+ sense to humans. But money was made, and fairly reliably.
+ Note: It should be pointed out that if only one Siren Server is milking a
+ particular fluctuation in this way, a reasonable argument could be made that a
+ service is being performed, in that the fluctuation reveals inefficiency, and
+ the Siren is canceling it out. However, when many Sirens milk the same
+ fluctuation, they lock into a feedback system with each other and inadvertently
+ conspire to milk the rest of the world to no purpose.
+ [...]
+ What is absolutely essential to a financial Siren Server, however, is a
+ superior information position. If everyone else knew what you were doing, they
+ could securitize you. If anyone could buy stock in a mathematical “sure thing”
+ scheme, then the benefits of it would be copied like a shared music file, and
+ spread out until it was nullified. So, in today’s world your mortgage can be
+ securitized in someone else’s secretive bunker, but you can’t know about the
+ bunker and securitize it. If it weren’t for that differential, the new kind of
+ sure thing wouldn’t exist.
+## If Life Gives You EULAs, Make Lemonade
+ The information economy that we are currently building doesn’t really embrace
+ capitalism, but rather a new form of feudalism.
+## Your Lack of Privacy Is Someone Else’s Wealth
+ Occasionally the rich embrace a new token and drive up its value. The fine art
+ market is a great example. Expensive art is essentially a private form of
+ currency traded among the very rich. The better an artist is at making art that
+ can function this way, the more valuable the art will become. Andy Warhol is
+ often associated with this trick, though Pablo Picasso and others were
+ certainly playing the same game earlier. The art has to be stylistically
+ distinct and available in suitable small runs. It becomes a private form of
+ money, as instantly recognizable as a hundred-dollar bill.
+ A related trend of our times is that troves of dossiers on the private lives
+ and inner beings of ordinary people, collected over digital networks, are
+ packaged into a new private form of elite money. The actual data in these
+ troves need not be valid. In fact, it might be better that it is not valid, for
+ actual knowledge brings liabilities.
+## The Nature of Our Confusion
+ Our core illusion is that we imagine big data as a substance, like a natural
+ resource waiting to be mined. We use terms like data-mining routinely to
+ reinforce that illusion. Indeed some data is like that. Scientific big data,
+ like data about galaxy formation, weather, or flu outbreaks, can be gathered
+ and mined, just like gold, provided you put in the hard work.
+ But big data about people is different. It doesn’t sit there; it plays against
+ you. It isn’t like a view through a microscope, but more like a view of a
+ chessboard.
+## The Most Elite Naïveté
+ As technology advances, Siren Servers will be ever more the objects of the
+ struggle for wealth and power, because they are the only links in the chain
+ that will not be commoditized. If present trends continue, you’ll always be
+ able to seek information supremacy, just as old-fashioned barons could struggle
+ for supremacy over land or natural resources. A new energy cycle will someday
+ make oil much less central to geopolitics, but the information system that
+ manages that new kind of energy could easily become an impregnable castle. The
+ illusory golden vase becomes more and more valuable.
+### Mapping out where the conversation can go
+ An endgame for civilization has been foreseen since Aristotle. As technology
+ reaches heights of efficiency, civilization will have to find a way to resolve
+ a peculiar puzzle: What should the role of “extra” humans be if not everyone is
+ still strictly needed? Do the extra people—the ones whose roles have
+ withered—starve? Or get easy lives? Who decides? How?
+ The same core questions, stated in a multitude of ways, have elicited only a
+ small number of answers, because only a few are possible.
+ What will people be when technology becomes much more advanced? With each
+ passing year our abilities to act on our ideas are increased by technological
+ progress. Ideas matter more and more. The ancient conversations about where
+ human purpose is headed continue today, with rising implications.
+ Suppose that machines eventually gain sufficient functionality that one will be
+ able to say that a lot of people have become extraneous. This might take place
+ in nursing, pharmaceuticals, transportation, manufacturing, or in any other
+ imaginable field of employment.
+ The right question to then ask isn’t really about what should be done with the
+ people who used to perform the tasks now colonized by machines. By the time one
+ gets to that question, a conceptual mistake has already been made.
+ Instead, it has to be pointed out that outside of the spell of bad philosophy
+ human obsolescence wouldn’t in fact happen. The data that drives “automation”
+ has to ultimately come from people, in the form of “big data.” Automation can
+ always be understood as elaborate puppetry.
+ The most crucial quality of our response to very high-functioning machines,
+ artificial intelligences and the like, is how we conceive of the things that
+ the machines can’t do, and whether those tasks are considered real jobs for
+ people or not. We used to imagine that elite engineers would be automation’s
+ only puppeteers. It turns out instead that big data coming from vast numbers of
+ people is needed to make machines appear to be “automated.” Do the puppeteers
+ still get paid once the whole audience has joined their ranks?
+## The Technology of Ambient Cheating
+ Siren Servers do what comes naturally due to the very idea of computation.
+ Computation is the demarcation of a little part of the universe, called a
+ computer, which is engineered to be very well understood and controllable, so
+ that it closely approximates a deterministic, non-entropic process. But in
+ order for a computer to run, the surrounding parts of the universe must take on
+ the waste heat, the randomness. You can create a local shield against entropy,
+ but your neighbors will always pay for it.
+ Note: A rare experimental machine called a “reversible” computer never forgets,
+ so that any computation can be run backward as well as forward. Such devices
+ run cool! This is an example of how thermodynamics and computation interact.
+ Reversible computers don’t radiate as much heat; forgetting radiates
+ randomness, which is the same thing as heating up the neighborhood.
+## The Insanity of the Local/Global Flip
+ A Siren Server can become so successful—sometimes in the blink of an eye—that
+ it optimizes its environment—changes it—instead of changing in order to adapt
+ to the environment. A successful Siren Server no longer acts only as a player
+ within a larger system. Instead it becomes a central planner. This makes it
+ stupid, like a central planner in a communist regime.
+## The Conservation of Free Will
+ A story must have actors, not automatons. Different people become more or less
+ like automatons in our Sirenic era.
+ Sirenic entrepreneurs intuitively cast free will—so long as it is their own—as
+ an ever more magical, elite, and “meta” quality of personhood. The entrepreneur
+ hopes to “dent the universe”* or achieve some other heroic, Nietzschean
+ validation. Ordinary people, however, who will be attached to the nodes of the
+ network created by the hero, will become more effectively mechanical.
+ [...]
+ We’re setting up barriers between cases where we choose to give over some
+ judgment to cloud software, as if we were predictable machines, and those where
+ we elevate our judgments to pious, absolute standards.
+ Making choices of where to place the barrier between ego and algorithm is
+ unavoidable in the age of cloud software. Drawing the line between what we
+ forfeit to calculation and what we reserve for the heroics of free will is the
+ story of our time.
+## Rewarding and Punishing Network Effects
+ To understand how Siren Servers work, it’s useful to divide network effects
+ into those that are “rewarding” and those that are “punishing.” Siren Servers
+ gain dominance through rewarding network effects, but keep dominance through
+ punishing network effects.
+## The Closing Act
+ Competition becomes mostly about who can out-meta who, and only secondarily
+ about specialization.
+ [...]
+ Individual Siren Servers can die and yet the Siren Server pattern perseveres,
+ and it is that pattern that is the real problem. The systematic decoupling of
+ risk from reward in the rising information economy is the problem, not any
+ particular server.
+## The limits of emergence as an explanation
+ But the problem with freestanding concentrations of power is that you never
+ know who will inherit them. If social networking has the power to synchronize
+ great crowds to dethrone a pharaoh, why might it not also coordinate lynchings
+ or pogroms?
+ [...]
+ The core ideal of the Internet is that one trusts people, and that given an
+ opportunity, people will find their way to be reasonably decent. I happily
+ restate my loyalty to that ideal. It’s all we have.
+ But the demonstrated capability of Facebook to effortlessly engage in mass
+ social engineering proves that the Internet as it exists today is not a
+ purists’ emergent system, as is so often claimed, but largely a top-down,
+ directed one.
+ [...]
+ We pretend that an emergent meta-human being is appearing in the computing
+ clouds—an artificial intelligence—but actually it is humans, the operators of
+ Siren Servers, pulling the levers.
+ [...]
+ The nuts and bolts of artificial-intelligence research can often be more
+ usefully interpreted without the concept of AI at all. For example, in 2011,
+ IBM scientists unveiled a “question answering” machine that is designed to play
+ the TV quiz show Jeopardy. Suppose IBM had dispensed with the theatrics, and
+ declared it had done Google one better and come up with a new phrase-based
+ search engine. This framing of exactly the same technology would have gained
+ IBM’s team as much (deserved) recognition as the claim of an artificial
+ intelligence, but it would also have educated the public about how such a
+ technology might actually be used most effectively.
+ AI technologies typically operate on a variation of the process described
+ earlier that accomplishes translations between languages. While innovation in
+ algorithms is vital, it is just as vital to feed algorithms with “big data”
+ gathered from ordinary people. The supposedly artificially intelligent result
+ can be understood as a mash-up of what real people did before. People have
+ answered a lot of questions before, and a multitude of these answers are
+ gathered up by the algorithms and regurgitated by the program. This in no way
+ denigrates it or proposes it isn’t useful. It is not, however, supernatural.
+ The real people from whom the initial answers were gathered deserve to be paid
+ for each new answer given by the machine.
+ [...]
+ What all this comes down to is that the very idea of artificial intelligence
+ gives us the cover to avoid accountability by pretending that machines can take
+ on more and more human responsibility. This holds for things that we don’t even
+ think of as artificial intelligence, like the recommendations made by Netflix
+ and Pandora. Seeing movies and listening to music suggested to us by algorithms
+ is relatively harmless, I suppose. But I hope that once in a while the users of
+ those services resist the recommendations; our exposure to art shouldn’t be
+ hemmed in by an algorithm that we merely want to believe predicts our tastes
+ accurately. These algorithms do not represent emotion or meaning, only
+ statistics and correlations.
+ What makes this doubly confounding is that while Silicon Valley might sell
+ artificial intelligence to consumers, our industry certainly wouldn’t apply the
+ same automated techniques to some of its own work. Choosing design features in
+ a new smartphone, say, is considered too consequential a game. Engineers don’t
+ seem quite ready to believe in their smart algorithms enough to put them up
+ against Apple’s late chief executive, Steve Jobs, or some other person with a
+ real design sensibility.
+ But the rest of us, lulled by the concept of ever-more intelligent AIs, are
+ expected to trust algorithms to assess our aesthetic choices, the progress of a
+ student, the credit risk of a homeowner or an institution. In doing so, we only
+ end up misreading the capability of our machines and distorting our own
+ capabilities as human beings. We must instead take responsibility for every
+ task undertaken by a machine and double-check every conclusion offered by an
+ algorithm, just as we always look both ways when crossing an intersection, even
+ though the signal has been given to walk.
+ When we think of computers as inert, passive tools instead of people, we are
+ rewarded with a clearer, less ideological view of what is going on—with the
+ machines and with ourselves. So, why, aside from the theatrical appeal to
+ consumers and reporters, must engineering results so often be presented in
+ Frankensteinian light?
+ The answer is simply that computer scientists are human, and are as terrified
+ by the human condition as anyone else. We, the technical elite, seek some way
+ of thinking that gives us an answer to death, for instance. This helps explain
+ the allure of a place like the Singularity University. The influential Silicon
+ Valley institution preaches a story that goes like this: One day in the
+ not-so-distant future, the Internet will suddenly coalesce into a
+ superintelligent AI, infinitely smarter than any of us individually and all of
+ us combined; it will become alive in the blink of an eye, and take over the
+ world before humans even realize what’s happening.
+ Some think the newly sentient Internet would then choose to kill us; others
+ think it would be generous and digitize us the way Google is digitizing old
+ books, so that we can live forever as algorithms inside the global brain. Yes,
+ this sounds like many different science fiction movies. Yes, it sounds nutty
+ when stated so bluntly. But these are ideas with tremendous currency in Silicon
+ Valley; these are guiding principles, not just amusements, for many of the most
+ influential technologists.
+ It should go without saying that we can’t count on the appearance of a
+ soul-detecting sensor that will verify that a person’s consciousness has been
+ virtualized and immortalized. There is certainly no such sensor with us today
+ to confirm metaphysical ideas about people. All thoughts about consciousness,
+ souls, and the like are bound up equally in faith, which suggests something
+ remarkable: What we are seeing is a new religion, expressed through an
+ engineering culture.