aboutsummaryrefslogtreecommitdiff
path: root/books/sociedade/who-owns-the-future.md
diff options
context:
space:
mode:
authorSilvio Rhatto <rhatto@riseup.net>2018-08-07 10:05:58 -0300
committerSilvio Rhatto <rhatto@riseup.net>2018-08-07 10:05:58 -0300
commitb6c0ffcaf707ee1968a7f29021d20357692a84d0 (patch)
treebb1ec89dce7ca4072fdfa794e3492b71ef93ff94 /books/sociedade/who-owns-the-future.md
parent180d1f6ebc346308df7f84150604a100f18118f9 (diff)
downloadblog-b6c0ffcaf707ee1968a7f29021d20357692a84d0.tar.gz
blog-b6c0ffcaf707ee1968a7f29021d20357692a84d0.tar.bz2
Reorganization
Diffstat (limited to 'books/sociedade/who-owns-the-future.md')
-rw-r--r--books/sociedade/who-owns-the-future.md673
1 files changed, 0 insertions, 673 deletions
diff --git a/books/sociedade/who-owns-the-future.md b/books/sociedade/who-owns-the-future.md
deleted file mode 100644
index e53d275..0000000
--- a/books/sociedade/who-owns-the-future.md
+++ /dev/null
@@ -1,673 +0,0 @@
-[[!meta title="Who owns the future?"]]
-
-* Author: Jaron Lanier
-* Year: 2013
-* Publisher: Simon & Schuster
-
-## Index
-
-* Star system versus the bell curve as network designs.
-* Siren Servers: narcissism, hyperamplified risk aversion, and extreme information asymmetry.
-* Siren Servers and Maxwell’s Demon.
-* Disruptive innovation as the tedious scheme to shrink markets.
-* Science isn't automatic.
-* Nine dismal humors of futurism, and a hopeful one.
-* Marx as one of the first technology writers (when discussing Luddites).
-* Human obsolescence is avoidable.
-* Keynes Considered as a Big Data Pioneer.
-* Amazon's Mechanical Turk.
-* Humanistic information economics.
-* What is experience? If personal experience were missing from the universe, how would things be different?
-* Gurus and New Age at the Sillicon Valley: Gurdjieff, Steve Jobs.
-
-## Prelude
-
- Instagram isn’t worth a billion dollars just because those thirteen employees
- are extraordinary. Instead, its value comes from the millions of users who
- contribute to the network without being paid for it. Networks need a great
- number of people to participate in them to generate significant value. But when
- they have them, only a small number of people get paid. That has the net effect
- of centralizing wealth and limiting overall economic growth.
-
- [...]
-
- By “digital networking” I mean not only the Internet and the Web, but also
- other networks operated by outfits like financial institutions and intelligence
- agencies. In all these cases, we see the phenomenon of power and money becoming
- concentrated around the people who operate the most central computers in a
- network, undervaluing everyone else. That is the pattern we have come to
- expect, but it is not the only way things can go.
-
-## The Price of Heaven
-
- Utopians presume the advent of abundance not because it will be affordable, but
- because it will be free, provided we accept surveillance.
-
- Starting back in the early 1980s, an initially tiny stratum of gifted
- technologists conceived new interpretations of concepts like privacy, liberty,
- and power. I was an early participant in the process and helped to formulate
- many of the ideas I am criticizing in this book. What was once a tiny
- subculture has blossomed into the dominant interpretation of computation and
- software-mediated society.
-
- One strain of what might be called “hacker culture” held that liberty means
- absolute privacy through the use of cryptography. I remember the thrill of
- using military-grade stealth just to argue about who should pay for a pizza at
- MIT in 1983 or so.
-
- On the other hand, some of my friends from that era, who consumed that pizza,
- eventually became very rich building giant cross-referenced dossiers on masses
- of people, which were put to use by financiers, advertisers, insurers, or other
- concerns nurturing fantasies of operating the world by remote control.
-
- It is typical of human nature to ignore hypocrisy. The greater a hypocrisy, the
- more invisible it typically becomes, but we technical folk are inclined to seek
- an airtight whole of ideas. Here is one such synthesis—of cryptography for
- techies and massive spying on others—which I continue to hear fairly often:
- Privacy for ordinary people can be forfeited in the near term because it will
- become moot anyway.
-
- Surveillance by the technical few on the less technical many can be tolerated
- for now because of hopes for an endgame in which everything will become
- transparent to everyone. Network entrepreneurs and cyber-activists alike seem
- to imagine that today’s elite network servers in positions of information
- supremacy will eventually become eternally benign, or just dissolve.
-
- Bizarrely, the endgame utopias of even the most ardent high-tech libertarians
- always seem to take socialist turns. The joys of life will be too cheap to
- meter, we imagine. So abundance will go ambient.
-
- This is what diverse cyber-enlightened business concerns and political groups
- all share in common, from Facebook to WikiLeaks. Eventually, they imagine,
- there will be no more secrets, no more barriers to access; all the world will
- be opened up as if the planet were transformed into a crystal ball. In the
- meantime, those true believers encrypt their servers even as they seek to
- gather the rest of the world’s information and find the best way to leverage
- it.
-
- It is all too easy to forget that “free” inevitably means that someone else
- will be deciding how you live.
-
-## Just Blurt the Idea Out
-
- So we begin with the simple question of how to design digital networks to
- deliver more help than harm in aligning human intention to meet great
- challenges. A starting point for an answer can be summarized: “Digital
- information is really just people in disguise.”
-
-### Aristotle frets
-
- Aristotle directly addressed the role of people in a hypothetical high-tech
- world: If every instrument could accomplish its own work, obeying or
- anticipating the will of others, like the statues of Daedalus, or the tripods
- of Hephaestus, which, says the poet, of their own accord entered the assembly
- of the Gods; if, in like manner, the shuttle would weave and the plectrum touch
- the lyre without a hand to guide them, chief workmen would not want servants,
- nor masters slaves.1
-
- At this ancient date, a number of possibilities were at least slightly visible
- to Aristotle’s imagination. One was that the human condition was in part a
- function of what machines could not do. Another was that it was possible to
- imagine, at least hypothetically, that machines could do more. The synthesis
- was also conceived: Better machines could free and elevate people, even slaves.
-
- If we could show Aristotle the technology of our times, I wonder what he would
- make of the problem of unemployment. Would he take Marx’s position that better
- machines create an obligation (to be carried out by political bodies) to
- provide care and dignity to people who no longer need to work? Or would
- Aristotle say, “Kick the unneeded ones out of town. The polis is only for the
- people who own the machines, or do what machines still cannot do.” Would he
- stand by idly as Athens was eventually depopulated?
-
- I’d like to think the best of Aristotle, and assume he would realize that both
- choices are bogus; machine autonomy is nothing but theater. Information needn’t
- be thought of as a freestanding thing, but rather as a human product. It is
- entirely legitimate to understand that people are still needed and valuable
- even when the loom can run without human muscle power. It is still running on
- human thought.
-
- [...]
-
- Note: How prescient that Aristotle chose musical instruments and looms as his
- examples for machines that might one day operate automatically! These two types
- of machines did indeed turn out to be central to the prehistory of computation.
- The Jacquard programmable loom helped inspire calculating engines, while music
- theory and notation helped further the concept of abstract computation, as when
- Mozart wrote algorithmic, nondeterministic music incorporating dice throws.
- Both developments occurred around the turn of the 19th century.
-
- [...]
-
- Aristotle seems to want to escape the burden of accommodating lesser people.
- His quote about self-operating lutes and looms could be interpreted as a
- daydream that better technology will free us to some degree from having to deal
- with one another.
-
- It’s not as if everyone wanted to be closer to all of humanity when cities
- first formed. Athens was a necessity first, and a luxury second. No one wants
- to accommodate the diversity of strangers. People deal with each other
- politically because the material advantages are compelling. We find relative
- safety and sustenance in numbers. Agriculture and armies happened to work
- better as those enterprises got bigger, and cities built walls.
-
- But in Aristotle’s words you get a taste of what a nuisance it can be to
- accommodate others. Something was lost with the advent of the polis, and we
- still dream of getting it back.
-
- [...]
-
- The reward for a Roman general, upon retiring after years of combat, was a plot
- of land he could farm for himself. To be left alone, to be able to live off the
- land with the illusion of no polis to bug you, that was the dream. The American
- West offered that dream again, and still loathes giving it up. Justice Louis
- Brandeis famously defined privacy as the “right to be left alone.”
-
- In every case, however, abundance without politics was an illusion that could
- only be sustained in temporary bubbles, supported by armies. The ghosts of the
- losers haunt every acre of easy abundance. The greatest beneficiaries of
- civilization use all their power to create a temporary illusion of freedom from
- politics. The rich live behind gates, not just to protect themselves, but to
- pretend to not need anyone else, if only for a moment. In Aristotle’s quote, we
- find the earliest glimmer of the hope that technological advancement could
- replace territorial conquest as a way of implementing an insulating bubble
- around a person.
-
- [...]
-
- People naturally seek the benefits of society, meaning the accommodation of
- strangers, while avoiding direct vulnerabilities to specific others as much as
- possible. This is a clichéd criticism of the online culture of the moment.
- People have thousands of “friends” and yet stare at a little screen when in the
- proximity of other people. As it was in Athens, so it is online.
-
-## Money
-
- Money might have begun as a mnemonic counter for assets you couldn’t keep under
- direct observation, like wandering sheep. A stone per sheep, so the shepherd
- would be confident all had been reunited after a day at pasture. In other
- words, artifacts took on information storage duties.
-
- [...]
-
- Ancient money was information storage that represented events in the past. To
- the ears of many a financier, at this early stage “money” had not been born
- yet, only accounting. That kind of money can be called “past-oriented money.”
-
-## Noise and luck
-
- Consider the problem of noise, or what is known as luck in human affairs.
-
- [...]
-
- And yet the rewards of winning and losing are vastly different. While some
- critics might have aesthetic or ethical objections to winner-take-all outcomes,
- a mathematical problem with them is that noise is amplified. Therefore, if a
- societal system depends too much on winner-take-all contests, then the acuity
- of that system will suffer. It will become less reality-based.
-
- When a bell curve distribution is appreciated as a bell curve instead of as a
- winner-take-all distribution, then noise, luck, and conceptual ambiguity aren’t
- amplified. It makes statistical sense to talk about average intelligence or
- high intelligence, but not to identify the single most intelligent person.
-
-## Letting Bell Curves Be Bell Curves
-
- In a star system, the top players are rewarded tremendously, while almost
- everyone else—facing in our era an ever-larger, more global body of competitive
- peers—is driven toward poverty (because of competition or perhaps automation).
-
-## Absolutism
-
- Being an absolutist is a certain way to become a failed technologist.
-
- Markets are an information technology. A technology is useless if it can’t be
- tweaked. If market technology can’t be fully automatic and needs some
- “buttons,” then there’s no use in trying to pretend otherwise. You don’t stay
- attached to poorly performing quests for perfection. You fix bugs.
-
-## The Taste of Politics
-
- Despite my favorable regard for organized labor, for the purposes of this book
- I have to focus somewhat on certain failings. The problems of interest to me
- are not really with the labor movement, but with the nature of levees. What
- might be called “upper-class levees,” like exclusive investment funds, have
- been known to blur into Ponzi schemes or other criminal enterprises, and the
- same pattern exists for levees at all levels.
-
- Levees are more human than algorithmic, and that is not an entirely good thing.
- Whether for the rich or the middle class, levees are inevitably a little
- conspiratorial, and conspiracy naturally attracts corruption. Criminals easily
- exploited certain classic middle-class levees; the mob famously infiltrated
- unions and repurposed music royalties as a money-laundering scheme.
-
- Levees are a rejection of unbridled algorithm and an insertion of human will
- into the flow of capital. Inevitably, human oversight brings with it all the
- flaws of humans. And yet despite their rough and troubled nature, antenimbosian
- levees worked well enough to preserve middle classes despite the floods,
- storms, twisters, and droughts of a world contoured by finance. Without our
- system of levees, rising like a glimmering bell-curved mountain of rice
- paddies, capitalism would probably have decayed into Marx’s “attractor
- nightmare” in which markets decay into plutocracy.
-
-## A First Pass at a Definition
-
- A Siren Server, as I will refer to such a thing, is an elite computer,
- or coordinated collection of computers, on a network. It is
- characterized by narcissism, hyperamplified risk aversion, and extreme
- information asymmetry. It is the winner of an all-or-nothing contest,
- and it inflicts smaller all-or-nothing contests on those who interact
- with it.
-
- Siren Servers gather data from the network, often without having to pay
- for it. The data is analyzed using the most powerful available
- computers, run by the very best available technical people. The results
- of the analysis are kept secret, but are used to manipulate the rest of
- the world to advantage.
-
- That plan will always eventually backfire, because the rest of the world
- cannot indefinitely absorb the increased risk, cost, and waste dispersed
- by a Siren Server. Homer sternly warned sailors to not succumb to the
- call of the sirens, and yet was entirely complacent about Hephaestus’s
- golden female robots. But Sirens might be even more dangerous in
- inorganic form, because it is then that we are really most looking at
- ourselves in disguise. It is not the siren who harms the sailor, but the
- sailor’s inability to think straight. So it is with us and our machines.
-
- Siren Servers are fated by their nature to sow illusions. They are
- cousins to another seductive literary creature, star of the famous
- thought experiment known as Maxwell’s Demon, after the great 19th
- century physicist James Clerk Maxwell. The demon is an imaginary
- creature that, if it could only exist, would be able to implement a
- perpetual motion machine and perform other supernatural tricks.
-
- Maxwell’s Demon might be stationed at a tiny door separating two
- chambers filled with water or air. It would only allow hot molecules to
- pass one way, and cold molecules to pass in the opposite direction.
- After a while, one side would be hot and the other cold, and you could
- let them mix again, rushing together so quickly that the stream could
- run a generator. In that way, the tiny act of discriminating between hot
- and cold would produce infinite energy, because you could repeat the
- process forever.
-
- The reason Maxwell’s Demon cannot exist is that it does take resources
- to perform an act of discrimination. We imagine computation is free, but
- it never is. The very act of choosing which particle is cold or hot
- itself becomes an energy drain and a source of waste heat. The principle
- is also known as “no free lunch.”
-
- We do our best to implement Maxwell’s Demon whenever we manipulate
- reality with our technologies, but we can never do so perfectly; we
- certainly can’t get ahead of the game, which is known as entropy. All
- the air conditioners in a city emit heat that makes the city hotter
- overall. While you can implement what seems to be a Maxwell’s Demon if
- you don’t look too far or too closely, in the big picture you always
- lose more than you gain.
-
- Every bit in a computer is a wannabe Maxwell’s Demon, separating the
- state of “one” from the state of “zero” for a while, at a cost. A
- computer on a network can also act like a wannabe demon if it tries to
- sort data from networked people into one or the other side of some
- imaginary door, while pretending there is no cost or risk involved. For
- instance, a Siren Server might allow only those who would be cheap to
- insure through a doorway (to become insured) in order to make a
- supernaturally ideal, low-risk insurance company. Such a scheme would
- let high-risk people pass one way, and low-risk ones pass the other way,
- in order to implement a phony perpetual motion machine out of a human
- society. However, the uninsured would not cease to exist; rather, they
- would instead add to the cost of the whole system, which includes the
- people who run the Siren Server. A short-term illusion of risk reduction
- would actually lead to increased risk in the longer term.
-
-## Candy
-
- The primary business of digital networking has come to be the creation of
- ultrasecret mega-dossiers about what others are doing, and using this
- information to concentrate money and power. It doesn’t matter whether the
- concentration is called a social network, an insurance company, a derivatives
- fund, a search engine, or an online store. It’s all fundamentally the same.
- Whatever the intent might have been, the result is a wielding of digital
- technology against the future of the middle class.
-
- [...]
-
- We loved the crazy cheap easy mortgages, motivated by crazed overleveraging. We
- love the free music, enabled by crazed copying. We love cheap online prices,
- offered by what would have once seemed like national intelligence agencies.
- These newer spy services do not struggle on behalf of our security, but instead
- figure out just how little payment everyone in the chain can be made to accept.
- We are not benefiting from the benevolence of some artificial intelligence
- superbeing. We are exploiting each other off the books while those
- concentrating our information remain on the books. We love our treats but will
- eventually discover we are depleting our own value.
-
- That’s how we can have economic troubles despite there being so much wealth in
- the system, and during a period of increasing efficiencies. Great fortunes are
- being made on shrinking the economy instead of growing it. It’s not a result of
- some evil scheme, but a side effect of an idiotic elevation of the fantasy that
- technology is getting smart and standing on its own, without people.
-
-## From Autocollate to Autocollude
-
- It seems as though online services are bringing bargains to everyone, and yet
- wealth disparity is increasing while social mobility is decreasing. If everyone
- were getting better options, wouldn’t everyone be doing better as well?
-
-## From the Customer’s Point of View
-
- Wal-Mart confronted the ordinary shopper with two interesting pieces of news.
- One was that stuff they wanted to buy got cheaper, which of course was great.
- This news was delivered first, and caused cheering.
-
- But there was another piece of news that emerged more gradually. It has often
- been claimed that Wal-Mart plays a role in the reduction of employment
- prospects for the very people who tend to be its customers.1 Wal-Mart has
- certainly made the world more efficient in a certain sense. It moved
- manufacturing to any spot in the world that could accomplish it at the very
- lowest cost; it rewarded vendors willing to cut corners to the maximum degree.
-
- [...]
-
- All Siren Servers deliver dual messages similar to the pair pioneered by
- Wal-Mart. On the one hand, “Good news! Treats await! Information systems have
- made the world more efficient for you.”
-
- On the other hand, a little later: “It turns out you, your needs, and your
- expectations are not maximally efficient from the lofty point of view of our
- server. Therefore, we are reshaping the world so that in the long term, your
- prospects are being reduced.”
-
- The initial benefits don’t remotely balance the long-term degradations.
- Initially you made some money day trading or getting an insanely easy loan, or
- saved some money couch-surfing or by using coupons from an Internet site, but
- then came the pink slip, the eviction notice, and the halving of your savings
- when the market drooped. Or you loved getting music for free, but then realized
- that you couldn’t pursue a music career yourself because there were hardly any
- middle-class, secure jobs left in what was once the music industry. Maybe you
- loved the supercheap prices at your favorite store, but then noticed that the
- factory you might have worked for closed up for good.
-
-## Financial Siren Servers
-
- The schemes were remarkably similar to Silicon Valley designs. A few of them
- took as input everything they possibly could scrape from the Internet as well
- as other, proprietary networks. As in Google’s data centers, stupendous
- correlative algorithms would crunch on the whole ’net’s data overnight, looking
- for correlations. Maybe a sudden increase in comments about mosquito bites
- would cause an automatic, instant investment in a company that sold lotions.
- Actually, that’s an artificially sensible example. The real examples made no
- sense to humans. But money was made, and fairly reliably.
-
- Note: It should be pointed out that if only one Siren Server is milking a
- particular fluctuation in this way, a reasonable argument could be made that a
- service is being performed, in that the fluctuation reveals inefficiency, and
- the Siren is canceling it out. However, when many Sirens milk the same
- fluctuation, they lock into a feedback system with each other and inadvertently
- conspire to milk the rest of the world to no purpose.
-
- [...]
-
- What is absolutely essential to a financial Siren Server, however, is a
- superior information position. If everyone else knew what you were doing, they
- could securitize you. If anyone could buy stock in a mathematical “sure thing”
- scheme, then the benefits of it would be copied like a shared music file, and
- spread out until it was nullified. So, in today’s world your mortgage can be
- securitized in someone else’s secretive bunker, but you can’t know about the
- bunker and securitize it. If it weren’t for that differential, the new kind of
- sure thing wouldn’t exist.
-
-## If Life Gives You EULAs, Make Lemonade
-
- The information economy that we are currently building doesn’t really embrace
- capitalism, but rather a new form of feudalism.
-
-## Your Lack of Privacy Is Someone Else’s Wealth
-
- Occasionally the rich embrace a new token and drive up its value. The fine art
- market is a great example. Expensive art is essentially a private form of
- currency traded among the very rich. The better an artist is at making art that
- can function this way, the more valuable the art will become. Andy Warhol is
- often associated with this trick, though Pablo Picasso and others were
- certainly playing the same game earlier. The art has to be stylistically
- distinct and available in suitable small runs. It becomes a private form of
- money, as instantly recognizable as a hundred-dollar bill.
-
- A related trend of our times is that troves of dossiers on the private lives
- and inner beings of ordinary people, collected over digital networks, are
- packaged into a new private form of elite money. The actual data in these
- troves need not be valid. In fact, it might be better that it is not valid, for
- actual knowledge brings liabilities.
-
-## The Nature of Our Confusion
-
- Our core illusion is that we imagine big data as a substance, like a natural
- resource waiting to be mined. We use terms like data-mining routinely to
- reinforce that illusion. Indeed some data is like that. Scientific big data,
- like data about galaxy formation, weather, or flu outbreaks, can be gathered
- and mined, just like gold, provided you put in the hard work.
-
- But big data about people is different. It doesn’t sit there; it plays against
- you. It isn’t like a view through a microscope, but more like a view of a
- chessboard.
-
-## The Most Elite Naïveté
-
- As technology advances, Siren Servers will be ever more the objects of the
- struggle for wealth and power, because they are the only links in the chain
- that will not be commoditized. If present trends continue, you’ll always be
- able to seek information supremacy, just as old-fashioned barons could struggle
- for supremacy over land or natural resources. A new energy cycle will someday
- make oil much less central to geopolitics, but the information system that
- manages that new kind of energy could easily become an impregnable castle. The
- illusory golden vase becomes more and more valuable.
-
-### Mapping out where the conversation can go
-
- An endgame for civilization has been foreseen since Aristotle. As technology
- reaches heights of efficiency, civilization will have to find a way to resolve
- a peculiar puzzle: What should the role of “extra” humans be if not everyone is
- still strictly needed? Do the extra people—the ones whose roles have
- withered—starve? Or get easy lives? Who decides? How?
-
- The same core questions, stated in a multitude of ways, have elicited only a
- small number of answers, because only a few are possible.
-
- What will people be when technology becomes much more advanced? With each
- passing year our abilities to act on our ideas are increased by technological
- progress. Ideas matter more and more. The ancient conversations about where
- human purpose is headed continue today, with rising implications.
-
- Suppose that machines eventually gain sufficient functionality that one will be
- able to say that a lot of people have become extraneous. This might take place
- in nursing, pharmaceuticals, transportation, manufacturing, or in any other
- imaginable field of employment.
-
- The right question to then ask isn’t really about what should be done with the
- people who used to perform the tasks now colonized by machines. By the time one
- gets to that question, a conceptual mistake has already been made.
-
- Instead, it has to be pointed out that outside of the spell of bad philosophy
- human obsolescence wouldn’t in fact happen. The data that drives “automation”
- has to ultimately come from people, in the form of “big data.” Automation can
- always be understood as elaborate puppetry.
-
- The most crucial quality of our response to very high-functioning machines,
- artificial intelligences and the like, is how we conceive of the things that
- the machines can’t do, and whether those tasks are considered real jobs for
- people or not. We used to imagine that elite engineers would be automation’s
- only puppeteers. It turns out instead that big data coming from vast numbers of
- people is needed to make machines appear to be “automated.” Do the puppeteers
- still get paid once the whole audience has joined their ranks?
-
-## The Technology of Ambient Cheating
-
- Siren Servers do what comes naturally due to the very idea of computation.
- Computation is the demarcation of a little part of the universe, called a
- computer, which is engineered to be very well understood and controllable, so
- that it closely approximates a deterministic, non-entropic process. But in
- order for a computer to run, the surrounding parts of the universe must take on
- the waste heat, the randomness. You can create a local shield against entropy,
- but your neighbors will always pay for it.
-
- Note: A rare experimental machine called a “reversible” computer never forgets,
- so that any computation can be run backward as well as forward. Such devices
- run cool! This is an example of how thermodynamics and computation interact.
- Reversible computers don’t radiate as much heat; forgetting radiates
- randomness, which is the same thing as heating up the neighborhood.
-
-## The Insanity of the Local/Global Flip
-
- A Siren Server can become so successful—sometimes in the blink of an eye—that
- it optimizes its environment—changes it—instead of changing in order to adapt
- to the environment. A successful Siren Server no longer acts only as a player
- within a larger system. Instead it becomes a central planner. This makes it
- stupid, like a central planner in a communist regime.
-
-## The Conservation of Free Will
-
- A story must have actors, not automatons. Different people become more or less
- like automatons in our Sirenic era.
-
- Sirenic entrepreneurs intuitively cast free will—so long as it is their own—as
- an ever more magical, elite, and “meta” quality of personhood. The entrepreneur
- hopes to “dent the universe”* or achieve some other heroic, Nietzschean
- validation. Ordinary people, however, who will be attached to the nodes of the
- network created by the hero, will become more effectively mechanical.
-
- [...]
-
- We’re setting up barriers between cases where we choose to give over some
- judgment to cloud software, as if we were predictable machines, and those where
- we elevate our judgments to pious, absolute standards.
-
- Making choices of where to place the barrier between ego and algorithm is
- unavoidable in the age of cloud software. Drawing the line between what we
- forfeit to calculation and what we reserve for the heroics of free will is the
- story of our time.
-
-## Rewarding and Punishing Network Effects
-
- To understand how Siren Servers work, it’s useful to divide network effects
- into those that are “rewarding” and those that are “punishing.” Siren Servers
- gain dominance through rewarding network effects, but keep dominance through
- punishing network effects.
-
-## The Closing Act
-
- Competition becomes mostly about who can out-meta who, and only secondarily
- about specialization.
-
- [...]
-
- Individual Siren Servers can die and yet the Siren Server pattern perseveres,
- and it is that pattern that is the real problem. The systematic decoupling of
- risk from reward in the rising information economy is the problem, not any
- particular server.
-
-## The limits of emergence as an explanation
-
- But the problem with freestanding concentrations of power is that you never
- know who will inherit them. If social networking has the power to synchronize
- great crowds to dethrone a pharaoh, why might it not also coordinate lynchings
- or pogroms?
-
- [...]
-
- The core ideal of the Internet is that one trusts people, and that given an
- opportunity, people will find their way to be reasonably decent. I happily
- restate my loyalty to that ideal. It’s all we have.
-
- But the demonstrated capability of Facebook to effortlessly engage in mass
- social engineering proves that the Internet as it exists today is not a
- purists’ emergent system, as is so often claimed, but largely a top-down,
- directed one.
-
- [...]
-
- We pretend that an emergent meta-human being is appearing in the computing
- clouds—an artificial intelligence—but actually it is humans, the operators of
- Siren Servers, pulling the levers.
-
- [...]
-
- The nuts and bolts of artificial-intelligence research can often be more
- usefully interpreted without the concept of AI at all. For example, in 2011,
- IBM scientists unveiled a “question answering” machine that is designed to play
- the TV quiz show Jeopardy. Suppose IBM had dispensed with the theatrics, and
- declared it had done Google one better and come up with a new phrase-based
- search engine. This framing of exactly the same technology would have gained
- IBM’s team as much (deserved) recognition as the claim of an artificial
- intelligence, but it would also have educated the public about how such a
- technology might actually be used most effectively.
-
- AI technologies typically operate on a variation of the process described
- earlier that accomplishes translations between languages. While innovation in
- algorithms is vital, it is just as vital to feed algorithms with “big data”
- gathered from ordinary people. The supposedly artificially intelligent result
- can be understood as a mash-up of what real people did before. People have
- answered a lot of questions before, and a multitude of these answers are
- gathered up by the algorithms and regurgitated by the program. This in no way
- denigrates it or proposes it isn’t useful. It is not, however, supernatural.
- The real people from whom the initial answers were gathered deserve to be paid
- for each new answer given by the machine.
-
- [...]
-
- What all this comes down to is that the very idea of artificial intelligence
- gives us the cover to avoid accountability by pretending that machines can take
- on more and more human responsibility. This holds for things that we don’t even
- think of as artificial intelligence, like the recommendations made by Netflix
- and Pandora. Seeing movies and listening to music suggested to us by algorithms
- is relatively harmless, I suppose. But I hope that once in a while the users of
- those services resist the recommendations; our exposure to art shouldn’t be
- hemmed in by an algorithm that we merely want to believe predicts our tastes
- accurately. These algorithms do not represent emotion or meaning, only
- statistics and correlations.
-
- What makes this doubly confounding is that while Silicon Valley might sell
- artificial intelligence to consumers, our industry certainly wouldn’t apply the
- same automated techniques to some of its own work. Choosing design features in
- a new smartphone, say, is considered too consequential a game. Engineers don’t
- seem quite ready to believe in their smart algorithms enough to put them up
- against Apple’s late chief executive, Steve Jobs, or some other person with a
- real design sensibility.
-
- But the rest of us, lulled by the concept of ever-more intelligent AIs, are
- expected to trust algorithms to assess our aesthetic choices, the progress of a
- student, the credit risk of a homeowner or an institution. In doing so, we only
- end up misreading the capability of our machines and distorting our own
- capabilities as human beings. We must instead take responsibility for every
- task undertaken by a machine and double-check every conclusion offered by an
- algorithm, just as we always look both ways when crossing an intersection, even
- though the signal has been given to walk.
-
- When we think of computers as inert, passive tools instead of people, we are
- rewarded with a clearer, less ideological view of what is going on—with the
- machines and with ourselves. So, why, aside from the theatrical appeal to
- consumers and reporters, must engineering results so often be presented in
- Frankensteinian light?
-
- The answer is simply that computer scientists are human, and are as terrified
- by the human condition as anyone else. We, the technical elite, seek some way
- of thinking that gives us an answer to death, for instance. This helps explain
- the allure of a place like the Singularity University. The influential Silicon
- Valley institution preaches a story that goes like this: One day in the
- not-so-distant future, the Internet will suddenly coalesce into a
- superintelligent AI, infinitely smarter than any of us individually and all of
- us combined; it will become alive in the blink of an eye, and take over the
- world before humans even realize what’s happening.
-
- Some think the newly sentient Internet would then choose to kill us; others
- think it would be generous and digitize us the way Google is digitizing old
- books, so that we can live forever as algorithms inside the global brain. Yes,
- this sounds like many different science fiction movies. Yes, it sounds nutty
- when stated so bluntly. But these are ideas with tremendous currency in Silicon
- Valley; these are guiding principles, not just amusements, for many of the most
- influential technologists.
-
- It should go without saying that we can’t count on the appearance of a
- soul-detecting sensor that will verify that a person’s consciousness has been
- virtualized and immortalized. There is certainly no such sensor with us today
- to confirm metaphysical ideas about people. All thoughts about consciousness,
- souls, and the like are bound up equally in faith, which suggests something
- remarkable: What we are seeing is a new religion, expressed through an
- engineering culture.