Predicting the Future with Crowds
Crowds are odd things. They seldom behave the way we expect – but studying them can help to solve us many different problems. Google uses its own crowd of engineers to help predict the future, running a prediction market as a game on its intranet. Questions and answers are bought and sold, and the results help Google’s executive team make decisions. It turns out that if you ask a large number of people a question, their answers tend to cluster around the right one. Make the question a bet, and the answers become more accurate still. Google asks its staff a mix of different questions – from serious ones like “when will Google Mail leave beta” to more frivolous ones – which are used to calibrate responses. As well as helping make decisions, the answers and the patterns behind them have given Google insights into its own internal social network and how relationships between people affect internal communications.
Google also uses the crowds that use Google as a search engine to help it work with the huge amounts of data that are found online. People tend to use similar words for things, making it easier to classify and categorize content. The Web is large enough that even if Google misses 99% of samples, the 1% it catches will be enough to give good answers. The same techniques help run its machine translation service, where parallel texts like news stories and hotel information build a probabilistic model of how languages work. It’s an approach that needs a lot of data, but once the system has been trained it works well – though Google formally proves just how it works. IBM came up with the idea of translating by probability but didn’t have enough data to work with. Google has the data but found it was more efficient to change the formula from what mathematics says it should be.
Professor Paul Torrens of Arizona State University thinks crowds are important, too. He’s been using computer modeling to show how large groups of people respond – and how they’ll behave given specific physical situations. Some of his work is used to help design safer buildings and streets – showing how bottlenecks can be avoided that can cause injury and deaths when panicked people run away from an emergency. He’s also been working on modeling how demonstrations turn into riots, aiming to design cities that make it safer to have large demonstrations without an unruly minority making them violent. The same approaches can be used for behavioral forensics, which involves using phone data to show just how people move around a city – and if seemingly unrelated actions are connected.
MIT research scientist and Fulbright lecturer Nathan Eagle follows crowd dynamics by tracking just a few people – again using cell phone location information. His Media Lab research project has been tracking people, collecting over 400,000 hours of data in a year. It turns out some people have set habits, while others behave quite randomly. The same data explains social networks as well as the rhythms that govern the behavior of large organizations, and you can infer demographics with 90% accuracy. Your information could produce an automated life log, which can help answer questions about the way you live your life: How much sleep did I get? What did I do last Saturday night after midnight? How much time do I spend driving?