John Ferrell/<a href="http://www.loc.gov/pictures/item/fsa1998022982/PP/">Library of Congress</a>
Last week, researchers from the Universities of Indiana and Manchester published a paper in the online journal arXiv.org claiming that Twitter can be used to predict the stock market. It’s a strange idea, though not without precedent. To understand the Twitter-Dow Jones link, first consider the relationship between ice cream and crime. If you were to compare ice cream sales and crime rates in a city like Brooklyn, you’d probably notice something odd: while one is up, so is the other. When people start buying more ice cream, it seems they also start committing more crimes.
Crime rates and ice cream sales move together because they are both influenced by warm weather. When it’s hot, the days are longer, and more people are outside on the street, talking, walking, laughing, buying ice cream, and, inevitably, making trouble. The relationship between ice cream sales and crime rates is used in statistics classes to illustrate the old truism that correlation does not imply causation. Just because two variables move together, that doesn’t mean they affect the other’s behavior.
But the ice cream example also illustrates another point. We can look at ice cream sales to gain some insight into crime rates. They don’t affect one another, but they nonetheless move together, and so we can use information we have about one system to learn about another less-understood one. Getting back to the stock market, it’s notoriously difficult to predict. But perhaps there is another system that moves like the stock market—one that is easier to analyze. The paper from Indiana and Manchester argues, essentially, that Twitter might be to the stock market what ice cream is to crime rates.
First reported at Technology Review, the research investigates whether measurements of public mood taken from Twitter can be used to predict future behavior in the market. According to the researchers, using the measurement of “calmness” in more than 9 million tweets in 2008, they predicted the changes in the Dow Jones Industrial Average a few days in advance, with an accuracy of 87.6%. As Technology Review points out, there are reasons to be skeptical of this percentage: “The first is the lack of plausible mechanism: How could the Twitter mood measured by the calmness index actually affect the Dow Jones Industrial Average up to six days later? Nobody knows.” Another is that the tweets were taken from around the world. “By this reckoning, tweeters in Timbuktu somehow help predict the Dow Jones Industrial Average.”
That nobody knows why the mood on Twitter would be linked to behavior six days later on Wall Street does not mean it is impossible: In Glasgow in the 1980s, rival ice cream truck companies began warring for territory. The conflicts erupted into daily violence, and in April of 1984 resulted in the killing of six people, one of Glasgow’s largest mass murders. As it turns out, the trucks were fronts for crime families who sold drugs—in addition to “pokey hats, iced lollies and single cigarettes”—along their routes. The Ice Cream War, as it has come to be called, was a turf war that had little to do with ice cream. But the fact remains that in Glasgow in 1984, banning ice cream trucks would have reduced crime.
