Computer scientists at Hewlett-Packard recently published a research study that showed that Twitter can predict the success of a movie better than any other measure, including the Hollywood Stock Exchange, considered the gold standard in the movie industry.

The paper "Predicting the Future With Social Media" by Sitaram Asur and Bernardo Huberman, analyzed 2.89 million tweets from 1.2 million users referencing 24 movies released over a three month period.

The study looked at the pre-launch buzz such as the release of trailers, it found:

– Very few retweets because "people tend to describe their own expectations and experiences, which are not necessarily propaganda."

– URLs in the pre-release tweets didn’t seem to have much effect on popularity of the movie, a surprising discovery since pre-launch publicity should contribute to success in the box office.

The study then looked at the first week of release and found a high correlation between the rate of pre-launch tweeting and the success of the opening weekend.

The study authors compared their findings with the Hollywood Stock Exchange (HSX)-this is considered the ‘gold standard’ in predicting the success of movies by offering a trading platform that allows people to buy and sell virtual shares in movies.

The study was "consistently better at predicting the actual values than the historical HSX prices."

The researchers also applied a sentiment analysis on the tweets, classifying them as positive, negative, or neutral. They hired thousands of people through Amazon’s Mechanical Turk, to perform the sentiment analysis.

As would be expected, a majority of positive sentiments in Tweets predicted movie success. However, a movie could have a poor opening weekend but then win a growing audience in the following weeks, a trend predicted by positive tweets.

The study authors say that this method could be used for predictions for many other topics, including the outcome of elections:

While in this study we focused on the problem of predicting box office revenues of movies for the sake of having a clear metric of comparison with other methods, this method can be extended to a large panoply of topics, ranging from the future rating of products to agenda setting and election outcomes. At a deeper level, this work shows how social media expresses a collective wisdom which, when properly tapped, can yield an extremely powerful and accurate indicator of future outcomes.

“We’re very interested in how social attention is allocated.”

I spoke with Bernardo Huberman, head of HP’s Social Computing Lab, about the Twitter research project. Here are some notes from our conversation:

– We’re very interested in social attention and how attention is allocated, especially in a very fragmented media world that we now have.

– Also, the study is about the wisdom of crowds and collective intelligence. I’m not sure about the ‘wisdom’ part, but on average, there does seem to be a collective intelligence.

– We are interested in how agendas can be set when the influence of traditional media is waning. In the past, for example, the editors at the Wall Street Journal or Financial Times, or CNN, would decide which stories they would publish and that’s what we would see but now the world is fragmented.

– The Twitter movie study helps to show how agendas can be established and how it could be applied elsewhere.

– The Twitter movie research was very difficult, we looked at a lot Tweets. We made use of Mechanical Turk for sentiment analysis, hiring thousands of workers.

– The Twitter movie study was interesting, we have some departments within HP that want to use the same methods for predicting the success of products. We have applied for a patent on the process.

– I don’t think the movie tweets could be gamed because we looked at a very large number from a large number of people.

– We have thought about looking at financial information such as stock market tweets and also at where other online conversations are happening.

– I’m very interested in the problem facing media companies, how to monetize content. In a world where everything is free, information rapidly becomes commoditized and when everyone has the same information, it has little value. Our studies have shown that if people pay for information they value it more highly, they have something others don’t have. It’s a very hard problem. I’m not yet sure how to tackle it.