On Dec. 6, 1941, the Foreign Broadcast Information Service (FBIS), a radio monitoring operation set up by the U.S. intelligence community and one of the earliest experiments in what it now called open-source intelligence, delivered its very first report, an analysis of Japanese media sentiment. The report noted that Japanese radio stations had sharply increased their level of criticism of the United States and dropped their calls for peace. The next day, Pearl Harbor was attacked.
COMMENTS (15)
More...
Obviously, no amount of media monitoring would have revealed when and where the attack would take place (that's what spies are for), but it's certainly possible that with a better sense of the likelihood of an attack, U.S. forces might not have been caught quite so unawares. Some 70 years later, one computer scientist believes that a somewhat more ambitious version of the same type of news monitoring may soon be able to predict social upheavals and conflicts -- such as the recent revolutions in the Arab world -- with a remarkable degree of accuracy.
Kalev Leetaru, the assistant director for text and digital media analytics at the University of Illinois's Institute for Computing in the Humanities, Arts, and Social Science, is one of the leading researchers in the emerging field of conflict early-warning. In a paper published this month in the peer-reviewed online technology journal First Monday, Leetaru argues that "computational analysis of large text archives can yield novel insights to the functioning of society."
Leetaru's study builds on recent economics research looking at how the tone of news and social media coverage can predict economic events. One recent paper, for instance, found that the general mood state on Twitter can anticipate the movements of the Dow Jones Industrial Average. Leetaru was curious whether the same type of analysis could predict social events.
Leetaru employed several massive databases of news articles over the last 30 years, including the "Summary of World Broadcasts" -- English translations of foreign broadcasts done by the British equivalent of the FBIS -- the complete digital archives of the New York Times, and a web crawl of online news sites, to create a dataset of around 100 million news articles dating back to 1979. He then fed this raw material into one of the world's most powerful supercomputers, the University of Tennessee's "Nautilus," and began to look for patterns.
In recent years, companies have increasingly deployed "sentiment mining" software to gauge the tone of news coverage. Think of it as a hypersophisticated Google Alerts: These programs scan news articles for positive and negative words and can also distinguish the severity of feeling, knowing the difference between "loathe" and "dislike" for instance. The software misses a lot of nuance and can be fooled by sarcasm, but at the scale of data Leetaru was working with, it gives a pretty good indication of global media sentiment on a given topic.
SUBJECTS:
(15)
HIDE COMMENTS LOGIN OR REGISTER REPORT ABUSE