Democracy Lab

The Signal and the Noise in Middle East Elections

Social media can tell us a lot about voting patterns -- but only up to a point.

What can social media and "big data" tell us about electoral behavior in non-Western countries? At its best, social media can act as a relatively accurate metric for public opinion and reliably forecast political behavior. At its worst, however, it can obscure those opinions and present a skewed version of public preferences.

Social and computational scientists have debated whether we can forecast United States congressional election outcomes with various metrics gained from Twitter. In a Washington Post op-ed, Indiana University professor Fabio Rojas proclaimed:

"Digital democracy will put these campaign [polling] professionals out of work. New research in computer science, sociology, and political science shows that data extracted from social media platforms yield accurate measurements of public opinion."

This claim has its critics. Other political scientists point out that candidate incumbency itself can often predict elections to a high degree of accuracy. A number of computer scientists (here, here, and here) hold that a true "forecast" should predict elections "out-of-sample" (that is, researchers build a model with a certain set of data and use it to predict another separate set of data). As a result, real forecasts must be published before the election.

Yet there may be some support for the argument that social media can act as a reliable proxy for public opinion and electoral behavior. The major assumption here is that the electoral preferences of social media and Internet users looks something like the electoral preferences of the electorate. In non-Western or more closed political systems, where polling may be less common or people are less likely to express their true partisan positions to pollsters, social media may be a good indicator of public opinion. Then again, it may also be incredibly misleading.

We can look into this question by using the recent presidential race in Iran as an electoral case study. This is one case in which social media and other Internet data gave some indication of electoral dynamics against the conventional wisdom. But is this always accurate, or merely a function of the web presence of particular candidates? Here, we'll compare the results to the 2012 Egyptian presidential election, in which social media fails (and fails badly) in forecasting the winner of that race.

Speculation on candidates in the run-up to Iran's June 2013 presidential election ran wild. In March, the conservative website Mashregh listed no less than 26 possible contenders from across the political spectrum.  On May 21, the Guardian Council -- the body which vets electoral contenders based on the vague notion of constitutional "suitability" -- approved eight candidates. Prominent conservatives permitted to run included Tehran's mayor Mohammad-Bagher Qalibaf, National Security Council member and nuclear dossier negotiator Said Jalili, and former Revolutionary Guards Commander Mohsen Rezai. In addition, the council approved two candidates from the center-liberal side of Iran's political establishment: former vice president Mohammad-Reza Aref and a former negotiator of the nuclear dossier, Hassan Rouhani.

Live televised debates took place on May 31, June 5, and June 7. One of us (Kevan) was in Tehran at the time. At the outset it was difficult to tell each candidate's chances. All the candidates tried to disassociate their own platform from then-President Mahmoud Ahmadinejad's lame duck record. Quite a few previous supporters of reformist politicians, citing the events of 2009, loudly announced they would not be voting at all. Western news outlets, believing all decisions great and small to securely be in the hands of Iran's Leader, Ali Khamenei, forecast a conservative victory. However, the TV debates, especially the rowdy third debate on June 7, reportedly watched by over half of the population, appeared to show that a real race was afoot. On the ground in Tehran and provincial cities, closer to the action, we could sense something new: a reawakening of electoral mobilization that had been politically dormant since 2009. Yet would it be big enough to register at the ballot box, given well-known popular misgivings? And could it be gauged at all?

Polling in Iran is notoriously spotty, but sociologist Hossen Ghazian ran a well-constructed and independently funded daily phone tracking poll in the two weeks preceding the election. With over 1,000 randomly sampled respondents each day, Ghazian's organization, Information and Public Opinion Solutions (IPOS), provided a crucial measure of changing support for various candidates -- even with 40 percent of respondents answering as undecided, even up until the last day. Nevertheless, IPOS captured the most important shift in the election. When Mohammad-Reza Aref pulled out of the race on June 11 and threw his support to Hassan Rouhani, the election became a choice between a single center-liberal candidate and numerous conservative candidates. With former presidents Rafsanjani and Mohammad Khatami backing Rouhani, the mobilization surge also brought in three key voting blocs: those who had previously decided to abstain, those who were still undecided, and those who switched their vote away from conservatives.

Ghazian captured the trend, but his poll lagged slightly behind a rapidly changing reality. The final IPOS poll on June 13 put Rouhani at 38 percent of decided voters, but 42 percent of the respondents stated they were still undecided. Rouhani won the vote the next day with 50.7 percent of the vote.

Could the hive mind of Twitter and Facebook have predicted the outcome better? This is what Malaysia-based blogger and "social media specialist" Pooria Asteraky claimed, suggesting that social media was a sufficient reflection of political opinion. Using aggregate counts of each candidate from Facebook, Twitter, blogs, and Google Trends, he forecasted Rouhani's victory as soon as the third debate on June 7, far in advance of Aref's withdrawal. In his final forecast, he predicted a 52.1 percent  vote share for Rouhani, within 1.5 percent of the true vote share.

The above figure shows Pooria's estimation of the popularity of each candidate. Although he seems to capture Rouhani's bump from Aref's withdrawal, we had some methodological doubts over Pooria's approach and attempted to replicate his analysis. We searched for candidates' names in Google Trends and Twitter (using the Twitter search tool Topsy), since full historical data for Facebook and blogs are generally unavailable.

Following Pooria, we looked solely at the volume of candidate mentions within Iran in the Persian language. In the Google Trends data, there doesn't seem to be any clear front-runner on the day of the last debate. However, after the debate, Rouhani and Aref begin separating themselves from the rest of the pack, suggesting that the debates themselves may have swayed some voters. Or, at least, their interest was piqued enough to search for these two names.

As for Twitter, which has a much lower penetration rate in Iran than Google, the trends are similar but with minor differences. Much more Twitter traffic centered around the debate. After the debates, Rouhani maintained a narrow lead in tweet volume over Aref and Jalili but didn't substantially break away until much closer to the election date.

These are, to be sure, somewhat rough and naïve estimators of public opinion, predicated on the idea that "all publicity is good publicity." But if one buys into the "more tweets, more votes" thesis, then this is a good enough reflection of electoral preferences of Iranians that manifested in Rouhani's June 14 victory.

Does the same kind of relationship hold in other hotly contested elections in the Middle East? Egypt held its first open presidential election in 2012 after the ouster of Hosni Mubarak. With most political forces still consolidating power, the field of contestation seemed wide open between a variety of Islamists, reformists, those regarded as feloul, or remnants of the old regime, and all in between. The Muslim Brotherhood, considered the most organized political organization in post-revolution Egypt, broke their initial promise to refrain from fielding a presidential candidate when it ran one of its leading members, Khairat al-Shater, and again when they ran Mohamed Morsi after Shater was disqualified. Hamdeen Sabahi, a long-time reformist politician and activist associated with the Nasserist Karama party, made an impressive and unexpected showing in the first round. From the former regime, former Foreign Minister Amr Moussa and Mubarak's final Prime Minister Ahmed Shafik appealed to many Egyptians' want for order and their fear of an Islamist president. The reform Islamist Abdel Moneim Aboul Fotouh -- a former Muslim Brother -- appealed to many, including a wide swath of youth.

After nearly seven months of uncertainty (the Supreme Council of the Armed Forces had announced that elections would take place as early as November 2011), the election campaign began to coalesce. On May 10, Moussa debated toe-to-toe with Aboul Fotouh for nearly six hours on national TV. Two weeks later, the first round of the elections were held, with Morsi and Shafik making the most impressive showings, Sabahi in third, and those who had actually debated, Aboul Fotouh and Moussa, taking fourth and fifth, respectively. In the second round, on June 16-17, Morsi was claimed as the winner with 51.7 percent of the vote. He was not officially certified until June 24.

What could social media tell us about these election dynamics? Using the same approach as we did in Iran, we looked at Google Trends and Twitter volume for mention of these candidates in Arabic within Egypt.

Google Trends in Egypt first show a peak after the debate between Aboul Fotouh and Moussa. Before the first round, Fotouh, Shafik, and Sabahi are nearly tied in search volume. After the first round, however, the buzz about Shafik dwarfs that of any other candidate. His volume remains high and he stays ahead of Morsi even after the first round. It isn't until the election results are finally announced that Morsi displaces Shafik.

Twitter tells a largely similar story, with two important differences: Fotouh is more referenced on Twitter in the first round, possibly reflecting his popularity with youth. Second, Morsi doesn't displace Shafik until days after he is formally certified as the election victor.

In this case, relying on search volume and web mentions would have been an incredibly poor way to predict the outcome of the Egyptian presidential election.

The lesson here isn't that Twitter and other web data simply provide no important information. There is indeed something happening when we can retrieve trends from what people search and what they say. The fact that they reflect electoral events, whether mediated through broadcast media or otherwise, means we're picking up on salient political behavior. What isn't clear, however, is how these data are produced and what that means for their quality as an indicator of political preferences.

In Iran and Egypt, as elsewhere, web data tend to be produced by those who have frequent access to the Internet, usually urbanites and the middle classes. These groups tend to have particular political biases that lean reformist and anti-Islamist. In Iran, they learned towards Rouhani, in Tehran as well as in nearly every urban area. In Egypt, those biases were against Morsi -- the most populous governorates went for Shafik (with the exception of Alexandria, a stronghold of the Muslim Brotherhood).

To put it in statistical terms, using data from social media in both cases results in a biased sample. There are ways to correct for particular known biases, but as we move from country to country and election to election, the sample changes in ways that we can't easily measure.

Given these issues, it may be somewhat premature to proclaim the "death of the pollster," especially in the Middle East, where a distant gaze on public expression of political preferences is bound to give a skewed view of reality.



Stolen History

Why should we care about saving Syria's antiquities?

Art is often a forgotten victim of wars. As the toll of human suffering builds, worrying about the fate of paintings, sculptures, and antiquities might seem frivolous, even callous. But there is good reason to care about preserving culture both in conflict and after -- and there are plenty of proponents of this view, including among governments. For instance, the upcoming movie Monuments Men, starring George Clooney, tells the true story of the group of individuals tasked by the U.S. government during World War II with finding art stolen by the Nazis and returning it to the rightful owners.

Today, a new conversation about how to protect the priceless when people are trying to survive is playing out, this time with regard to war-torn Syria. At a gathering this fall in New York, the U.N. Educational, Scientific and Cultural Organization (UNESCO), the U.S. Department of State, and the International Council of Museums (ICOM) announced the publication of the Emergency Red List of Syrian Cultural Objects, which aims to prevent the transport and trade of Syria's invaluable cultural goods.

Syria is rich with ancient and medieval treasures: Greek and Roman cities, Byzantine villages, Bronze and Iron Age sites, centuries-old castles, and ornate Islamic art and structures. But the State Department says that nearly 90 percent of these invaluable historical sites and objects are within areas of conflict.

Much like the Nazis, looters have taken note and ruthlessly pillaged Syrian cultural sites, seeking to sell treasures on the black market. Just last spring, a cobble-stone, columned street built by Roman emperor Marcus Aurelius in the city of Apamea was plundered and damaged. Architectural gems have also fallen prey to armed conflict. In April 2013, the nearly 1,000-year-old minaret of Aleppo's Umayyad mosque collapsed during an intense battle.

With these problems becoming more serious, the Red List is an initiative that notifies law-enforcement personnel, customs inspectors, art dealers, auction houses, and museums around the world of the types of pilfered objects that may be on the market and moving through legitimate shipping channels. Speaking at one of America's premier sanctuaries for cultural heritage, the Metropolitan Museum of Art in New York, U.S. Assistant Secretary of State Anne Richard explained that the Red List includes nearly every object imaginable, such as ancient writings, vessels, coins, stamps, sculptures, and accessories. Giving notice helps prevent stolen objects from becoming ill-gotten spoils of war.

Red Lists have been created before for other countries. In the past five years, lists have helped French officials identify and recover cultural goods from Iraq and Togo. In 2007, Switzerland stopped the illegal online sale of a cuneiform tablet, one of the earliest examples of written language, thought to have been smuggled out of Iraq. More recently, U.S. customs inspectors recovered and returned stolen Afghan items, including a Roman wine pitcher, taken by looters. ICOM envisions the Red List for Syria will lead to similar success stories.

The rationale behind the Red List, however, extends beyond a desire to keep things where they belong. Indeed, there are other important reasons to protect Syria's historical gems.

The preservation of Syria's cultural heritage is critical to its reconstruction, reconciliation, and re-building of civil society, Richard argued at the Met event. Historical sites and objects "are a part of Syrian life -- a source of pride and self-definition for their present and future," she said. Losing its cultural history would rob Syria of the economic opportunities linked to tourism and cultural preservation; in 2010, tourism accounted for 12 percent of the country's GDP and employed 11 percent of its workers.

The Red List is also part of a larger project to combat corruption and poor governance that benefits from illicit commerce, which the World Economic Forum estimates might include up to 15 to 20 percent of annual global trade. Illicit trade networks, which facilitate the exchange of trafficked persons and wildlife, ill-gotten funds, and cultural objects, also allow corrupt leaders and officials to retain and grow their power. And the need for international cooperation to combat regional and global illicit trade is paramount.

This is the reason that, halfway around the world from New York, political leaders at the Asia-Pacific Economic Cooperation (APEC) Pathfinder Dialogue in Bangkok met almost simultaneously to the Red List gathering. On the APEC agenda was a discussion of the global fight against illicit trade and corruption. Dialogue participants shared their best practices and agreed to support the drafting of new international documents and investigations to combat illegal commerce.

Together, the Pathfinder Dialogue and the Red List demonstrate the importance that the international community places on preserving art and artifacts. And as Monuments Men shows, this support is nothing new.

General Dwight D. Eisenhower, then supreme allied commander in Europe, took a particular personal interest in protecting, preserving, and repatriating cultural property in World War II. Led by American and British soldiers, the Monuments, Fine Arts, and Archives Section of the Allied military effort included a collection of 345 men and women from 13 countries who recovered thousands of stolen artworks between 1943 and 1951, including works by Johannes Vermeer, Leonardo da Vinci, and Michelangelo.

Their efforts to protect and preserve Europe's cultural history -- now on display in the great museums of Europe -- are a living legacy to those striving to protect Syria's cultural relics. Much like our collective efforts in the 1940s helped preserve the foundation of European cultural identity, preventing looters and illicit markets from robbing Syria of its past will protect an important component of the country's future, peaceful identity.