Democracy Lab

Pictures at a Revolution

Data visualization can offer some unique insights into social upheaval. But the data artists are just getting started.

At first glance you don't quite know what you're looking at. It could be strands of mitochondrial DNA, or sperm fertilizing an egg, or perhaps the product of an incongruous union between a sea creature and a hairball. As the thing grows, it passes through a series of increasingly complex mutations -- ending up as a kind of Death Star under construction.

You're looking at data -- or, to be more precise, a visual rendering of tweets and retweets from a several-hour period on February 11, 2011, starting shortly before the resignation of Egyptian President Hosni Mubarak.

More than ever before, we inhabit a cosmos of data. With smart phones and the ability to share reams of information publicly on social media, our digital footprints are everywhere -- from what we “like” on Facebook to our store loyalty cards to banking transactions on our smart phones.


With that explosion of data have come new ways to visualize it. From Sumerian cuneiform to the humble pie chart, visual symbols transform the abstract into the concrete, helping us to see patterns and relationships we might have otherwise missed. The bigger the data sets, the more useful pictures of them become -- and at no time in human history have we ever had to deal with floods of data greater than the ones that wash over us now.

Of all the social phenomena that invite analysis, few are as complex, or as volatile, as revolutions. The petabytes of social media data generated by the upheavals of the Arab Spring are fertile ground for social scientists studying those events. For years we've been snapping photos of demonstrators and protests; now the new cosmos of data potentially enables us to map the ebb and flow of the ideas that drive them, something like a magnetic-resonance imagery technique for visualizing the mechanisms of dramatic change.

Interest in the role of social media first exploded during Iran's post-election unrest in 2009. In his "Retweet Revolution," Gilad Lotan, a designer and computer scientist, tracked Twitter conversation threads around popular hashtags such as #iranelection, #ahmadinejad, and #mousavi. By visualizing flows, Lotan says that we can "put a face" to an audience -- as, for example, in this striking project about the Arab Spring, authored by Lotan and several of his collaborators (click to expand):

In 2011, Kovas Boguta beautifully mapped the "Egypt Influence Network," showing Arab- and English-language tweeters -- and, perhaps most strikingly, the bridge nodes that spanned the linguistic divide.

For social scientists, big data sets and visualizations are already proving to be powerful tools. Writing about network diagrams of tweets around Iran's 2009 post-election crisis, researcher Devin Gaffney saw that by using tag clouds, "we can identify the key terms used in tweets," as he wrote. "By looking at different tag clouds over time, we can perhaps even see terms reflect a general shift from awareness/advocacy towards organization/mobilization, and eventually action/reaction."

Data scientists can also map the growth of ideas over time. In 2008, John Kelly, of Morningside Analytics, a company that specializes in social network analysis, visualized the Iranian blogosphere. When he did the same a year later and compared the maps, he discovered not only that the number of blogs had grown, but also that new segments had sprung up. One of them, which he dubbed “CyberShia,” revealed a dramatic increase in blogging activity by religiously oriented users. While this could point to an effort by the pro-regime Basij militia to contain dissident discourse on the Web, another theory sees the data as evidence of an intensifying debate about Islamic law and its role within the country’s political system.

Iranian blogosphere 2009:

Iranian blogosphere 2008:

Similarly, by looking back at social-media data emerging from the Arab Spring, it's possible to see political ideas congeal and take shape. By evaluating data from 2010 and 2011, the Arab Media Influence Report, which captured over 10 million online conversations per day, showed how discourse became politicized by the start of 2011. In the first quarter of 2010, 57 percent of the Arabic conversations on social media included socio-economic terms (such as income, housing, and minimum wage). By 2011, that number had dropped to 37 percent. In 2010, 35 percent of the conversations on social media included political terms such as revolution, corruption, and freedom). In 2011, the number shot up to 88 percent.

Of course, we hardly need social media data to tell us that Arab society was becoming more politically aware in the period leading up to the revolts. Arguably one might have arrived at a comparable conclusion by sitting in Cairo coffee houses for a year or by tracking debates in the Arabic press.

Data visualization really comes into its own when it allows us to see patterns we might otherwise have missed, patterns that can be modeled and applied to other contexts. If, as some sociologists believe, structure is destiny, then network graphs might be able to tell us about the life spans of political movements, their likely growth and their eventual demise. "People tend to think about the qualities of the individual when they figure out whether they will be likely to succeed or fail," says Marc A. Smith, the director of the Social Media Research Foundation, a California-based nonprofit. "Network theory people, and sociologists more generally, like to think about the properties of a person's network as having equal, if not greater consequence to their likely outcomes."

As an example Smith cites two visualizations he made of the Occupy movement and the Tea Party on Twitter. In his renderings, the Tea Party appears as a far more tight-knit group, with many of them following each other, whereas Occupy is made up of looser clusters with a few high-profile accounts receiving plenty of retweets. In the lower right-hand corner of the visualizations there is a grid, a matrix of "isolates:" People who are talking about the ideas of Occupy or the Tea Party, but who don't have connections to others on the graph. For Occupy, the number of isolates is greater, which according to Smith could indicate a larger potential for growth and stronger brand cachet.

TeaParty on Twitter:

Occupy on Twitter:

When interrogating the data, the answers they yield are only as good as the questions we ask. Looking at the Twitter data obviously doesn't tell us how the Occupy forces in Zuccotti Park behave, sound, or smell. (The Twitterverse, after all, is not the universe.) But as Smith notes, fleshing out the data shows that that the group's structure strikingly reflects its self-description as a decentralized and bottom-up movement. "There's no question that big data can be very, very useful," says Zeynep Tufekci, a sociologist at the University of North Carolina. "But it's less useful and even misleading at times if it is not evaluated by people who understand the context of what they're looking for."

Without any context about Internet penetration or the demographic of Twitter users, network diagrams of Egyptian tweeters can give us the impression that Twitter played an oversized role in the unrest there. We might also get an overblown impression of the liberal character of the Arab Spring. But instead of being broadly representative of Egyptian society, these tweeters are rather a small sub-set of young, educated, often English-speaking elites, with a propensity for liberal ideas. They are perhaps as revealing about Internet connectivity as they are about Egyptian society. (Gregor Aische, a German designer, has produced his own striking visualization of the global digital divide.)

But perhaps it helps to ask a different question. If we want to analyze what's going on in the minds of Egypt's most influential elites, then that social media data set can certainly offer useful answers. "That may only be a sampling of a small [segment] of population, but if that's how they're organizing and communicating, then that's an important population," says Noah Iliinsky, an expert in the theory and practice of information visualization. "So I wouldn't discount that population simply because it's not representative."

What has got businesses, governments, and academics excited about big data and visualization is the ability to detect patterns in real time rather than mapping perceptions post-factum -- and even to use this data to make predictions. We're already seeing cases in which visualized data enable policy makers to make quick informed choices about public health, poverty, or energy efficiency. Google Flu Trends, which estimates current flu activity around the world in real-time by monitoring search terms, has been shown to predict confirmed cases of flu with a level of accuracy comparable to the Centers for Disease Control and Prevention. In 2010, researchers managed to predict with an accuracy of 87.6 percent the daily changes in the closing values of the Dow Jones Industrial Average by analyzing Twitter users' moods.

Could comparable techniques work for predicting social upheaval? The UN has launched an initiative, Global Pulse, that uses new technologies to collect, analyze, and filter information to help governments and organizations better understand what is happening in certain at-risk communities. In 2010 and 2011, the Ushahidi group, famous for its pioneering crowdsourcing and real-time visualization software, created a website that tracked potential disturbances during Liberia's elections.


On a similar note, the Associated Press reported in November 2011 that the CIA was monitoring five million tweets a day to monitor revolutionary change. (Of course, the same data can be used by repressive governments who want to track dissent and unrest.)

One of the key problems in making assumptions about societal change based on social media data is our limited understanding of the relevant conversion rates. Just as online marketers, advertisers, and political campaigners are frantically trying to understand the relationship between tweets and dollars or tweets and votes, the conversion rates for social change are murkier still. What assumptions, for example, can data scientists make between tweet volume and the amount of people likely to attend a protest? Or can the sentiments expressed in Facebook status updates be mined to produce an accurate indication of support for Vladimir Putin?

Given that social media analysis is still in its infancy, the answers to those questions remain elusive. So-called sentiment analysis still struggles to distinguish the signals from the noise. Network diagrams mapping relationships between tweeters and "likers" tell us that there is a big crowd, but they are pretty unhelpful in telling us what that crowd is thinking and why. While programs will become better at parsing huge amounts of data, they are still more comfortable counting than they are at interpreting. Computers still struggle with slang, sarcasm, and subtexts.

Take these tweets from user DanielNothing, who was tweeting about the London riots on August 6, 2011:

Heading to Tottenham to join the riot! who's with me? #ANARCHY

Clear enough, right? If thousands were retweeting DanielNothing's tweet, or tweeting similar sentiments, police officers might be well-advised to deploy resources accordingly. But then DanielNothing tweets again:

Hang on, that last tweet should've read 'Curling up on the sofa with an Avengers DVD and my missus, who's with me?' What a klutz I am!

Only friends of DanielNothing could say for sure what he meant. Is he just being sarcastic? Or could his first tweet be taken at face value, with his second being read as an attempt to mask his true intent? If a human struggles to decipher the true meaning, how would a computer fare?

Unpacking the complex performance art of social media is not easy. Our emotions on display might not always be attuned to the emotions we feel and our stated preferences might be approximations of our true desires. The cultural and societal meaning of a retweet or a Facebook "like" is more complex than we may think and varies across societies and platforms. "Not everyone is talking about it while mentioning it," says Fadi Salem, a fellow of governance and innovation at the Dubai School of Government.

People regularly retweet links to things they haven't read, not necessarily because they endorse them, but because they value the source or succumb to peer pressure. "At the moment, most [social media] analyses are not only superficial, but kind of crude," says Tufekci. "You don't even know were these tweets positive or negative... and what does this excitement correspond with."

Despite these limitations, the amount of data is unprecedented, and is correspondingly poised to transform sociological research. "We are now getting moment-by-moment statements from hundreds of millions of people in a native machine-readable format," says Smith. "This has never happened before."

Finding more innovative ways to process, parse, manage, and visualize this information will be crucial as the mountains of data grow. Piers Fawkes, who runs PSFK, a trends analysis company that has consulted for UN Global Pulse, says the data-rich future will encompass not just social-media data but also search queries, YouTube queries, and financial and retail transactions. The open data movement is pushing for more and more data sets to be made public so that others can benefit from them. Global notions of privacy are likely to loosen as we grow more comfortable (or clueless) about sharing our information.

As more people get online around the world, especially through their cheap smart phones (of which there are now some 4 billion), data streams will proliferate and most segments of society will reveal themselves through various kinds of social media data. (Last year, for example, Twitter user @Arasmus used information from Twitter accounts to map out violence against pro-democracy protestors in Libya.)

Advocates of open data are excited by the possibilities -- as no doubt are certain governments interested in more sophisticated ways of snooping on their citizens. Mobile phones allow us to become nodes in a human sensor network, feeding data sets on the weather or the state of water pumps. Big data visualized in real time can help to manage traffic congestion or to allow medical workers to better allocate supplies in hospitals.

"Data visualization will bring us deep knowledge. It will bring us awareness of things that have been too data-intensive to get good answers from in the past," says Iliinsky. "It will get us answers more quickly, if we can collect and analyze the data more quickly than we have in the past. And it's going to show us areas that we may have overlooked." Take the case of Dataminr, an analytics company that has just announced a partnership with Twitter that will presumably give the company greater access to tweets and metadata. The company claims that its system got wind of Osama bin Laden’s death minutes before news media -- minutes that could have enabled the company’s financial clients to get ahead of major moves in the markets.

No one should expect social media analysis to replace surveys, existing early-warning networks, or traditional ethnographic research any time soon. Yet if these data visualization tools can make good on their promise, we'll soon have some powerful new ways of telling stories about our social universe with a speed and clarity that would have been hard to imagine just a few years ago. If we want to take a meaningful snapshot of the next iteration of Tahrir Square, reaching for a camera will no longer be our only option.



After Chávez, the Narcostate

There are powerful men in Venezuela who are far worse than Hugo Chávez. And if Obama keeps "leading from behind" in Latin America, that's who we very well might get.

Venezuelan leader Hugo Chávez has tried for 10 months to conceal the fact that he is losing his bout with cancer, determined to appear in command of his revolutionary regime and the nation's future. This past Holy Week, however, television cameras captured him pleading for his life before a crucifix in his hometown church, his mother looking on without the slightest glint of hope on her face. Chávez's raw emotion startled his inner circle and led some to question his mental health. As a result, according to my sources inside the presidential palace, Minister of Defense Gen. Henry Rangel Silva has developed a plan to impose martial law if Chávez's deteriorating condition causes any hint of instability.

Pretty dramatic stuff. So why isn't anyone outside Venezuela paying attention? Some cynics in that country still believe Chávez is hyping his illness for political advantage, while his most fervent followers expect him to make a miraculous recovery. The democratic opposition is cautiously preparing for a competitive presidential election set for Oct. 7 -- against Chávez or a substitute. And policymakers in Washington and most regional capitals are slumbering on the sidelines.

In my estimation, the approaching death of the Venezuelan caudillo could put the country on the path toward a political and social meltdown. The military cadre installed by Chávez in January already is behaving like a de facto regime determined to hold onto power at all costs. And Havana, Tehran, Moscow, and Beijing are moving to protect their interests. If U.S. President Barack Obama were to show some energetic engagement as Chávez fades, he could begin to put the brakes on Venezuela's slide, reverse Chávismo's destructive agenda, and reclaim a role for the United States in its own neighborhood. But if he fails to act, there will be hell to pay.

Sources close to Chávez's medical team tell me that for months, his doctors have been doing little more than treating symptoms, trying to stabilize their workaholic patient long enough to administer last-ditch chemo and radiation therapies. In that moment of Chávez's very public prayer for a miracle, he set aside his obsession with routing his opposition or engineering a succession of power to hardline loyalists. Perhaps he knows that his lieutenants and foreign allies are behaving as if he were already dead -- consolidating power, fashioning a "revolutionary junta," and plotting repressive measures.

One of them is longtime Chávista operator and military man Diosdado Cabello, who was installed by Chávez to lead the ruling party as well as the National Assembly in January. Cabello's appointment was meant to reassure a powerful cadre of narcomilitares -- Gen. Rangel Silva, Army Gen. Cliver Alcalá, retired intelligence chief Gen. Hugo Carvajal, and half a dozen other senior officers who have been branded drug "kingpins" by the U.S. government. These ruthless men will never surrender power and the impunity that goes with it -- and they have no illusions that elections will confer "legitimacy" on a Venezuelan narco-state, relying instead on billions of dollars in ill-gotten gain and tens of thousands of soldiers under their command.

Chavismo's civilian leadership -- including Foreign Minister Nicolás Maduro, Vice President Elías Jaua, and the president's brother, Adán Chávez, the governor of the Chávez family's home state of Barinas -- are eager to vindicate their movement's ideological agenda at the polls this fall. Maduro is extraordinarily loyal to the president, and is considered by Venezuelan political observers as the most viable substitute on the ballot. Above all, these men crave political power and will jockey to make themselves indispensable to the military leaders who are calling the shots today.

Cuba's Fidel and Raúl Castro are desperate to preserve the life-blood of Venezuelan oil that sustains their bankrupt regime. According to a source who was briefed on conversations in Cuba, Raúl has counseled Chávez to prepare to pass power to a "revolutionary junta"; Venezuelans who are suspicious of the Castros expect them to pack the junta with men loyal to Havana. Cabello does not trust the Castros, but with thousands of Cuban intelligence officers and triggermen on the ground in Venezuela, the Castro brothers are a force to be reckoned with.

The Chinese have provided more than $20 billion in quickie loans to Chávez in the last 18 months, which are to be repaid by oil at well below the market price. Most of these funds were paid into Chávez's slush funds before the Chinese knew of his terminal condition. Another $4 billion is being negotiated now, but my sources in the Venezuelan Foreign Ministry say the Chinese are demanding new guarantees. Beijing also is angling to ensure that any post-Chávez government will honor its sweetheart deals. However, these predatory contracts are being scrutinized by leading opposition members of the National Assembly.

Iran is more dependent than ever on its banks and other ventures in Venezuela as a means to launder billions in funds to evade tightening international financial sanctions. Companies associated with the Iranian Revolutionary Guard Corps, the Qods Force, and illicit nuclear and ballistic missile programs have invested millions in infrastructure in shadowy facilities throughout Venezuela. Tehran will struggle to keep its beachhead near U.S. soil, which is vital to its survival strategy in the critical months ahead.

Russia is considering making $1-2 billion in payments in the weeks ahead to lock in natural gas and oil deals signed with Chávez. Some in Moscow, however, are weary of the Venezuelan shakedown, particularly because they know that Chávez's days are numbered. Russian firms are deciding now whether to double down on the Chávez regime, which has been a reliable customer of more than $13 billion in Russian arms, or wait to see if a successor government will honor its agreements in the oil and gas sector.

The Soviet-style succession that corrupt Chavistas and their Cuban handlers are trying to impose on the Venezuelan people is anything but a done deal. There is room and time for friends of democracy to play a constructive role.

Cabello and company, my sources tell me, are far more likely to resort to unconstitutional measures and repression if they can count on support from Moscow and Beijing. The Chavistas intend to promise continued cheap oil and sweetheart contracts to leverage this support. Discreet U.S. diplomacy -- working in concert with like-minded allies -- can help scuttle these plans. The Chinese and Russians may not be eager to defend yet another violent pariah regime, and Washington should rally Latin American leaders to draw the line against a Syria scenario in the Western Hemisphere.

At the heart of the Chavista strategy is a narco-state, led by men with well-documented ties to narco-trafficking. The White House should instruct U.S. law enforcement agencies to smash the foundations of this regime. One Venezuelan general or corrupt judge in a witness box in a U.S. federal courthouse will strike the regime at the very top and destroy any illusion of legitimacy or survivability.

U.S. intelligence agencies have been virtually blind to the Iranian presence in Venezuela. If they were instructed to kick over the rocks to see what is crawling underneath, I am convinced that they would discover a grave and growing threat against the security of the United States and its allies in the region. Such evidence will help motivate Venezuela's neighbors to take a stand against an even more unaccountable regime taking shape in Caracas.

Venezuela's military is not a monolith, and Chávez has undermined his own succession strategy by giving the narco-generals such visible and operational roles. The fact that the narco-generals will be more willing to resort to unconstitutional measures and repression to keep power and carry the "narco" label sets them apart from the rank-and-file soldiers and institutionalist generals. The United States military still carries a lot of weight with these men. A simple admonition to respect their constitution and serve their people may split the bulk of the force away from the narcos and deny them the means to impose their will. (Institutionalist generals may react in a similar way to news that Iran is conducting secret operations on Venezuelan territory that are both unconstitutional and a dangerous provocation.)

There is much the United States and the international community can do without interfering in Venezuela's internal politics. Although the leaders of the democratic opposition are determined to keep their distance from Washington, they must at least show the flag in the United States and other key countries to elicit the solidarity they deserve. Moreover, anyone who thinks the opposition can take on Cuba, China, Russia, Iran, drug traffickers, and Hezbollah without international backing is just not thinking straight.

Unfortunately, the career U.S. diplomats in Washington responsible for Venezuela have spent the last two years downplaying the mess there and the three years before that neglecting it altogether. So if there is any hope for U.S. leadership, it will require the attention of Secretary of State Hillary Clinton or President Obama. Alas, in our own neighborhood, "leading from behind" is not an option.