National Security

The NSA Can't Tell the Difference Between an American and a Foreigner

That's why it sucks up information on everyone.

The National Security Agency has said for years that its global surveillance apparatus is only aimed at foreigners, and that ordinary Americans are only captured by accident. There's only one problem with this long-standing contention, people who've worked within the system say: it's more-or-less technically impossible to keep average Americans out of the surveillance driftnet.

"There is physically no way to ensure that you're only gathering U.S. person e-mails," said a telecommunications executive who has implemented U.S. government orders to collect data on foreign targets. "The system doesn't make any distinction about the nationality" of the individual who sent the message.

While it's technically true that the NSA is not "targeting" the communications of Americans without a warrant, this is a narrow and legalistic statement. It belies the vast and indiscriminate scooping up of records on Americans' phone calls, e-mails, and Internet communications that has occurred for more than a decade under the cover of "foreign intelligence" gathering. 

The NSA is routinely capturing and storing vast amounts of the electronic communications of American citizens and legal residents, even though they were never individually the subject of a terrorism or criminal investigation, according to interviews with current and former intelligence officials, technology experts, and newly released government documents.

A significant portion of this secret information-gathering is the result of so-called "incidental collection" of U.S. persons' information; Americans' communications just happen to be in the way when foreigners' data is scooped up.

This incidental collection is partly the result of the way the global communications network is constructed. When the agency receives authorization from the Foreign Intelligence Surveillance Court to collect a broad range of e-mails or electronic communications that it believes are coming out of a foreign country, it's inevitable that it will collect some U.S. persons' information, too.

"There are U.S. persons in every country," said a former intelligence official. "The NSA knows that when it collects great gobs [of communications] there are going to be U.S. persons in that country. They know that happens."

But new documents reveal that the NSA has also deliberately gathered communications metadata that it had reason to believe was associated with Americans.

On Thursday, the Guardian reported that NSA had been collecting vast amounts of e-mail data in bulk, stemming from a secret program that was first authorized by President George W. Bush soon after the 9/11 attacks.

The Guardian also disclosed a November 2007 memorandum prepared for then-Attorney General Michael Mukasey by Kenneth Wainstein, who was in charge of the Justice Department's National Security Division. On behalf of the NSA, Wainstein requested that the attorney general approve a powerful form of computer-assisted analysis of U.S. persons' metadata, including their phone and e-mail records, as well as Internet Protocol addresses of individual computers. This information was obtained "by various methods, including pursuant to the Foreign Intelligence Surveillance Act," the memo states.

"NSA has in its databases a large amount of communications metadata associated with persons in the United States," the memo states.  

NSA wanted to subject this large store of metadata to a form of link analysis known as contact chaining, in which an analyst starts with a particular phone number, e-mail address, or Internet Protocol address, and then uses algorithms to find the corresponding communications to which the "seed" target is linked. Contact chaining also finds the communications to which that first layer of communication is linked. Each one of these steps outward in the original target's network is sometimes called a "hop."  In just a few hops, the number of individuals swept up in the analysis multiplies exponentially.

The memo states that the NSA had already been conducting contact-chaining, but that based on the "informal advice" of the Justice Department office that represents the government before the FISA court, "NSA's present practice is to 'stop' when a chain hits a telephone number or address believed to be a United States person." The agency wanted to keep going, however, even when it encountered communications believed to belong to Americans and legal residents. The hope, the memo states, was that by chaining through "all telephone numbers and addresses," the NSA would "yield valuable foreign intelligence information primarily concerning non-United States persons outside the United States."

In effect, the NSA was arguing that it needed to see everyone's metadata in order to find meaningful information about foreigners. Mukasey approved the new contact chaining procedures.

In the memo, Wainstein argued, as other government officials have over the years and continue today, that metadata is not content, and therefore is not subject to protections under the Fourth Amendment. Nevertheless, technology experts say that metadata can reveal deep and meaningful information about who a person knows, where they go, and what they are doing, both online and off. (It's worth remembering that the U.S. government authorizes lethal U.S. drone strikes based on a target's associates and movement -- an analog version of metadata--and that information about those foreign terrorists and their associates is gathered using FISA.)

The memo also asked Mukasey's permission to give metadata on U.S. persons directly to the Central Intelligence Agency and other Defense Department "entities." It doesn't elaborate on what those organizations were doing with the data or why they wanted it.

The Guardian, citing a senior Obama administration official, reported that the intentional collection of Internet metadata was stopped in 2011. However, the paper found that "it is clear that the [NSA] collects and analyzes significant amounts of data from U.S. communications systems in the course of monitoring foreign targets."

When the agency collects the communications of Americans, it is supposed to follow a set of minimization procedures designed to protect individual privacy and keep innocent Americans from being implicated in terrorism investigations. But first, the agency has to determine if, in fact, the sender of a particular message is a U.S. person.

That's hard to do. The breadth of global communications, and the digital mixing of messages from all corners of the world, can make it difficult to know with precision who is being targeted, and where that person is located, without subjecting a particular email to closer inspection.

According to former intelligence officials, the NSA routinely opens e-mails and reads their contents to determine if the sender was a U.S. person. Reading that message doesn't require the agency to obtain a warrant, and if an analyst discovers that the communication belongs to a U.S. person, he is supposed to destroy it if it has no intelligence value and does not contain information about a crime. But the NSA's guidelines allow the agency to hang onto this information for up to five years before trying to determine its origin.

"I think it's important to understand that there are certain things that the government is doing that by their very nature are going to involve vast amounts of information about Americans, even if that's not their intent," said Chris Soghoian, an expert on privacy and technology at the American Civil Liberties Union.

In congressional testimony earlier this month, Gen. Keith Alexander, the NSA Director, discussed two programs that had recently been disclosed in press reports: The NSA's collection of telephone metadata in the United States, and the system known as PRISM that gives the agency access to information from Internet companies including Google and Facebook.

"These programs are limited, focused, and subject to rigorous oversight," Alexander said. "They have distinct purposes and oversight mechanisms. We have rigorous training programs for our analysts and their supervisors to understand their responsibilities regarding compliance."

Alexander did not address the collection of Internet metadata that began under the Bush administration, nor did he discuss the 2007 memo, which had not yet been disclosed. Current and former intelligence officials stressed in interviews that agency employees are trained to follow specific rules and procedures when handling U.S. person data, and that in light of recent revelations they have become more cautious.

Precisely how much U.S. person data is being collected in the course of spying on foreigners has been a subject of considerable debate, but clearly it has been large. In 2009, the New York Times reported a "significant and systematic" collection of Americans' emails and phone calls during the course of searches authorized by the Foreign Intelligence Surveillance Act.

The NSA has avoided saying how much data on U.S. persons it is collecting, even though it appears to have a way to find out. Last year, the NSA told a pair of senators looking into the issue that the agency could not estimate how many Americans' communications had been collected, in part because it would "violate the privacy of U.S. persons" to try answer the question. That implied that those communications were stored somewhere and accessible, but that reading them to see who was the sender would effectively constitute a search under the law.

Former officials contacted for this story were also reluctant to say how many Americans' communications were incidentally collected during broad FISA searches. But they suggested that the number was large and knowable.

Among the U.S. person communications that the agency may retain, even though they weren't directly targeted, are those "acquired because of limitations on NSA's ability to filter communications," according to a set of procedures that the agency uses to minimize the intrusion into Americans' privacy. The document was disclosed last week by the Guardian.

"They do know that U.S. person data will get through. They admit that," the former intelligence official said with respect to this provision in the rules. Sometimes a communication may slip through the filters because it's encrypted and the system cannot scan it for keywords that might help determine the nationality of the sender. Or, the NSA could be collecting information at such a high volume that's practically impossible to filter every message. "They don't listen to everything and process everything," the former official said. "Sometimes they may keep it and look at it later." 

When there's a question about the sender's nationality or location, a human analyst steps in and examines the content of the communication, former officials said. One former analyst said this only happens if there's some indication that the communication is suspect. For instance, a known terrorist is communicating repeatedly with someone who is not yet on the agency's radar.

There appear to be some high-level controls on how much U.S. person data the NSA gathers inadvertently, but they are relatively crude. The former intelligence official said that when the government asks the FISA court for the authority to collect communications from a particular cable, it estimates based on historical information and geography how likely it is that most of the data moving on that cable will be coming from foreigners. The court is not likely to approve broad surveillance on a cable that contains a "significant" amount of U.S. person data, the former official said.

How can the NSA know? A fiber optic cable routing traffic out of Saudi Arabia, for example, is likely to contain mostly foreigners' communications. However, network routing is dynamic, and can change day to day. If, for instance, that same line was suddenly getting traffic from Malta, where there's likely to be a larger number of U.S. persons, the NSA can block the Maltese traffic, the former official said. If that happened, the agency is required to inform the FISA court and describe the steps they took to filter out those communications.

Soghoian, the ACLU technology expert, said that if the NSA were tapping into undersea cables emanating from foreign countries, the likelihood of them containing U.S. persons data would be low. The likelihood increases, however, if those cables were located in the United States, where foreign data would be mingling with Americans' communications. Using the PRISM system, the NSA collects electronic communications from service providers such as Google and Facebook that are based in the United States and use equipment here.

A U.S. person is also more likely to have his communications intercepted if he's communicating with someone overseas, Soghoian said. But Americans who only talk with other U.S. persons can be caught in the driftnet, too -- in part because of the NSA's push into so-called cloud computing.

The NSA's impulse to collect more information has been encouraged by the agency's investments in big data and distributed databases. The agency bet big on Hadoop, a piece of open source software that allows massive amounts of data to be both stored and processed across a seemingly unlimited array of computers. It also lets that data sit on servers uncharacterized until the nanosecond an analyst needs the information. In other words, NSA doesn't have to drop its information into discrete compartments like "foreigner" or "American." The data can be stored, and those characterizations can be made later. This is a great advantage for the agency: It's slurping up billions of records but doesn't have to make sense of them all at once.

The NSA also reverse-engineered Google's most important database, layered it on top of their Hadoop-based system, and added inventive security controls. Older databases can be divided like spreadsheets into rows and columns; analysts can be authorized to access the data from a given column or a given row. The NSA's database, called Accumulo, allows for much more fine-grained permissions; a single cell -- the intersection of a row and column -- can be hidden from an analyst. And even if it is hidden, and analyst can still use that data (even if he can't see it) to help him spot trends and build models.

In his recent testimony, Gen. Alexander said that individual NSA analysts don't have the authority to read someone's e-mails or listen to his phone calls. But with Accumulo and Hadoop, it doesn't matter. Americans' information can be used anyway.



The Inside Story of Russia's Fight to Keep the U.N. Corrupt

From bullying out reformers to blocking efforts to save millions.

When U.N. Secretary-General Ban Ki-moon and Russian President Vladimir Putin met in Sochi, Russia, they were supposed to discuss the civil war in Syria. But the Russian leader -- joined by his top diplomat, Sergei Lavrov, and defense secretary, Sergei Shoigu -- suddenly changed the subject to more mundane matters. A series of U.N. reforms aimed at streamlining billions of dollars of spending on U.N. peacekeeping was posing a threat to Russia's commercial interests. Putin and his national security team politely but firmly pressed the U.N. leader to back off, according to several senior U.N.-based sources briefed on the meeting.

The high-level intervention on U.N. spending marked only the latest example of Russia flexing its diplomatic muscle to protect its commercial position at the United Nations. For much of the past decade, Russia has been engaged in a systematic effort to stymie attempts to root out corruption in U.N. spending. The Russians have pushed out U.N. reformers. They've defanged watchdogs. And they've blocked internal budget reforms aimed at saving costs.

Russia's zeal for turning back reform has been felt most powerfully in the U.N.'s leasing of aircraft -- a $1 billion a year market -- that provide transport for the world's second-largest expeditionary force. An examination of U.N. procurement practices in the air-transport sector -- drawing on dozens of interviews with U.N.-based officials and diplomats, as well as a review of internal U.N. communications and audits -- suggests that Russia has enjoyed unfair advantages, including contracts that all but demand that the United Nations lease Russia's Soviet-era aircraft.

The dispute provides a textbook example of the difficulties of implementing basic financial reforms at the United Nations when major powers have conflicting commercial interests in the outcome. As such, the secretary general and key countries have been unwilling to openly confront Russia because its cooperation is required on a wide range of critical issues at the United Nations.

Since the end of the Cold War, Russian entrepreneurs have turned the Soviet-era air fleet into a thriving business, supplying the U.N. and other international agencies with low-cost surplus aircraft, including Antonov transport planes and Mi-8 and Mi-26 helicopters. The low-cost aircraft -- which Russian factories continue to produce -- have largely dissuaded Western air operators from competing for U.N. contracts, which must go to the lowest bidder. Russian companies now account for about 75 percent of all contracts for commercial helicopters, the most lucrative segment of U.N. peacekeeping's multibillion-dollar marketplace.

But the near Russian monopoly is facing challenges from neighbors such as Ukraine, which produces similar helicopters. The United States and European powers like Germany, France, Italy, and Spain are also looking for new business opportunities as the NATO mission in Afghanistan winds down. Those countries have privately raised concern with the U.N. about the integrity of its procurement process. They claim that the U.N.'s purchasing system is rigged to favor Russian aircraft; its bidding specifications -- for instance, requirements of seating capacity for more than 20 passengers -- are tailored to exclude most competitors. "Procurement is done in a way which directly specifies a Russian helicopter," said one senior European diplomat. "We have asked for more transparency; we want to change to a new [bidding] system as soon as possible."

Requests for helicopters and transport planes originate from the U.N.'s 15 peacekeeping missions and are routed through headquarters' air-transport section before being sent on to the U.N. procurement department, which invites companies to bid. Western diplomats have expressed concern that many of the key players -- including a Ukrainian procurement chief and a Russian aviation specialist -- come from countries with a major stake in the aircraft market.

But a spokesman for the U.N. peacekeeping department, Kieran Dwyer, dismissed those concerns. "The secretariat has a system of management checks and balances that mean that no one individual can unilaterally set the procurement specifications for aviation requirements," he said. "It is true that helicopters from the Mi-8 family of aircraft do play a leading role in peacekeeping aviation assets and operations. These helicopters have key features which make them suitable to peacekeeping needs, including their flying range and payload capacities and the fact that they are economical."

Despite Dwyer's claim, the U.N.'s internal corruption watchdog, the Office of Internal Oversight Services, said that the failure to open up the bidding to a broader range of aircraft has exposed the U.N. to a "high risk of acquiring air charter services at a higher cost than necessary," according to a confidential internal audit.

The U.N. audit, which was obtained by Foreign Policy, bears out some of the concerns voiced by Western powers and flags the risks of possible collusion among helicopter providers. The December 2012 audit notes that the U.N.'s largest helicopter vendor has competed for contracts against wholly owned subsidiaries, a practice that "further emphasizes the urgent need for measures to mitigate the risk of collusion. "

The audit also expresses concern that U.N. aviation officials were drawing up bids "in a manner that can often be associated with certain aircraft types and models." The practice, according to U.N. officials and diplomats, effectively eliminates potential competitors who might be able to fulfill the terms of the contract with different types of planes and helicopters.

The audit does not mention which aircraft get preferential treatment. Nor does it name the favored vendors or identify their nationalities. But it does raise concern about the fairness of the U.N. bidding process, which, for instance, fails to measure fuel efficiency in determining a helicopter's cost. It's a lapse that favors older, cheaper, but less fuel efficient helicopters. That, according to diplomats, gives Russian operators, with their aging fleets, an unfair advantage.

Several years ago, the U.N. launched an effort to enact a series of procurement reforms. One idea was to replace the practice of issuing vendors "invitations to bid" -- which sometimes specify the particular aircraft being sought -- with "requests for proposals," which define the U.N.'s general needs and allow helicopter operators the freedom to propose their own solutions using a wider variety of aircraft. "There are many cases where different types of flying equipment can perform the required tasks," according to a confidential review of U.N. bidding practices by the International Civil Aviation Organization. "A tender process requesting offers only for a specific type of equipment does not allow maximization of choice in the selection process."

Russia has vigorously opposed the United Nations' plans to change its bidding procedures, one of a series of steps the country has taken to block changes in U.N. procurement. In 2012, Russia sought to force out an aviation specialist who had been transferred to New York to try to strengthen oversight of the United Nations' helicopter leasing practices. Russia seized on an internal audit that criticized the official's management of air operations in Africa. (The official's alleged misdeed? Hiring a local aircraft for a mission in sub-Saharan Africa instead of leasing it through headquarters. According to a Western diplomat, the move not only saved money -- it filled an urgent requirement.)

For several years, the Russian government has also dragged out negotiations in the U.N. budget committee aimed at implementing the U.N. chief's procurement reforms, according to senior Western diplomats. Russia's U.N. envoy, Vitaly Churkin, said his government has no objections to reforming the U.N.'s buying practices, but he sees a raft of reforms as a direct threat to Russia's commercial interests at the United Nations. "Generally speaking, we are a little bit concerned about the number of reforms," Churkin said in an interview with FP. "We don't mind the competition. We understand that business is about competition. We don't want monkey business."

In May, Secretary-General Ban traveled to Sochi, Russia, to meet with President Putin and his top diplomat, Lavrov, to forge a diplomatic strategy for ending the war in Syria.

But the conversation quickly segued into a discussion of Russian misgivings over procurement matters. The Russian leadership was especially alarmed by a plan to delegate authority for leasing helicopters to U.N. field missions and logistical hubs in Entebbe, Uganda, and Brindisi, Italy, a move that would limit the ability of Moscow's powerful U.N. headquarters delegation to monitor and influence decisions.

The Russians also objected to the plan to upgrade the bidding process. They pushed back on a plan to promote "staff mobility," a key element of Ban's reform effort. The initiative is aimed at offering U.N. officials a greater range of experiences and skills by having them periodically serve in the U.N.'s far-flung missions. Several diplomats said that the Russians are concerned that the move would dilute Russia's influence in New York and potentially force a Russian national who is involved with drawing up helicopter specifications to rotate out of his job.

Churkin said that Russia is worried that the U.N. reforms pose a direct threat to its legitimate commercial interests and that the initiative to decentralize helicopter operations may undercut fair competition. 

Russia has previously been the focus of concern about irregularities in U.N. procurement. In 2006, the U.N. established a Procurement Task Force to look into allegations of corruption within the United Nations after Alexander Yakovlev, a U.N. purchasing officer from Russia, pleaded guilty to U.S. federal charges that he received hundreds of thousands of dollars in bribes from companies doing business with the world body. The case led to the conviction on similar charges of Russian diplomat Vladimir Kuznetsov, who led the chief U.N. budget committee.

Although the task force -- which was headed by former Connecticut prosecutor Robert Appleton -- played no role in prosecuting Yakovlev or Kuznetsov, it has conducted several investigations into their roles in the scheme, earning the ire of Russia and other U.N. member governments. In 2008, Russia proposed a resolution that would have forced Appleton and his team out of the United Nations. Although the measure failed to pass, Appleton was ultimately forced out of the United Nations by Ban, blocked his appointment as the U.N.'s chief of internal investigations on a technicality: No female candidates were included on a shortlist of candidates. The U.N.'s then top anti-corruption official, Inga-Britt Ahlenius, who had tried to hire Appleton, resigned over the matter and accused Ban of undercutting her independence.

In the following years, the U.N.'s capacity to police itself has suffered, and the internal financial controls have not performed up to expectations, particularly in aviation. "The secretariat's governance, risk management and control process examined were unsatisfactory in providing reasonable assurance regarding the efficient, cost effective and timely acquisition and management of air charter services agreements," according to one U.N. audit. Russia has resisted efforts to modernize the U.N.'s air fleets with greener technologies, upgraded safety systems, and more fuel efficiency.

The issue came to a head after a Russian Mi-8 slammed into a mountainside in the Democratic Republic of the Congo. Following the crash, a U.N. aviation official at the U.N.'s headquarters sent an email requiring that U.N. helicopters be upgraded with a safety device -- known as an enhanced ground proximity warning system -- that relies on a digital terrain mapping system to detect large physical objects, including buildings and mountains, in bad weather. But a senior U.N. official subsequently overruled that decision, saying that the U.N. safety review had not yet determined whether the system would be mandatory or not.

The question circulating among the U.N. diplomatic community is whether the U.N. backed down under Russian pressure. Russian carriers have not installed the devices in their helicopters, and the leading Russian operator, UTair, was competing for a multimillion-dollar contract for three helicopters for the Democratic Republic of the Congo. A chief competitor, Ukraine, has been equipping its helicopters with the safety systems. In the end, UTair came in as the lowest bidder, making it highly likely that it will formally win the contract.

A spokeswoman for UTair, Elena Galanova, wrote FP by email to say that the Russian Mi-8 and Mi-26 helicopters it provides to U.N. missions are equipped to faithfully comply with any of the U.N.'s requirements. But she said that UTair would be prepared to install the new warning system if the U.N. demanded it, but she noted that the warning system "is in fact not a mandatory requirement."

Russia's U.N. envoy, Churkin, said that Moscow is also committed to embracing modern technologies and safety features, but not if progress is a cover for seeking an unfair edge in the marketplace.

"We think we can continue to be very competitive if things are done fairly, but if there is going to be an effort to avoid fair competition," he added, "that's going to cause a problem for us and for the image of the United Nations."