It's not clear how many CDRs (each representing an individual) were in each of those files. But they were stored on the server for more than five years, past the cut off point at which the information is supposed to be destroyed, pursuant to NSA rules that are meant to protect the privacy of Americans.
How the records got there is a mystery. The report says they were "potentially collected" under business records orders, which are authorized by the Patriot Act. But that's not certain.
What is known, however, is that the records were stored with information that shouldn't have been anywhere near them. It came from the agency's highly classified Stellar Wind program, which covered the warrantless interception of phone calls and emails (not just their metadata) that was secretly authorized by President George W. Bush in 2001. Joining the CDRs and the Stellar Wind records was data from yet another program that was unrelated to the two.
Mixing or "co-mingling" information obtained from different programs, and under different laws or authorizations, is a dangerous practice in the intelligence profession. Information is segregated to restrict and monitor the number of people who have access to it. An analyst cleared to look at CDRs might not be authorized to listen to phone calls intercepted under Stellar Wind. But if it's all on the same server, he might be able to do just that.
That may have happened in 2011, according to the audit. Some personnel may have been granted access to a cache of information that was recently modified so that they were no longer allowed to look at it. But not all the employees were informed about the change.
Storing different intelligence streams in one place also increases the risk of revealing valuable sources and methods for how it was obtained--a basic violation of intelligence tradecraft. It also it makes it easier to steal. (Just ask Edward Snowden.)
And segregation creates a bulwark against privacy violations. Information about Americans is generally kept clear of foreign intelligence because the rules on how the former can be used and disseminated are stricter.
But infractions and mistakes weren't always reported to the NSA's overseers, either in Congress or at the Foreign Intelligence Surveillance Court. Partly that's because the NSA doesn't view unintentional or "incidental" collection of Americans' communications as a violation of the rules. It was an accident, the result of what the agency called in a previously declassified document "problems [that] generally involved the implementation of highly sophisticated technology in a complex and ever-changing communications environment..." Translation: Surveillance is hard. Our computers aren't perfect. We acted in good faith.
Not that the court can verify if that's true. In a candid admission to the Post, the chief judge, Reggie Walton, said he and his colleagues must "rely upon the accuracy of the information" the government provides, and that the court "does not have the capacity to investigate issues of noncompliance..."
In one case where the court did curtail a new kind of surveillance, it was only months after learning that it was put in place. The court deemed the still-undisclosed activities unconstitutional, and the NSA had to make changes before it could restart them.
The NSA is also instructing its employees not to provide full information about infractions to Congress, which is supposed to oversee intelligence collection efforts and ensure they comply with the law.
The newly released documents affirm something we've long known: the NSA gathers up large amounts of information on foreigners and U.S. citizens and then tries to separate the proverbial wheat from the chaff, with imperfect results. That's alarming, but from a technological standpoint, understandable.
What members of Congress and the public may find more troubling is that the NSA wasn't honest about these shortcomings. Officials hid them from the same judges and lawmakers that President Obama recently said were engaged in a rigorous process of checks and balances that keeps electronic spying within the bounds of the law.
Perhaps that system, like the NSA's data vacuums, could use a tune up.