America's Other Army

Interviews with diplomats in the line of fire -- an exclusive excerpt from the new book America's Other Army: The U.S. Foreign Service and 21st Century Diplomacy.

The mob that had gathered at a soccer stadium descended on the U.S. Embassy in Belgrade, determined to avenge Washington's recognition of Kosovo -- a Serbian province until five days earlier -- as an independent state. On that day in February 2008, the Serbian riot police stationed in front of the embassy at the request of U.S. Ambassador Cameron Munter conveniently vanished just before the hundreds-strong horde arrived. "The police marched away, got on buses, and drove away, so when the hoodlums came there was no one there," Munter recalled.

A part of the embassy was soon ablaze. "One of the protesters who was drunk managed to get in and burned himself to death," Munter said. Several others climbed the fence. The U.S. Marines guarding the compound had every right to shoot, but they managed to drive the intruders away with warnings and instructions instead. "I was very impressed that the Marines knew how to make judgment calls as well as to be defenders," Munter, a Foreign Service officer since 1985 and until recently the U.S. ambassador to Pakistan, told me in an early 2012 interview in Islamabad.

U.S. diplomats saw the embassy attack coming. And as a result of their preparations, no Americans were hurt during the incident. Only a small crew, including the ambassador, was still in the building at the time of the assault. As soon as the protesters tried to penetrate the compound, some of the Americans began destroying millions of dollars' worth of communications equipment, which is a standard procedure in such cases, Munter said. The next day, about three-quarters of the embassy staff and all family members were evacuated out of Belgrade. "We were fairly sure there would be an angry reaction" to the recognition of Kosovo "and had made all necessary preparations," he said. "We had already arranged for hotel rooms and space at the embassy in Zagreb," the Croatian capital. "We even had space at the American school in Zagreb for our kids."

Most diplomats, however, aren't so lucky. Preparing for a specific attack on a U.S. diplomatic facility overseas is almost never possible, and evacuations are rarely as orderly as the one in Belgrade. The events in Libya on the night of Sept. 11 this week were a tragic reminder of that reality. When the U.S. Consulate in Benghazi came under attack, Ambassador Christopher Stevens was trapped in the burning building and reportedly died of smoke inhalation. Three other U.S. officials were also killed during the assault. A breach of the U.S. Embassy in Cairo that same day did not result in any deaths, but the incident significantly heightened tensions between the White House and the Egyptian government.

Munter, who didn't abandon his embassy in Belgrade in 2008, wanted those responsible for the attack to be punished. Once evidence surfaced that Serbian Prime Minister Vojislav Kostunica had personally approved the assault, Munter decided that he was "going to ensure the prime minister was gone" and that "the best revenge was making sure this guy lost the next election," which was less than five months away.

Munter determined that the key to weakening Kostunica's 2008 reelection chances was taking away the support of the Socialist Party of Serbia, once led by former Serbian President Slobodan Milosevic. Its new leader was Ivica Dacic, who had once challenged Milosevic for the top post. "We got him to flip over and join the pro-Europeans," Munter said. "We didn't pay him off; we just persuaded him. What he really wanted was international legitimacy. So we got [José Luis Rodríguez] Zapatero, the Spanish prime minister at the time, and George Papandreou, the future Greek prime minister, who ran Socialist International at the time, to invite Dacic to visit them abroad, where they wined and dined him. They told him they would let him in [to the Socialist International] if he joined the pro-European forces, and he did. He put a knife in Kostunica's back."

Munter got his revenge: Kostunica's party lost the election. Dacic's party didn't join Socialist International, the global organization of left-of-center political parties, but he became deputy prime minister and rose to prime minister four years later.

Like Stevens and thousands of other U.S. diplomats, Munter has served in places much more dangerous than Serbia, including Iraq and Pakistan. As he prepares to retire from the Foreign Service after 27 years, the longtime diplomat would like Americans to know that modern diplomacy is not all glitz and glamour.

While U.S. diplomats still spend time in the company of kings and queens, presidents and prime ministers -- and remain some of the most sought-after people in foreign capitals -- they also contend with security threats, the stresses involved in constant relocation, and the challenges of being a jack-of-all-trades in a foreign land. I have interviewed some 600 Foreign Service members at more than 50 embassies and consulates in the course of research for my new book, America's Other Army. While most said they could not imagine doing anything else, many also added that being constantly on the move and far from home means giving up much of what most Americans take for granted.

American diplomats risk their lives just by showing up for work every day. During my travels researching the book, I heard many stories about carjackings, kidnappings, robberies, and diplomats being held at gunpoint. Some have been murdered. On New Year's Day in 2008, John Granville, a 33-year-old officer with the U.S. Agency for International Development (USAID), was fatally shot in Sudan while returning from a holiday party at the British Embassy. In 2002, Laurence Foley, 60, also with USAID, was gunned down in front of his home in Jordan. In 1968, John Gordon Mein, 54, became the first U.S. ambassador to be assassinated while in office when he was shot by rebels in Guatemala. Stevens in Libya was the first ambassador to be killed in the line of duty since 1979, when Adolph Dubs died in Afghanistan.

Philip Frayne, a Foreign Service officer whom I first met in Cairo in 2003, said he was driving to a meeting in Yemen in 1993 when his embassy vehicle was carjacked by "three guys with Kalashnikovs" pointed at him. "I asked them in Arabic if I could get my bag from the back seat, but by then they were already in the car and driving away," Frayne recalled. "About six months later, someone from the embassy security office saw the car in the parking lot of the presidential palace -- the diplomatic license plates hadn't been changed. I don't think it was taken by the president's men, but it was probably taken by tribesmen and later traded or confiscated by the presidential forces." That same year, Frayne's boss in Yemen, public affairs officer Haynes Mahoney, was kidnapped for a week.

Laura Clerici, a now-retired officer I met in Mexico City in 2003, said she was "ambushed by bandits" in Guatemala in the late 1970s while driving with three colleagues and six children. "When they saw that the guy from the defense attaché's office had a handgun, they started shooting at us," Clerici said. "Fortunately, all they wanted was our money, but when we got back, I absolutely fell apart." Still, work in such places gave "richness to my life that would have never happened, no matter what I had done in Washington," she said. "I've had one of the most exciting lives anybody could possibly want."

Top diplomats are the U.S. government's version of firefighters: They move around the globe rapidly, from flash point to flash point. Just when they have managed one crisis, it's on to the next assignment. Two years before taking up his post in Serbia, for instance, Munter had led a team that taught Iraqi provincial authorities how to govern effectively and trained local judges how to conduct trials and other court proceedings -- amid frequent shootings, roadside bombs, and rockets.

U.S. diplomats are charged with an extraordinarily diverse array of assignments -- and they have the stories to prove it. In 2008, around the time Munter worked to evacuate the Belgrade Embassy, Yuri Kim was in North Korea. She accompanied the New York Philharmonic during a rare concert tour in the communist country. Kim had helped negotiate the unprecedented visit, which Washington hoped would improve Pyongyang's cooperation in efforts to dismantle its nuclear weapons program. She was involved in those efforts as well. This year, now in Turkey, Kim tried to persuade Ankara to use its influence over Syrian President Bashar al-Assad to end the Damascus regime's violent crackdown on anti-government protesters.

Also this year, in another one of Turkey's neighbors, Iraq, David Lindwall worked on deals to sell U.S. military aircraft and other equipment worth billions of dollars to the government in Baghdad. A few years earlier, on the other side of the world, Lindwall had participated in successful efforts to reform Guatemala's previously corrupt child-adoption system, which many Americans use, and to improve child nutrition. In Haiti in 2010, he managed the search for missing U.S. citizens after the country's devastating earthquake and helped the local government recover from the disaster. Lindwall's house collapsed from the seismic shock -- had he been there at the time, he would have certainly been crushed to death

Gavin Sundwall's first time in a Panamanian jail was in 1998. Two Satanist killers sat across from him -- one staring at Sundwall with menacing green eyes, as if sizing him up for execution. Fortunately for Sundwall, he was just visiting the criminals, who were U.S. citizens, to make sure they were being treated humanely and to relay any messages to their families back home. Early this year, Sundwall, now in Kabul, worked to put out major public relations fires after U.S. service members burned copies of the Quran and another soldier killed 16 Afghan civilians, mostly women and children, in cold blood.

It's hard to imagine how all these people can be diplomats. How can teaching effective governance, participating in nuclear negotiations, organizing a cultural event, reforming a child-adoption system, selling weapons, recovering from a natural disaster, visiting prisoners, and fixing public relations problems be part of the same profession?

Welcome to the U.S. Foreign Service.

Given the United States' expansive global role, it should come as no surprise that America's diplomats have to take on an ever-wider variety of tasks during their careers. The most recent U.S. National Security Strategy lays out the core national interests that an American diplomat is charged with upholding -- the security of the United States, the country's prosperity, and the values it stands for (human rights, democracy, and equality). These interests are so fundamental that there is usually political agreement on them regardless of which political party is in power. Another document, the first-ever Quadrennial Diplomacy and Development Review issued by Secretary of State Hillary Clinton in 2010, elaborates on the importance of the international system to U.S. interests. It says that to ensure the security of the United States, the entire world has to be secure and stable because today's threats, such as terrorism, transnational crime, climate change, and pandemic disease, are "global, interconnected, and beyond the power of any one state to resolve."

"We cannot expect to be protected by our geographic position, which historically has been such an advantage for America -- I think Sept. 11 demonstrated that conclusively," Clinton told me. "In order to maximize the chances that we will enjoy security and tranquility here at home, we have to be in effect the chairman of the board of the world -- to try to get friends and allies to work with us, to mitigate problems, to bring about solutions that neutralize or prevent nonstate actors, as well as rogue states, from taking actions that put the lives and property of our people and our friends and allies at risk."

Got that? For the United States to be truly secure and prosperous, the whole world has to be secure and prosperous -- and that is "the world we seek," according to the National Security Strategy. At the same time, the White House recognizes "the world as it is" and acknowledges that the U.S. government must deal with it. This is where the U.S. diplomats come in: It's their job to reconcile the sometimes contradictory goals of protecting American interests in the short term while also -- somehow -- working to reshape the world into a more secure and prosperous place for future generations.

Munter's mobile phone rang at 3 a.m. Sleep was the last thing on his mind, so he answered the call. "We hear there has been a helicopter crash. Was it one of yours?" the high-ranking Pakistani official asked. Munter, now the U.S. ambassador to Pakistan, could not answer the question -- not because he was uninformed, but because he was sworn to secrecy, at least for another few hours.

It was May 2, 2011, and Munter had just watched a live video feed of the raid that killed the world's most notorious terrorist, Osama bin Laden, just 30 miles from Munter's location in Islamabad. But he was not allowed to say a word about it to the Pakistani government -- as had been the case for months -- until the official announcement by President Barack Obama in Washington.

Keeping the secret had been easy compared with Munter's next challenge -- repairing the already tense U.S.-Pakistan relationship. Those ties had already been strained to their breaking point about three months earlier, when an undercover CIA contractor, Raymond Davis, fatally shot two Pakistanis in what he claimed was self-defense. In the days after bin Laden's death, it fell on Munter and his colleagues at the embassy to explain to the Pakistani government and military why they had been kept in the dark about a foreign military operation on their own territory.

Several days after the operation, Munter was in the office of Gen. Ashfaq Parvez Kayani, the Pakistani Army's powerful chief of staff. Kayani was seething. Munter had just handed him a démarche from Washington with "a list of things" the Obama administration wanted the Pakistani Army to do in the wake of the bin Laden raid. The U.S. government felt that now it had more leverage to secure better cooperation from the Pakistanis in the fight against extremists hiding in Pakistan, Munter said. The Pakistani government, having been kept in the dark, was still stunned by the raid, and Kayani knew that accepting the U.S. demands would be seen as capitulating to the Americans.

So he "tossed the piece of paper" at the ambassador and asked him to leave. "I've rarely been insulted to my face as a diplomat, but he just threw me out of his office," Munter said. "'Is this the way you treat people when they are down?" he recalled the general asking. "He is not a rude man, but it was about as rude as he gets."

A U.S. diplomat's relations with his host country do not have to be good at any cost -- but they should at least be business-like and respectful, so the two countries can work together when necessary. The key ingredient in any relationship, of course, is trust. That was exactly what Munter had tried to build with Kayani for months before the bin Laden raid -- not for the sake of having a good relationship, but because the Pakistani Army's cooperation could help save American lives. Thousands of al Qaeda militants are believed to be hiding in Pakistan's border areas with Afghanistan, posing a threat to the U.S. troops on the other side of the border. Because al Qaeda "can attack the homeland," it is "vital" to eliminate the threat, Munter said.

The bin Laden and Davis episodes are telling examples of how short-term U.S. goals in a foreign country can clash with the United States' long-term mission. Similar examples can be found around the world, including in Russia and China, where Washington has to play a careful balancing act between its advocacy of human rights and democracy, and the help it needs from Moscow and Beijing to address strategic issues, such as the North Korean and Iranian nuclear programs.

"I would do the bin Laden operation exactly the same way again," Munter said. "It was worth it, and the president had made the calculation that it was worth it. But let's not kid ourselves: It did damage the relationship very badly. There is a set of assumptions when you build a relationship, and when you talk with people, there has to be someone willing to listen. When that collapses, running an embassy and trying to get things done is very difficult, which is why this job is the hardest I've had in the Foreign Service by orders of magnitude."



Dark Soldiers of the New Order

The Soviet Union's spies haven't disappeared, they're just wearing new clothes. An exclusive excerpt from Edward Lucas's new book, Deception.

The cold breath of the Communist secret police state blighted countless lives behind the Iron Curtain. But it also touched my own childhood in 1970s Oxford. Olgica, our Yugoslav lodger, had an exciting secret: Uncle Dušan. The poet Matthew Arnold described Oxford as "home of lost causes, and forsaken beliefs, and unpopular names, and impossible loyalties." In Dušan's case, this was partly right: his surname (like those of most east European émigrés) if not exactly unpopular, was certainly baffling to British eyes and ears. I recall him as a glum, shadowy figure, with plenty to be glum about. His cause seemed irretrievably lost. He was a hero in his own twilight world, but in post-war Yugoslavia, the authorities denounced anti-communists like him as criminals and traitors. Many perished in mass graves or in the torture cells of the secret police. Dušan was one of the lucky ones. He had escaped to Britain, to a humble job as a mechanic and life in Crotch Crescent, a drab street in Oxford's outskirts -- a sad comedown for someone who in pre-war Yugoslavia had been a high-flying young civil servant.

But in one respect, Dušan did not fit Arnold's dictum. Despite his disappointments, he had not forsaken his beliefs: communism was evil and the people who ruled his homeland were usurpers. In fact, Yugoslavia's independent-minded communists had become mild by comparison with the much tougher regimes of the Soviet bloc. But they were still ruthless in their treatment of dissenters, particularly those with contacts with anti-communists abroad. Olgica's family maintained, at great risk, secret links with relatives abroad, flatly denying all knowledge of them under interrogation from the secret police. In Oxford, she visited her uncle each weekend. Had the authorities at home known that she was hobnobbing with a dangerous anti-communist émigré, her father's glittering medical career (which had even brought him, briefly, to Oxford) would end; her own future (she had stayed on to finish her schooling) would be jeopardized too. It could even be dangerous for her to return home, leaving her stranded in Britain as a teenage refugee.

My own childish preoccupations blundered into this grown-up world. Even before Olgica's arrival, my boyhood obsession had been Eastern Europe. I would spend hours looking at dusty atlases, and reading about the vanished kingdoms and republics of the pre-communist era, with their long-forgotten politicians, quaint postage stamps and exotic languages. Behind the Iron Curtain, they seemed as distant and unreal as Atlantis. In my early teens, I needed an example of communist propaganda for a school history project and decided to write to the Yugoslav embassy in London, asking for an official statement of how their government saw their defeated royalist rivals. That would, I thought, sit nicely alongside the other exhibits I had already assembled, including a passage from Winston Churchill's history of the war, a poignant account of life in Cambridge by the exiled Yugoslav boy-king Peter, and a sizzling history of a British military mission to his doomed soldiers.

I proudly announced my plan. To my consternation, Olgica turned white. My mother took me aside: didn't I understand that the Yugoslav embassy in London would at once hand over this letter to the secret police? (With its sinister-sounding acronym UDBA, the Uprava državne bezbednosti or Department of State Security was the bane of the regime's critics at home and abroad). It would be obvious that my childish inquiry came from the same Oxford address where the daughter of a top Yugoslav pediatrician was living while completing her A-levels. It was bad enough that the UDBA would instantly suspect her of propagandizing about the royalist past -- a crime in Yugoslavia. Worse, it would start checking up on her family history and might then discover her carefully concealed ties to the notorious inhabitant of Crotch Crescent. Her life could unravel in an instant.

This trivial episode taught me important lessons -- albeit in politics not history. First, that the power of the communist state was based on the relentless, intrusive, bureaucratic reach of the security and intelligence services, and their capacity to ruin the lives of those who displeased them. Secondly, that these agencies' reach extended far beyond their own grim dominions -- even to the seemingly safe and secure world of an English university town. The extraordinary idea that my actions could put me under scrutiny by hostile foreign officials sparked an interest that has gripped me for decades. In the years that followed I devoured spy literature, from defectors' memoirs to John le Carré's novels. I tracked down retired spies and quizzed them. I also kept a beady eye on contemporaries who were offered jobs by MI6, as Britain's Secret Intelligence Service, SIS, is colloquially known.

Even without knowing much about the intelligence world I was struck by the clumsiness of those approaches: on the same day a crop of identical government-issue buff envelopes would arrive in student pigeonholes. Some recipients ignored the strictures to keep silent. A friend even framed the letter and put it in his lavatory, so his friends could appreciate the unconvincing letterhead and the strangulated offer: "From time to time opportunities arise in government service overseas of a specialized and confidential nature." For those who did apply, the clumsy efforts of the vetting officers (also accompanied by dire warnings about secrecy) were similarly corrosive of confidence in the spooks' worldview. Did it really matter in the struggle against the Soviet empire, I wondered, if Tom had gay flings, if Dick smoked dope or if Harriet had a boyfriend in the Socialist Workers' Party?

I reckoned I could do more good on the outside, and searched for any Eastern European cause that would accept my help. Inspired by my father, who smuggled books to fellow-philosophers persecuted in Czechoslovakia, I helped organize a student campaign to support Poland's Solidarity movement, crushed by martial law in December 1981. I waved placards outside embassies and wrote letters of protest on behalf of political prisoners. I studied unfashionable languages like Polish, and practiced them by befriending bitter old émigrés in the dusty clubs and offices of west London -- the world of le Carré's Estonian "Colonel" in Smiley's People. Like the spy author's fictional émigrés, these real-life ones had been sponsored by Britain's spooks, then betrayed and dumped.

I would occasionally take the number 12 bus down Westminster Bridge Road in Lambeth, past the headquarters of Britain's MI6. The location was in those days, supposedly, a closely guarded official secret, though the bus conductor was prone to announce jovially "Century House -- all spies alight here." I never went inside. But I would gaze up at the grubby concrete structure, with a petrol station incongruously sited in its forecourt. Was this really our answer to the fearsome Soviet Lubyanka in Moscow? The imposing classical façade of the KGB citadel (originally an insurance company headquarters) would have suited the grandest streets in central London. But the MI6 building looked liked a scruffy Soviet tower block.

Spies, whether paid agents, idealistic volunteers, or professional intelligence officers, were foot soldiers in the struggle between east and west that shaped the lives of all postwar generations, including mine. They intrigued me as a student, activist, and journalist -- first in London and later behind the Iron Curtain. In the 1980s I rubbed shoulders and clinked glasses with spooks on both sides, dodging their blandishments while swapping jokes, jibes, arguments, and ideas. For a brief while, the collapse of communism looked set to doom the whole business. Now that the Soviet Union was gone, and with it the danger of the Cold War turning hot, what was left to spy on? But the champagne corks that spooks popped in Britain and America in August 1991 were as premature as the gloom in Lubyanka as the statue of Felix Dzerzhinsky -- Lenin's secret-police chief -- was hauled away by a crane to the cheers of exuberant Muscovites. MI6, the CIA, and their partner services rejiggered their budgets and turned to new targets: rogue arms dealers, terrorists, gangsters, and cybercriminals. But new crime and old espionage soon proved to be overlapping phenomena; the crooks in the foreground were sometimes new, but in the background lurked, more often than not, the wily and ruthless figures of the old Soviet-bloc intelligence world.

They proved a good fit: dark partners in the new order. Far from being swept into the dustbin of history with the rubble of the old system, the communist-era spooks have evolved to match the new conditions. Some figures from the old days stayed undercover, gaining trusted roles in the new state structures. Others turned to business, where their foreign languages and knowledge of the outside world gave them a flying start in the new game. All across the former Soviet empire, assets of the Communist Party and its front organizations speedily melted away, often ending up in hands of the wily and well connected. So too did the operational funds of the KGB and its allied agencies. Estimates of the money squirreled away abroad during the collapse of the Soviet Union are in the tens of billions of dollars; a crop of still-unexplained suicides in the old system's dying days disposed of those in a position to blab. These caches of illicitly acquired cash were a financial springboard for the fleet-footed members of the old elite in their new business careers. In effect, they turned their power into wealth, and then back into power.

In Russia itself, Soviet-era spies, chief among them Vladimir Putin, a former KGB officer, now run the country. They are known as the Siloviki or "men of power." The old KGB was decapitated in 1991 amid the Soviet collapse, but not uprooted. Instead it renamed itself, just as so often in the past. (Under Vladimir Lenin it was the Cheka; later it became the OGPU, then the NKVD, and finally the KGB.) It is now split into two: the FSB, which has inherited the repressive domestic apparatus of the old system, and the SVR, which is the heir to the Soviet foreign intelligence service; alongside both works the separate GRU military intelligence agency.

Their most potent weapon in their deception is ordinariness. Just as Russian politicians and officials seem at first sight to hail from the same besuited and unremarkable caste as their counterparts in other industrialized countries, Russian spies appear neither glamorous nor sinister. They lead normal lives and work in normal jobs, moving effortlessly and inconspicuously among us. They are the kind of people you might meet at the school-gates, work alongside in an office, bump into on a business trip, or see mowing the lawn next door. Yet their real job is to penetrate our society, to influence it for their own ends, and to steal our secrets.

The best known of this new generation of Russian spies was Anna Chapman, the young redhead who was made a global superstar by her arrest and deportation in June 2010. She has become an intimate friend of Putin's, a prized asset of his political machine, a prominent figure in Russian finance, and a television celebrity. But her main talents in working abroad were not the highly honed skills of spy-school legend. She started her life here in the humdrum London suburb of Stoke Newington, to the outside eye just another hard-partying, quick-witted, young Russian woman with an English husband and an eye for the main chance, enjoying the safety and comfort of life in Britain. But her ordinariness was deceptive. She was well-placed to carry out her espionage assignments precisely because she seemed so inconspicuous. Her later transformation into a trophy superspy adds another dimension. It is proof of the skills of her imidzhmekeri (image-makers) and casts a revealing light on Russia itself.

The spy scandal that made Chapman famous was part of a larger picture. She was one of 10 people arrested in the United States in June 2010, all of whom lived unremarkable middle-class lives, seemingly far away from traditional espionage targets such as the Pentagon or State Department. She and another Russian lived there under their own names. Seven others had fraudulently obtained identities -- American, British, Canadian, Irish, and Uruguayan (the 10th was the latter's Peruvian spouse). One more suspect, a Russian called Pavel Kapustin, working under the alias of Christopher Metsos, was arrested in Cyprus but allowed to escape by the authorities there -- an episode, never satisfactorily explained, which still arouses fury in U.S. officialdom. (In a related case, a Russian who once worked at Microsoft was deported on immigration grounds in mid-July of that year).

Some people reacted with derision to the idea that Russia would send spies to suburbia, others with surprise. Both reactions were mistaken. This was not a new or foolish initiative by the Kremlin's spymasters, but the latest twist in an old and sinister one. Only two years previously in 2008, the case of Herman Simm had highlighted Russia's penetration of NATO. A portly Estonian ex-policeman who had become that country's top national-security official, he was exposed as a Russian agent after some able work by Western spycatchers. His case officer -- the career spy in charge of his activities -- was unmasked too. This was "Antonio": a Russian masquerading as a Portuguese businessman, under an elaborately constructed illegal identity. But the media furor over that case soon died down, leaving most people unaware of the effort that Russia, like the Soviet Union before it, still puts into deception, infiltration and subversion.

The international media frenzy surrounding Chapman trivialized espionage as a branch of show business. The mistake was easily made: pouting and haughty, the Russian firecracker could easily be a fictional character, not a real one. She would fit in neatly as the sultry sidekick to the arch-villain in a Bond movie. 007's relationship with "90-60-90" (Chapman's Russian nickname, which comes from the millimeters her shapely figure) would provide appropriately cheesy sexual tension. The lurid and seemingly pointless affair invited ridicule. New York magazine's headline was "Russian Spies Too Useless, Sexy to Prosecute." In London, the Guardian said confidently that "none of the 10 Russians had culled any secrets from their hideouts in US suburbia." A grand old man of Anglo-American journalism opined that the Russian illegals' operation was marked by "complete futility." As the detainees were swapped in Vienna for four people jailed in Russia for spying, David Cornwell, who under the pseudonym John le Carré so ably captured the dark intrigues of Cold War espionage, even suggested that out-of-control "rightists" in America's intelligence agencies were trying to jinx the improvement in Russian-American relations. He asked: "As we watch live in glorious Technicolor the greatest spy-swap of the twenty-first century, and hear in our memories the zither twanging out the Harry Lime theme, do the spies expect us to go scurrying back to our cold war shelters? Is that the cunning plan?"

With respect to Britain's greatest spy writer, and with rather less to other commentators, that is an oddly complacent approach. Spies need to seem as boring and inconspicuous as possible, to develop the capabilities that their real jobs require. If they are to be humble errand-runners, ferrying money, false documents, and other wherewithal to more glamorous operatives, then they need jobs that allow them to travel. George Smiley, le Carré's best-known character, spent the war years working undercover as an official (supposedly Swiss) of a Swedish shipping company -- the perfect background for someone needing a regular excuse to visit Hamburg or other German ports. For some the task is to gain jobs, hobbies, or lifestyles that give access to secret information. If the mission is identifying potential sources and the weaknesses that will enable their recruitment, they should be good networkers. If they are case officers, who recruit, direct, motivate, and check the agents, they need a lifestyle in which meeting a wide range of people arouses no suspicion. If they are moles, aiming to penetrate the other side's security or intelligence services, they need educational and career paths that will make them credible candidates for recruitment there.

Charles Crawford, a long-serving British diplomat in the region, explains it well on his blog. Espionage means finding out where highly sensitive and useful information is stored or circulated, then using the human or physical weaknesses in its protection to copy the information in an undetectable way. All this must be done without anyone noticing or suspecting, and repeated many times over. In such work invisibility is a prime advantage. Spycatchers can watch the every waking and sleeping hour of a diplomat suspected of spying. They can comb through visa applications to spot foreign visitors who may be more or less than they seem. They can put suspects on their own side under surveillance to see if they are having odd meetings with strange people. Such techniques may be effective in catching a spook disguised as a diplomat, or a careless traitor. But they have almost no chance of catching a properly trained and targeted "illegal" -- someone working under an acquired or stolen identity.

Russians do not trivialize or ridicule espionage. They take it rather seriously, both as a threat from abroad and as something that their country excels in. Admittedly, people everywhere find fictional spies glamorous. America has the amnesiac but indestructible Jason Bourne; Commander Bond's high jinks sprinkle stardust over the reputation of SIS. But real-life spies in Western countries have only modest privileges compared to their counterparts elsewhere. In Britain, for example, they retire at 55, earlier than the diplomatic colleagues whose cover they use. They have rather larger and more loosely scrutinized expense accounts than other officials, but on the whole enjoy the same lifestyle as any other middle-class professional.

The Soviet legacy, however, has left a distinctive aura around espionage in Russia. For officers of the KGB (such as Chapman's father, Vasily, or Putin and hundreds of thousands like them) life was markedly nicer than for fellow-inmates of the workers' paradise. Housed in the KGB's special accommodation, its officers had access to shops stocked with otherwise unavailable products. They holidayed at KGB resorts and were spared some of the system's petty restrictions on daily life. Those in the elite foreign-espionage division, the First Chief Directorate, and some colleagues in cryptography and counter-intelligence, could even be sent to work abroad -- perhaps even a posting to the fabled Western cornucopia that the class warriors both despised and envied.

Privileges aside, the KGB also enjoyed a mystique that still lingers over its successor organizations. People saw it (rather inaccurately) as efficient, knowledgeable and incorruptible. Its officers had a job that mattered, in an organization that worked, and were well rewarded for it. Few in the claustrophobic, ill-run, and bribe-plagued Soviet Union could boast as much. Like the space program and sporting heroes, the KGB also touched another emotional chord: patriotism. Though its ultimate loyalty was to the Communist Party, not the Soviet state (it described itself as the Party's "sword and shield") it basked in the reflected glory of the defeat of Nazi Germany. Rather as the Battle of Britain provides Britain's "finest hour," as the Resistance epitomizes France's national myth, and as the Normandy beaches exemplify America's commitment to the freedom of Europe, the Great Patriotic War (as the Second World War is known in Russia) was the central plank in the Soviet Union's self-image -- and plays the same role in Russian identity today.

For all the heroism displayed by Soviet soldiers in defeating the Nazi invaders, the real role of the secret police in those years was a despicable mix of war crimes against the foe, ruthless pacification of "liberated" territories and persecution of real or imagined waverers on its own side. Yet Soviet wartime history mostly comes across in a quite different light: on the television screens later adorned by Chapman's lightweight program on unsolved mysteries, viewers used to watch the exploits of the best-known Soviet fictional spy, Max Otto von Stirlitz (to give him his German cover name). His wartime mission was to penetrate the Nazi high command. Unlike Bond, Stirlitz shuns gadgets, guns, and girls. His weapon is his mind, fuelled not by communist ideology but a plangent patriotism. Though implausible, books and films featuring his exploits were compelling and sympathetic by the hackneyed standards of Soviet propaganda. They so captivated one tough teenager in the backstreets of 1970s Leningrad that he took the unusual step of walking into the city's KGB headquarters and volunteering his services. But the young Vladimir Putin was told that the organization did not accept walk-ins; he should get an education first and wait to be approached.

The Soviet Union is gone, but the links between Russia's spies today and their dark and bloody past are real enough. Of course, the old and new are not identical. Chapman's Soviet-era predecessors wore ill-fitting grey suits and sought the shadows. She likes leather cat suits and the spotlight. They served a totalitarian superpower. She serves post-Soviet Russia, a country that is undeniably capitalist and claims to be democratic. But a lasting connection is privilege. The dispensations enjoyed by Russia's spooks now mean that they lead a life apart, just as KGB officers did in the Soviet era. The difference is not in salary and access to consumer goods, but in the privilege of living above and outside the law. The results range from the trivial to the monstrous. An officer of the FSB can drive while drunk (and mow down pedestrians) with impunity. A flash of his ID badge will intimidate any lesser official; he can triumph in any private legal or commercial dispute; he can ignore planning regulations when he builds his house in the country.

Chapman does not just hit the old Soviet buttons in the Russian psyche. She tickles its modern neuroses too. Her brand is based not on the steely Puritanism of the wartime Soviet military but on the sleazy glitz of modern Russia. Her role was to spy not on the hated Nazis of long ago, but on a new bugbear: Western countries such as Britain and America, which the Russian regime sees as duplicitous, arrogant, and greedy. Though the elite likes to shop, bank, frolic, and school their children in and around London, many of its members despise Britain -- just as they resent what they see as American hegemony and the bossiness of the European Union.

This hostility stems in part from an inferiority complex: for all the West's ills, it provides a quality of life that is missing in Russia. This is despite what many Russians see as its baffling weakness and indolence. Another reason is that Russians object to what they see as the West's political interference -- for example, by sponsoring media freedom and pro-democracy causes, and sheltering fugitives, who claim to be persecuted for their political beliefs, but are seen (at least by the authorities in Moscow) as mere swindlers and terrorists.

Despite the Putin Kremlin's lip-service to the free-market, the business community has been a frequent target as well. Few cases highlight the FSB's corruption and brutality better than the torture and death in 2009 of Sergei Magnitsky, a Russian lawyer working for a British investor. He exposed a $230 million fraud by a criminal group led by the FSB and backed at the highest level in the regime. He paid for this discovery with his life; since his death, the authorities have tried to cover up his murder, and their fraud, with a mixture of bombast, lies, bullying, and evasion. The scandal exemplifies the overlap between gangsterdom and power in Russia, the abuse of the legal system, and the bravery of those Russians willing to defend the rule of law. The tentacles of FSB power stretch to the West too, not least because Russian officials have snooped on and intimidated Magnitsky's colleagues and defenders in London and elsewhere. The Magnitsky case shows that the ruling regime represents not just a tragedy for Russia: it is a direct threat to our own wellbeing and safety.

The passage of time and other priorities have eroded the expertise and institutional memory that in Cold War days helped spycatchers keep track of Soviet penetration attempts. Concerns for privacy have made vetting procedures flimsy. Officials can make money on the side, take lucrative jobs on retirement, take unexplained foreign trips, copy documents onto memory sticks from supposedly secure laptops, and carry an array of electronic gadgets that never come under scrutiny. A mistaken complacency has also surrounded the expansion of NATO to the ex-communist countries. It was right to enlarge the alliance (chiefly because of Russia's neo-imperialist saber-rattling) but intelligence and security services have grossly underestimated the Soviet-era shadow that still lies over the region. The liberation of 1989-1991 was intoxicating, but the effect was only skin-deep. Replacing the planned economy with free markets, state censorship with free media, and one-party rule with free elections were hugely important changes. But the transformation of the political and economic systems could not be matched by an instant change in the human beings that inhabit them. Millions of people in the region have grown up under communism and collaborated with it. The toxic legacy of secret police files, with the shabby compromises and sordid secrets they contain, still taints public life. It provides plenty of scope for blackmail of the guilty -- and the smearing of the innocent. Even those seen in the West as heroes, such as Poland's former president Lech Walesa, have come under a cloud of suspicion about past collaboration. Although not everything in the secret police files is true, and many true things are not in the files, the dirty secrets of the past, many of them spirited away to Russia in the dying days of the old regimes, create great possibilities for pressurizing anyone born before, roughly, 1970. In short, the collapse of communism left a series of human time-bombs all over the former empire -- with the Kremlin in charge of the remote controls.

Neither the Simm case, nor the exposure of Chapman and her colleagues, have properly woken up public opinion and officialdom to the fact that Russian spies' activities are not just a lingering spasm of old Soviet institutions, twitching like the tail of a dying dinosaur, but are part of a wider effort to penetrate and manipulate, which targets the weakest parts of our system: its open and trusting approach to outsiders and newcomers. Because this threat is underestimated or outright ignored, it is especially potent. It is part of a world, espionage, of which outsiders mostly know little and understand less.

The battle lines were more clearly drawn in the days of the Cold War, when the threat was of communist victory. The corrupt autocracy that rules Russia now is playing by capitalist rules -- and the threat is even more corrosive. However, Russia's new spies, like their Soviet predecessors, engage in the subversion, manipulation and penetration of the West. They also defend a regime that is tyrannical, criminal and even murderous. Some things never change.