Data, WikiLeaks and War

I’m not going to get into the rights and wrongs of the WikiLeaks thing. Nor am I going to look at the bigger implications for the balance of power between governed and governing, and between the U.S. and its allies and foes. Others have written much better than I can on these topics.

I want to look at what the cables tell us about the sorting, sifting and accessing of this information. In short, what does this tell us about how the world’s most powerful nation organized some of its most prized data?

To start, with, I want to revisit a conversation I had sitting in the garden of a Kabul pub called the Gandermack a few weeks back when it struck me: the biggest problem facing NATO in winning the war in Afghanistan is data.

I was talking to a buff security guy—very buff, in fact, as my female companions kept remarking—who was what might have once been a rare breed, but are now in big demand in Afghanistan. He was a former marine (I think), but was also a computer guy with an anthropology or sociology degree under his black belt somewhere. This guy knew his stuff.

And he was telling the NATO forces where they were going wrong: data management.

The problem, he explained, is not that there isn’t enough of it. It’s that there’s too much of it, and it’s not being shared in a useful way. Connections are not being made. Soldiers are drowning in intelligence.

All the allied forces in Afghanistan have their own data systems. But, I was told, there’s no system to make sense of it. Nor is there one to share it. So data collected by a garrison from one country in one part of the country is not accessible by any of the other 48 nations.

On the surface it seems this problem was fixed. In the wake of 9/11 U.S. departments were told to stop being so secretive. Which is why we got to WikiLeaks–one guy apparently able to access millions of classified documents from pretty much every corner of the planet. If he could do then so could thousands of other people. And, one would have to assume, so could more than a few people who weren’t supposed to have access. To give you an idea of the trove unearthed, WikiLeaks has released about 1,000 so far, meaning it’s going to take them nearly seven years to get all the cables out. Cable fatigue, anyone?

So, it would seem that the solution to the problem of not having enough pooled information is to just let anyone have it. But that, it turns out, isn’t enough. That’s because what we see from the WikiLeaks material is how old it looks.

I spent much of the early 1980s trawling through this kind of thing as a history student. Of course, they were all declassified documents going back to the 1950s, but the language was remarkably similar, the structure, the tone, the topics, the look and feel. A diplomatic cable in 2010 looks a lot like a cable from 50 years ago. In the meantime communication has gone from the telegraph to the fax to email to blogs to the iphone to twitter to Facebook.

This, to me, is the problem. It’s not that we’ve suddenly glimpsed inside another world: We would have seen a lot of this stuff at some point anyway, though it’s useful to see it earlier. Actually we can take some succour from the fact that diplomats seem to be doing a pretty good job of reporting on the countries they’re posted to. Journalists shouldn’t be surprised; we’ve relied on diplomats for a while. (And they might rightly feel somewhat aggrieved we now do this to them.)

No, the problem that WikiLeaks unearths is that the most powerful nation on earth doesn’t seem to have any better way of working with all this information than anyone else. Each cable has some header material—who it’s intended for, who it’s by, and when it was written. Then there’s a line called TAGS, which, in true U.S. bureaucratic style doesn’t actually mean tags but “Traffic Analysis by Geography and Subject”—a state department system to organize and manage the cables. Many are two letter country or regional tags—US, AF, PK etc—while others are four letter subject tags—from AADP for Automated Data Processing to PREL for external political relations, or SMIG for immigration related terms.

Of course there’s nothing wrong with this—the tag list is updated regularly (that last one seems to be in January 2008). You can filter a search by, say, a combination of countries, a subject tag and then what’s called a program tag, which always begins with K, such as KPAO for Public Affairs Office.

This is all very well, but it’s very dark ages. The trouble is, as my buff friend in the Kabul garden points out, there’s not much out there that’s better. A CIA or State Department analyst may use a computer to sift through the tags and other metadata, but that seems to be the only real difference between him and his Mum or Dad 50 years before.

My buff friend made a comparison with the political officer in today’s ISAF with a political officer (sometimes called an agent) back in the days of the British Raj. Back then the swashbuckling fella would ride a horse, sleep on the ground and know the Afghan hinterlands like the back of his hand, often riding alone, sipping tea with local chieftains to collect intelligence and use it to effect change (in this case meaning extend the already bulging British sphere of influence.) He would know the ins and outs of local tribal rivalries, who hated whom, etc. All of it stored in his head or in little notebooks.

His modern equivalent may actually have the same information, but it’ll be gleaned from the occasional photo opportunity, a squillion intelligence reports, all suitably tagged, and perhaps footage from a couple of drones. If the chieftain he’s interested in coopting straddles a regional command, chances are that he won’t be able to access anyone else’s information on him–assuming they have any.

In short, the problem in the military and diplomatic world is the same we’re facing in the open world. We have a lot more information than we can use—or keep track of—and it’s not necessarily making us any smarter. Computers haven’t helped us understand stuff better—they’ve just helped us collect, share, and lose more of it.

I must confess I’ve not made much progress on this myself. My main contribution is persuading a researcher friend to use a program called PersonalBrain, which helps you to join the dots between people, things, organisations, whatever you’re trying to figure out. It’s all manual though, which puts people off: What you mean I have to make the connections myself? Well, yes. Computers aren’t magic.

Yet. It’s clear to me that 10 years down the track, I hope, we’ll finally get that writing in prose, and then adding a hierarchy of labels to a document, is no longer the way to go. Instead, we’ll be writing into live forms that make connections as we write, annotate on the fly, draw spindly threads to other parts of our text, and make everything come to life. I will be able to pull into the document visuals, audio, other people, old records, chronologies, maps, and work with the data in three dimensions.

If this sounds familiar, it’s probably because it sounds like science fiction, something like Minority Report. But it’s not; it’s a glimpse inside the mind of our imperial political agent; how he would make those connections because they were all in his head—neurons firing transmitters, axons alive, binding synapses.

If I were the U.S. government, I would take Cablegate as a wake up call. Not at the affrontery of this humiliation, but as a chance to rethink how its data is being gathered and made use of. Cablegate tells us that the world of the cable is over.