The January 2013 issue of MIT’s Technology Review has an interesting article on the Big Data techniques used by the Obama and Romney campaigns in the US 2012 election: “A More Perfect Union” by Sasha Issenberg.  No, this is not the already familiar story about Narwhal vs Orca, the cellphone apps the campaigns used to aid their workers, and the total fiasco that Romney’s Orca turned out to be.    Instead, it’s a description of how the Obamans created detailed models of all 180M eligible voters in the United States.   They call it micro-targeting.  They used polls, commercial data bases and voting records to model three parameters of each person:

1. Will they vote?
2. Which way will they vote?
3. How firm is their preference?

The goal was to maximize the vote swing from mail, email, TV ads, phone calls, and most importantly,  visits by campaign volunteers.   They constantly updated their models based on polls, early voting, and results from fund-raising appeals.   They tried A/B comparisons where they would send different messages to random groups of voters in order to fine-tune the message.

They ultimately called the vote with uncanny precision on a county-by-county basis.   Even Nate Silver only did things  state-by-state, but the Obamans had 54 people working on these analytics in their Chicago office.

The Romney campaign’s approach was different.  Instead of optimizing the swing of individual voters, they optimized the pundits.  That is, they looked at the effect that each issue of the day had on TV news, newspapers, and Twitter feeds.   They too fine-tuned messages to improve their hit rate.  This explains a lot – many had wondered why they would beat the drum so loudly on some message (E.g. Jeep is moving jobs to China!) that was just untrue.  Their algorithms said it worked so they cranked it up.

Unfortunately, they optimized the wrong thing.  There’s no necessary connection between press coverage and voting.  No wonder they thought they would win.  It’s all reminiscent of the infamous polling failure of 1948, where telephone polls predicted that Dewey would beat Truman even though a lot of Truman voters didn’t have phones.   The consistently horrible US economy gave them a real opportunity to recapture the White House and Senate, and instead they lost ground everywhere.

The article doesn’t describe how the models were created, but statistical inference is an area of huge mathematical research these days.    It’s driven by several new things:

• The existence of gigantic data sets from sensors and Net traffic
• Vast computing power in the Cloud to compute things like the weighting factors of Bayesian inference.
• Monetary incentive from advertising

It’s now used routinely for optimizing hits by search engines and for targeting advertising, but it also interesting for things like  speech recognition and translation.  You can now find subtle patterns in data sets far too large for any person to look at.

It was inevitable that these techniques would come to politics.  The author, Issenberg, thinks it’s a good thing.   It shows a respect for voters, a care for what they each think.   She has written a book about it called “The Victory Lab: the Secret Science of Winning Campaigns”.

I’m less sanguine.  When someone targets you, that means they want to shoot things into you.  I don’t like to be told that my behavior is easily modeled, true though that may be.   Modern cognitive science is showing us that all kinds of decisions that we think are consciously made are actually controlled by a vast range of subconscious effects.    This irks me.  I want to be a free citizen, not a probabilistic factor graph.    Maybe these sort of techniques will fail once enough people become aware of them.