ProPublica

Journalism in the Public Interest

Cancel

Everything We Know (So Far) About Obama’s Big Data Tactics

A new look at what the Obama campaign did with its much-heralded data operation.

.

A narwhal, the namesake of Obama's secret data integration project. Source: iStockPhoto

This post is being kept up-to-date. It was first published on Nov. 13.

For the past nine months, we’ve been following how political campaigns use data about voters to target them in different ways. During the election, the Obama campaign, which had assembled a cutting-edge team of data scientists, developers, and digital advertising experts, refused to say anything about how it was targeting voters.

Now, members of the campaign are starting to open up about what their team actually did with all that data. Based on our own interviews with campaign officials, as well as reporting elsewhere, here’s an overview of some of the campaign’s key strategies, from connecting Facebook data to the national list of registered voters, to measuring how “persuadable” individual swing state voters might be.

Here’s what the nerds did.

What data did they use—and were they tracking you across the web?

It’s still not clear.

Chief Innovation and Integration Officer Michael Slaby and other campaign officials said again that they relied less on consumer data, and more on public voting records and the responses that voters provided directly to campaign canvassers. They would not comment on exactly what data the campaign collected about individual people.

Officials within the campaign said media reports had overemphasized the importance of online web cookie data to track voters across the web.

Slaby said the campaign primarily used web cookies to serve ads to people who had visited the campaign site — a tactic called “retargeting,” which Mitt Romney’s campaign also used.

The campaign also told us web cookies were useful for helping them analyze whether volunteers were actually reading the online training materials the campaign had prepared for them. (They didn’t track this on a volunteer-by-volunteer basis.)

The backbone of the campaign’s effort was a database on registered voters compiled by the Democratic National Committee. It was updated weekly with new public records from state level election officials, one official said.

The campaign then added its own voter data to the mix. Officials wouldn’t say exactly what information they added.

What will happen to the data about millions of voters collected during the campaign?

It's still not clear.

As the Washington Post reported earlier this month, other Democratic candidates are eager to use Obama's voter data for their own campaigns.

Some of the president's data will certainly go to the Democratic National Committee, where it can be used to help other Democrats.

But both the Post and the Wall Street Journal reported that it's unclear if the DNC has the resources — technological and financial — to manage all of the voter data and analysis the campaign produced.

The Wall Street Journal cited an anonymous Democratic Party official, who said that a new organization might be created to manage and update Obama's campaign data. This kind of organization "would have the potential to give the president leverage in the selection of the next Democratic presidential nominee," the Journal reported.

After his 2008 win, Obama created "Organizing for America," a group that worked within the DNC to advance the president's agenda.

How did the Obama campaign know which TV shows voters were watching?

Did the Obama campaign really "get a big list of the names of people who watched certain things on TV," as Gawker asked earlier this month?

No. But they were able to get very detailed information about the television habits of certain groups of voters.

In order to decide where to buy their TV ads, the Obama campaign matched lists of voters to the names and addresses of cable subscribers, as the Washington Post, The Atlantic, and others have reported.

This allowed them to analyze which channels the voters they wanted to reach were watching.

The campaign focused on swing state voters the campaign had scored as "persuadable," and voters who were supporters but needed to be encouraged to turn out at the polls, Carol Davidsen, who ran the campaign's television ad "Optimizer" project, told ProPublica. They also looked at voters who are Latino and African-American.

Working with Rentrak, a data company, the campaign tracked the television viewership of these groups across 60 channels, looking at how viewership changed for every quarter hour of the day, Davidsen said.

The campaign was able to identify the audience size of each group for a particular channel at a particular time — and then analyze where and when the campaign could advertise to key voters at the lowest cost.

For instance, the campaign was able to identify a subset of persuadable voters who lived in households that watched less than two hours of television a day. This allowed the campaign to schedule ads during the times these voters would actually be watching TV.

"Even if it's more expensive, it's worth it, because we can't catch them later," Davidsen told ProPublica.

So, how was the campaign able to get all this viewership data?

In Ohio, the campaign worked with FourthWall Media, a data and targeting company, to get television viewership data for individual homes, which had "anonymous but consistent" household ID numbers, Davidsen said. This allowed the campaign to track household viewing behavior over time, without knowing which exact voters they were analyzing. (FourthWall Media has yet to respond to a request for comment on how their process works.)

Working with Rentrak, the campaign could follow the viewing habits of larger groups of voters in different TV markets across the country.

Rentrak used a third-party data company to match lists of voters to TV operator data about subscribers — and then match that information to the anonymous ID numbers that Rentrak uses to track the usage patterns of television set-top boxes.

To protect users' privacy, none of the companies involved have all the information they would need to know what shows a specific voter watched on TV, Rentrak's Chris Wilson told ProPublica. For instance, Rentrak knows viewers' ID numbers and viewing habits, but doesn't know which ID numbers correspond with which name or address. In fact, Rentrak never knows the name or address of the household, Wilson said.

While commercial advertisers are beginning to do this kind of data matching and analysis, "What [the Obama campaign] did was probably on the more sophisticated side compared to a lot of folks," Wilson told ProPublica. He said the campaign had done "pioneering work" in television targeting.

Rentrak also works with large consumer data companies, including Experian and Epsilon, to match television viewer data with consumer data. According to the Federal Communications Commission privacy rules, cable operators are not allowed to disclose subscribers' "personally identifiable information" without their consent, but they can collect and share "aggregate" data.

How important was the data the campaign could access through its Facebook app about volunteers and their friends?

Observers noted that in the last days of the campaign, Obama supporters who used the campaign’s Facebook app received emails with the names and profile photos of friends in swing states. The e-mails urged supporters to contact their friends and encourage them to vote.

It wasn’t clear how well the effort went or what the response was. Some people had been encouraged to ask their Republican friends to vote. A Romney official who had signed up for the campaign’s e-mail list was told to contact his Facebook friend Eric Cantor, the Republican House Majority Leader.

What we now know is that the campaign did in fact try to match Facebook profile to people’s voting records. So if you got a message encouraging you to contact a friend in Ohio, the campaign may have targeted that friend based on their public voting record and other information the campaign had.

But the matching process was far from perfect, in part because the information the campaign could access about volunteers’ friends was limited.

Were privacy concerns about the campaign’s data collection justified?

We’ve reported on some of the concerns about the amount of data the campaign has amassed on individual voters.

Were those concerns at all justified? It’s hard to say right now, since we still don’t know where the campaign drew the line about what data they would and would not use.

Obama officials did dismiss the idea that the campaign cared about voters’ porn habits.

The analytics team estimated how “persuadable” voters are. What does that mean?

It all came down to four numbers.

The Obama campaign had a list of every registered voter in the battleground states. The job of the campaign’s much-heralded data scientists was to use the information they had amassed to determine which voters the campaign should target— and what each voter needed to hear.

They needed to go a little deeper than targeting “waitress moms.”

“White suburban women? They’re not all the same. The Latino community is very diverse with very different interests,” Dan Wagner, the campaign’s chief analytics officer, told The Los Angeles Times. “What the data permits you to do is figure out that diversity.”

What Obama’s data scientists produced was a set of four individual estimates for each swing state voter’s behavior. These four numbers were included in the campaign’s voter database, and each score, typically on a scale of 1 to 100, predicted a different element of how that voter was likely to behave.

Two of the numbers calculated voters’ likelihood of supporting Obama, and of actually showing up to the polls. These estimates had been used in 2008. But the analysts also used data about individual voters to make new, more complicated predictions.

If a voter supported Obama, but didn’t vote regularly, how likely was he or she to respond to the campaign’s reminders to get to the polls?

The final estimate was the one that had proved most elusive to earlier campaigns—and that may be most influential in the future. If voters were not strong Obama supporters, how likely was it that a conversation about a particular issue — like Obamacare or the Buffett rule—could persuade them to change their minds?

Slaby said that there was early evidence that the campaign’s estimate of how “persuadable” voters would be on certain issues had actually worked.

“This is very much a competitive advantage for us,” he said.

Wagner began working on persuasion targeting during the 2010 campaign, Slaby said — giving the campaign a long time to perfect their estimates.

Did everyone on the campaign have access to these scores?

No.

Campaign volunteers were not given access to these individual scores, one official said. “Oh, my neighbor Lucy is a 67 and her husband is a 72—we would probably consider that a distraction.”

Do other political campaigns also assign me a set of numerical scores?

Yes.

The use of microtargeting scores — a tactic from the commercial world — is a standard part of campaign data efforts, and one that has been well documented before.

In his book exploring the rise of experimentally focused campaigns, Sasha Issenberg compares microtargeting scores to credit scores for the political world.

In 2008, the Obama campaign ranked voters’ likely support of the senator from Illinois using a 1 to 100 scale. This year, The Guardian reported, Americans for Prosperity, a conservative group backed by the Koch brothers, ranked some swing state voters on a scale from 0 to 1, with 0 being “so leftwing and pro-government” that they are “not worth bothering with,” and 1 being “already so in favour of tax and spending cuts that to talk to them would be preaching to the converted.”

What’s different about what Obama’s data scientists did?

Obama campaign’s persuadability score tried to capture not just a voter’s current opinion, but how that individual opinion was likely to change after interactions with the campaign. Most importantly, Obama’s analysts did not assume that voters who said they were “undecided” were necessarily persuadable—a mistake campaigns have made in the past, according to targeting experts.

“Undecided is just a big lump,” said Daniel Kreiss, who wrote a book on the evolution of Democratic “networked” politics from Howard Dean through the 2008 Obama campaign.

Voters who call themselves “undecided” might actually be strong partisans who are unwilling to share their views—or simply people who are disengaged.

“Someone who is undecided and potentially not very persuadable, you might spend all the time in the world talking to that person, and their mind doesn’t change. They stay undecided,” Slaby said.

To pinpoint voters who might actually change their minds, the Obama campaign conducted randomized experiments, Slaby said. Voters received phone calls in which they were asked to rate their support for the president, and then engaged in a conversation about different policy issues. At the end of the conversation, they were asked to rate their support for the president again. Using the results of these experiments, combined with detailed demographic information about individual voters, the campaign was able to pinpoint both what kinds of voters had been persuaded to support the president, and which issues had persuaded them.

Avi Feller, a graduate student in statistics at Harvard who has worked on this kind of modeling, compared it to medical research.

“The statistics of drug trials are very similar to the statistics of experiments in campaigns,” he said. “I have some cancer drug, and I know it works well on some people—for whom is the cancer drug more or less effective?”

“Campaigns have always been about trying to persuade people. What’s new here is we’ve spent the time and energy to go through this randomization process,” Slaby said.

Issenberg reported that Democratic strategists have been experimenting with persuasion targeting since 2010, and that the Analyst Institute, an organization devoted to improving Democratic campaign tactics through experimentation, had played a key role in its development.

Slaby said the Obama campaign’s persuasion strategy built on these efforts, but at greater scale.

Aaron Strauss, the targeting director at the Democratic Congressional Campaign Committee, said in a statement that the DCCC was also running a persuasion targeting program this year using randomized experiments as part of its work on congressional races.

What were the persuasion scores good for—and how well did they work?

The persuasion scores allowed the campaign to focus its outreach efforts—and their volunteer phone calls—on voters who might actually change their minds as the result. It also guided them in what policy messages individual voters should hear.

Slaby said the campaign had some initial data suggesting that the persuasion score had been effective—but that the campaign was still working on an in-depth analysis of which of its individual tactics actually worked.

But a successful “persuasion” phone call may not change a voter’s mind forever—just like a single drug dose will not be effective forever.

One official with knowledge of the campaign’s data operation said that the campaign’s experiments also tested how long the “persuasion” effect lasted after the initial phone conversation—and found that it was only about three weeks.

“There is no generic conclusion to draw from this experimentation that persuasion via phone only lasts a certain amount of time,” Slaby said. “Any durability effects we saw were specific to this electorate and this race.”

jason benlevi

Nov. 13, 2012, 5:06 p.m.

I don’t think people realize how extensive the big data operations have become over the past decade. Although some would argue that Obama used it for a good purpose, much the technology has its origins in more nefarious companies.

I recommend my book as resource to understand what is going on invisibly around you. Get a copy or ask your library to get a copy.
You really need to be aware of this.

http://www.toomuchmagic.com

Hands down, I think this is going to be the single biggest story of the decade.  What we’re talking about is, for lack of a better term, the beginnings of “micromanipulation,” where candidates look for the most gullible voters and sway each of them with personalized lies, and the rest of us can go jump off a cliff, for all they care.

Seriously, if anybody has ever considered it unhealthy that Presidential candidates put in all their time in “battleground states,” do to the awkwardness of the Electoral College, wait for the day when half the population never even sees a candidate, because you probably won’t vote for him.

And once you can microtarget a message, why be consistent?  Who would (could!) question Romney’s positions if he only told the war-hawks that he wants to amp up the War on Terror, and ensure the rest of us only see the peace pitch he tried during the debates?

For all the talk about Dark Money, this is pretty much the end of an informed Democracy.

Everybody should read it, with all the dismissive (way to straw-man with “we don’t care about porn habits”—was that in question?) and condescending comments about “what they need to hear” to change their votes and who is the most persuadable voter highlighted.

I don’t know how to stop the trend, but at least if people are aware of it, there’s a chance that we’re not left with a population voting because they’ve been tricked the same way advertising jingles get caught in your head.

(Also, imagine if a company like Facebook allowed one campaign full run of their database for an election.  Everything you’ve written in the heat of the moment, every embarrassing picture, the result of every half-assed political poll, the times you log in, the feeds you subscribe to, the groups you belong to, and all your friends is available, there.  Any campaign that can’t sway an election with something like that would be totally incompetent.)

Interesting article. It does seem quite suspicious that many of the demographics do not match the will of the voting public at large. One must wonder how much “subliminal technology” was and is being used as tools of persuasion…......sub atomic…....radio isotopes…..grid manipulations…their possible existence seems infinite…....

Thanks for the feedback on your investigation! It’s obvious that much less information was gleaned from the Romney camp since it was far more likely that Obama emails would be sent to you by ProPublica readers. (I would assume most Romney supporters didn’t stray into reading publications with a broader, fact based, non-ideological point of view.) 

You’ve stated the overall “effectiveness” of tactics used by the Obama campaign isn’t clear yet, but given the information regarding retention of persuasive arguments, they had it figured out and they’ll continue targeting people with the clear intention of trying to influence thinking during the critical period when the “influence information” is still fresh in their minds. So we can expect (again!) an escalation of phone calls, mailers, and TV ads - full of miss-information and lies - just prior to the actual vote.

My only defense was to throw out ALL mailers, record ALL TV programs so I could fast forward through the commercials, use caller ID before answering my phone, and only use trusted sources for information in making choices on candidates/issues.

The most comforting news to come out of election results was that the enormous sums of PAC money spent to influence the vote was apparently a complete waste in the general election. What has me concerned is how much influence that same money has on our primaries - in which fewer people participate, and which actually determine our final candidates.

Daniel Robert Snodgrass

Nov. 13, 2012, 6:04 p.m.

Persuasion was discussing issues with organizations and campaign workers.

The NRA and RNC won big in the robocall vote.

Reminding this landline from New Hampshire to turn the ringer back on.

Brandt Hardin

Nov. 13, 2012, 8:27 p.m.

Despite all odds, our President prevailed.  He still has an uphill battle fighting a Red House which has blocked his every move in an attempt to squash his goals of bringing the Middle Class equal pay, women’s rights, gay rights and affordable healthcare.  The Bush Administration drove our economy into a swift nose dive and Obama is still the patsy.  Watch conservative hands paint him in Blackface with a visual commentary of how Barack has been bamboozled at http://dregstudiosart.blogspot.com/2012/10/bamboozling-obama.html

@John (5:30 pm comment)  - your micromanipulation comment is not true..  One could argue the Fox News/Hannity/O’Reilly viewers were manipulated by their programming, and thus not encouraged to turn out in the same way that likely Democratic voters were, a strategy that didn’t pay off for Republicans. As a participant in the canvassing, there was no lying to potential voters - in fact, folks were reminded of who the candidates were in the local races, and then encouraged to vote.

Having personally utilized microtargeting in a nonpartisan local commission election several years ago on data gleaned from a petition and a county voter registration file, I had great success in positively identifying several classes of voters. I used MS Excel, 30 precincts data, names, addresses, implied party preference to sort the database into 3 categories of support: Strong, Likely and Weak. We sent multiple mailers to the top 6 precincts appearing the strongest in order to nail down the most support; one mailer to the Likely group and phone calls to the remainder. Our opposition spent some $25K and we spent $6,900 and beat them 62 to 38%. It was great fun and our candidate brought truth and vastly improved control to the job. He was recently reelected.

clarence swinney

Nov. 14, 2012, 9:04 a.m.

How did we get from nearly $6 trillion projected surpluses in 2001 to $6 trillion in new debt?
As President Obama and Republican leaders in Congress square off on the debt-cutting measures that could send us over the fiscal cliff, “it’s crucial to remember the source of our deficit and debt. The Center for American Progress’ Michael Linden explains the laws passed during the Bush administration along with the wars and the bad economy over a decade made up 86 percent of the debt we’ve accumulated since 2001. President Obama’s budgets and the stimulus are responsible for only 14 percent.
Keep that in mind as Republicans demand concessions from the president and the middle class.
Tags: Bush Tax Cuts, debt, Deficit, fiscal cliff, surplus

Anne, that’s today.  I’m not accusing either 2012 campaign of anything except opening the can of worms.

Never concern yourself about how a tool will be used today by amateurs.  Always look at how it will be used after it has been mastered and people with fewer inhibitions are the guiding hands.

Imagine what will be done with this in the 2020 election, for example.  Imagine the technology in the hands of a new David Duke or pro-depopulation Ted Turner.  Do you really want your private information in the hands of someone willing to win a campaign at any cost?  Do you want your voice excluded from a discussion because a computer says that you’ve already made a decision?

These are the first stages, “adapting” the “message” to what will sway a particular “class” of voters.  I already find that much offensive, as much as you say it wasn’t lying (which usually doesn’t mean that a subset of the truth was avoided).  But it will absolutely get worse.

The only way the voters can make a rational decision is to be in possession of ALL the information, not just a subset that the party has approved for your consumption to maximally influence you.

Thanks for the thoughtful comments, everyone. Nice to hear about ClaudeM and Anne’s personal experiences with campaign data.

clarence swinney

Nov. 14, 2012, 5:44 p.m.

WHY THE DEBT?
In 2000, We had revenue of 20% and spending 18% of GDP. Surplus.
In 2009, we had 15% of Revenue and 25% of spending as % of GDP.
That was an increase in Spending and decline in Revenue.
Bush policies.  How will historians rate his administration?

STEPHEN PAUL DELSOL

Nov. 17, 2012, 2:33 p.m.

The main purpose of any election is to persuade likely voters to vote for candidates. All human beings are capable of being persuaded, some are more so than others. Since, the first public election was conducted, until this present one, candidates have used a wide variety of means to persuade voters to elect them.

We are in the age of Big Data, where sophisticated IT tools are being used. It is morally defensible for President Obama and his IT specialists to defend the tools and techniques that they have used to persuade voters to re-elect President Obama. The President’s election policies appeal to most Americans and promote the common good, therefore, the means justify the end.

None of this data mining stuff is all that remarkable.  Anyone who’s ever worked on a retail marketing website knows about it.  The magic isn’t in the data, it’s in asking the right questions.  Truth is, my only question is how do I find out what my score are.  I voted for the guy, seems only fair that I should be able to find out.

STEPHEN PAUL DELSOL

Nov. 20, 2012, 2:04 p.m.

Eric, I defend your right to know what your score is. We are in the age of massive amount of quantitative and qualitative data. Asking the right questions transform the data into information. What President Obama’s IT team did was to continue to interrogate that information to get the answers they wanted for specific purposes.

Asking the right kind of questions are the key components in data mining. The questions are like a drill that allow the IT specialist to crack and frack the data, and fractionate it into different sub-groups, such as 18 to 29 year olds; Blacks, Hispanics, Asians, single women, seniors, veterans, etc. There is nothing unethical about this process. No one is harmed. Politicians can direct their policies to suit particular groups. This targeting is no different from DNA mapping on genes to radiscover abnormalities, and direct drug therapy to these specific sites.

As a person who worked for the campaign (in one of the 800 field offices), I used some, not all, of the data available mentioned above.  One fact the author doesn’t seem to understand clearly is how much information is available in public voting records.  Both parties have access to detailed specific records of registerd voters.  That includes not only name, address and often phone number (provided by the voter at registration) but more important how often and in which elections someone has voted.  That’s how we can tell a super Democrat (who votes every time, from a presidential Dem, who is possibly less loyal.  Obviously the Obama database was much more sop[histicated, with information added from many other sources, committeepeople, volunteers, etc.  As a"mere” volunteer, I wasn’t aware of the “magic numbers” referred to in the article. 
    Another point that the article didn’t clarify was the difference between the voter database, and the volunteer database, where different information was available.That should surprise no one.  But the system worked so there was a sophisticated flagging, scheduing system and followup system, so that if a phonebanker calling a voter identified a potential volunteer, he/she received a call from the field organizer the next day and were scheduled for a meeting forthwith.
    That brings up the last pount overlooked by the author: the people involved.  Without the highly trained field organizers, trained volunteer team leaders and armies of canvassers and phonebankers, Obama’s databank would have been just a machine.  In the hands of the tens of thousands of OFA workers, it has become a legend.

“How did the Obama campaign know which TV shows voters were watching?”

Their answer seemed to be “Every show, every channel, every commercial break in Ohio.”

Romney seemed to have this same data.

Procedural question for the programmers:  Especially for articles like this that are going to be updated regularly (and ideally for all of them), would it be possible/useful/desirable to rig in something like version control software so we can see the changes (and the evolution, which is often very interesting) when they happen?

For example, I enjoyed this article greatly the first time, but I don’t know that my brain is going to remember enough to pull out the different bits, and the author (Ms. Beckett, in this case) shouldn’t be trying to figure out how to contort sentences to sound natural and highlight updated content.

I don’t see any identifying information in the HTML to tell me what software your server is running, but there may be a plug in for even as something as low-end as Subversion.

(I’d also flag them on the main page as updated—I almost bypassed this, recognizing it from the first time around.)

Why do I smell snake oil?

I’d like to see some actual proof that any of this rubbish actually works and that we’re not looking at a stopped clock that just happened to be right at the right time of day.

There was a novel way back in the early 1960s by a fellow named Eugene Burdick called “The 480” that “warned” of this sort of manipulation (it was supposedly based on data mining in the Kennedy victory in 1960).  I wasn’t convinced it worked then and I’m not convinced it worked now.

I suspect this is just a way for the white coat boffins to separate large sums of cash from campaign organizatins.

Stephen Paul Delsol

Nov. 30, 2012, 10:35 a.m.

From Steve to Steve!  You will agree that the proof of the pudding is in the eating! President Obama won re-election! The reasons for his win are not clear cut. But you have to include the ‘Data Drillers’ as one of the reasons for President Obama’s victory.

These ‘Data drillers’ tunneled deep down into the massive amount of dense data given to them. They were able to distill that data into sub-groups that made it easier to tailor specific messages for them.

I was an OFA volunteer who did door-to-door canvassing. I was amazed at how I was able to knock on particular doors in heavily GOP areas. These people were Democratic voters who were enthusiastic about voting for President Obama. Some were between the 18 to 29 age group. Others were single females in Section 8 housing. Hispanics and African-Americans were much in evidence. I even phoned my former boss, a Dean at a University! 

I take my hat off to these ‘Data Drillers’. They were the unsung heroes and sheroes of President Obama re-election victory!

Persuasion works on a continuum that includes (but is not limited to) the individual’s personality, the situation he is, and the organizational effectiveness of whoever is trying to persuade him or her.  This article focuses primarily on the last feature, because that’s really the “operational arena” of data mining.  But as several comments have pointed out, there are other factors -  such as the quality of questions asked face-to-face - that are also critical.  The right question can “revive” situations a person has been thinking about a lot (How can I afford health care? Am I sick of this endless war?) and they can also randomly “catch” the person in moods where s/he is more open to being persuaded.  One of the reasons commercial marketing pummels us with so much repetition is so it can act like a heat seeking missile, catching us at times when we’re too fatigued or otherwise unprotected to fend off a message, and hoping that during such a time we will be pushed to impulse buy/decide then and there.  This may seem ominous - and to an extent it is -  but it’s important to remember that whatever we’ve been persuaded to do, we’re always capable of being persuaded - or persuading ourselves - of something else.  That’s why even though much of the president’s success can be attributed to his campaign organization and to his devoted volunteers, I suspect the greater determining factor of his reelection has probably been people “weighing him up” over time as he’s done the things he’s done in the first term:  make decisions, avoid or take on particular fights, prioritize his work in particular ways, interact with other politicians, etc.  Yes, of course there’s lots to be upset about in those categories but our weighing up also takes into consideration what we imagine the alternative might have been.  I think this dynamic is in play in any environment where persuasion operates, from sales to converting someone to join a cult.  I haven’t exhaustively read the social science literature in this area, but there are a few well known people in the field (such as Philip Zimbardo) who seem to emphasize how “multi-factorial” persuasion is, and how moveable the needle is regarding what we’ll do at any given point in time.  So as important as the data and what’s done with it is - and as critical as it is to be concerned about it - I think it’s far more important to “know thyself” as clearly as possible, so that persuasion efforts are met with wakefulness and an independent mind:  no easy task.

Greg Jemsek, author

Steve to Steve:

Uh, maybe.

Thus far we have a sample size of one.  That’s not enough to draw any sort of conclusion quite yet.

And even if it did work, it may be the kind of “trick play” that will work one time and then never again, because the next time the opposition will be looking for it and develop a countermeasure.

Back in the Carter years, Pat Caddell’s polling was supposed to have been Carter’s “secret weapon.”  The astute observer will note how well that worked out in 1980.

Heck, Citizens United was supposed to have changed everything as we know it.  Now there are a bunch of billionaires asking Mitt Romney what the heck they got for their money.

These things have a tendency of working. . . until they stop working.

Until I see some sort of larger dataset, I’ll file this one under “unproven hypotheses.”

Steve, don’t look to politics to your dataset, because by the time you get enough data to “prove” anything, either the field is abandoned or we’ll all be signing up for the military to blindly serve our Fearless Leader.

Instead, look to commerce, which is where most political tactics come from anyway (see Edward Bernays).  I won’t try to convince you either way.  But I will point out that, IF a company can convince large numbers of people that they will be happier buying a product (whether it’s a computer, a car, or watching a sports team), it’s possible to sway your opinion of a political candidate.  And IF a salesman can isolate you, identify your interests and gullibilities, and sell directly to them, then a politician can do the same thing.

I see it in commerce (and illegal scams) all the time.  As I said when the article first came out, I don’t believe that Obama got a huge advantage here, because demographic groups are rarely useful.  But the direction of technology and the decreasing ethical standards of politicians means that the future will see a politician who “crafts his message” to each voter, telling the lies most likely to sway that individual.

Among other problems, one of Mitt Romney’s biggest was that he tried to play both candidates.  He was pro-war and anti-war, pro-tax and anti-tax, pro-Obamacare and anti-Obamacare.  What if he could have pitched the alternating views to people whose Google searches or Facebook reposts or whatever (all privately-held and legally-unprotected information he could conceivably buy) show a bias for that angle?

Again, the story isn’t Obama.  The story is what this snowball is going to look like in 2016, 2020, and 2024, as tracking technology becomes more sophisticated and we become more polarized.

This is really important, thank you for doing this.

Stephen Paul Delsol

Dec. 4, 2012, 10:21 p.m.

Steve-to-Steve, thank you very much for your thoughtful and considered responses. You are right to be display a healthy skepticism of the use, misuse and possible abuse of ‘big data’ within a political context. You forced me to reflect on Karl Popper’s theory of ‘falsifiability’.

According to Popper, ‘falsifiability’ is the belief that for any hypothesis to have credence, it must be inherently disprovable before it can become accepted as a scientific hypothesis or theory.

Steve. I sense that at a basic philosophical level, you are not prepared to accept the assumptions, methodology, tools, and results obtained by President Obama IT specialists. Our democracy needs more people like you to postpone judgment until more evidence is available. As a student of science and Popper, I concur with your view in part, but not in whole.

Thank God, that Karl Popper does not have the final word! I believe that President Obama does. Having read both of his books, and listened to his speeches over the years, I have discovered that President Obama’s theology, political philosophy, and policies are based upon the Preamble to the Constitution.

“We the people of the United States, in order to form a more perfect union, establish justice, insure domestic tranquility, provide for the common defense, promote the general welfare, and secure the blessings of liberty to ourselves and our posterity…”

There is a White House website called ‘We the People’. If one examines the results of presidential elections from George Washington to Barack Obama, the phrase, ‘We the People’ truly became a reality in both the 2008 and 2012 presidential elections. A larger proportion of our diverse population voted than ever before. Democracy wins when the different demographics in the society vote in large numbers. Our country then becomes ‘a more perfect union’ when that happens.

The mining, cracking and fracking of the data into different sub-groups and demographics, in order to get them to vote was predicated on “We the People of the United States, in order to form a more perfect union..” Hence, the ethics of the process was noble, because the cause was just. It is fully justified and worthy to persuade as many people as possible to vote in our participatory democracy.

If we do not do that, then we will be held hostage by what James Madison called ‘the tyranny of the majority’. And I will add, also, the ‘tyranny of the minority’.

Although white Americans make up 72% of the population, and are the majority, and only 39% of them voted for Barack Obama, yet their superior numbers did not determine the outcome of the presidential election result. For the first time in the history of our nation’s presidential elections, James Madison’s wish has come true.

The new coalition of progressive whites, and sizable minorities of African-Americans, Latinos, and Asians will from now on determine the outcome of future presidential elections. Our democracy has now become more inclusive. The new coalition broke the ‘tyranny of the minority’ of billionaires and millionaires who invested billions of dollars to ‘buy’ this presidential election! The new coalition have ‘saved’ our democracy from becoming an exclusive club dominated by rich folks.

If the ‘Data Drillers and Distillers’ contributed to freeing our democracy of these twin tyrannies, then they should be applauded.

Steve-to-Steve, redux:

Actually, I am “prepared to accept the assumptions, methodology, tools, and results obtained by President Obama IT specialists.”  I’m just not willing to accept it all as a thus far unproven assertion.

It may very well be that the Obama team has invented the political equivalent of the forward pass—something never done before that the opposition had no defense against.  Or they may have come up with a clever trick play like you occasionally see forwarded around in your email, a brilliant hack (in the positive not pejorative sense) that works once but because, well, it only works once.  Or we may have simply seen an actual groundswell (albeit a rather small one if you look at the popular vote) of support for President Obama which simply happened to coincide with this particular bit of campaign hocus pocus.

If the first hypothesis is true, then next time around both sides will have more or less equally matched data mining operations in place and the election result will probably be decided at the furry statistical edge—whichever operation is marginally more efficient.

The second hypothesis is sort of like the first—the opposition will be ready for it and have developed a countermeasure.

The third will simply mean that campaigns will pour millions of dollars into the capacious pockets of campaign consultants to no particular effect either way.

All we have at the moment is a hypothesis.  There is a good deal of evidence to say that it’s valid: commercial concerns like American Express have been data mining for decades and one assumes that they’re not prone to throwing money away on bogus enterprises.

But, there’s also a saying in the advertising industry that half of every ad budget is wasted.  They just don’t know which half.

You know, it took me a while to realize, but Stephen’s comment drives the point home:  A lot of these comments are saying that this is OK because the ends justify the means.

Would you be less gung-ho if the article was about someone you wouldn’t support?

More to the point, do you think nobody you don’t support will ever learn from the tactics?

Catherine Fitzpatrick

Dec. 10, 2012, 3:17 a.m.

I agree with John above that this is a very big story and it is the beginnings of what I have been calling for years the Wired State, or indeed, something like a kind of technocommunism or incipient totalitarianism—which is no less thorough just because it is pleasant and clickable.

ProPublica has done an excellent job of framing the questions here of what this NDA’d bunch are letting us know and not know.

But they have to probe further in other directions. For one, there’s the geek open-source software cult around this, which is basically “openness for thee and not for me”—and the authoritarianism that brooks no dissent.

http://3dblogger.typepad.com/wired_state/2012/11/harper-reed-and-the-soulessness-of-the-new-machine.html

Then there’s the entire question of “stories” and “storification”—what are the narratives developed? How are they pitched and to whom? Like “the war on women” is pitched to 20-something single women; “voter suppression” is pitched to minorities. There is never any possibility for serious journalistic study then of these issues because the mob has already been sicced on the stories emotionally, and no amount of persuasion will get them to see that in fact, all the prominent women who won, whether McCaskill or Warren, won handily, did not face serious challenges, and never a vaginal probe was in the works anywhere, really.

http://3dblogger.typepad.com/wired_state/2012/12/youre-hardly-the-judge-of-whos-big-brother-1.html

You imply that this system works with more precision than it might. I know of an Obama campaigner here in New York who was sent a list of people to call in Ohio, and had a number of them swear at her or hang up or rant because they weren’t Democrats or didn’t want to be bothered by agitators calling them from out of state.

To the bitter end, Joe Biden was DMing me on Twitter telling me to “use my influence” to get my Twitter followers to vote for Obama. I didn’t.

Regarding Stephen Paul Desol’s “fracking and cracking”. Here’s what’s wrong with all this socialist social science stuff right out of the Soviet identity politics playbook: it creates blocs of people, demographics, where views are tied to race or ethnicity or age or sex and then it becomes impossible to “break up the blocs” and reach consensus as people then feel they are traitors to their group. We cannot live in such a world. It will not end well.

This article is part of an ongoing investigation:
Buying Your Vote

Buying Your Vote: Dark Money and Big Data

ProPublica is following the money and exploring campaign issues in the 2012 election you won't read about elsewhere.

Get Updates

Stay on top of what we’re working on by subscribing to our email digest.

optional

Our Hottest Stories

  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •