Skip to main content Close Close Comment Creative Commons Donate Email Add Email Facebook Instagram Mastodon Facebook Messenger Mobile Nav Menu Podcast Print ProPublica RSS Search Secure Twitter WhatsApp YouTube
  • ProPublica
  • Local Initiatives
  • Data Store
  • Donate
  • Follow us on Twitter
  • Like us on Facebook
  • Search
  • Newsletters

ProPublica logo

  • Graphics & Data
  • Newsletters
  • About
  • Racial Justice
  • Trump Administration
  • Politics
  • Democracy
  • More…
  • Series
  • Video
  • Impact
  • Search
  • More

What should journalists and readers make of polls?

Welcome to Not Shutting Up, a newsletter from ProPublica’s leadership. You’re receiving this because you’ve supported ProPublica’s journalism; we’re grateful for that, and we hope to give you some context on how our newsroom works. If this email was forwarded to you, you can sign up to receive it here.


ProPublicans,

One reason we love the news business is that things seldom turn out as expected. In the 2020 elections, the voting process went surprisingly smoothly, with record numbers of Americans casting their ballots in new ways. The big failure of 2020, it turned out, was the political polling, which was so wrong in so many places that some that some people are now arguing that it’s time to spend a lot less effort trying to divine how people will vote.

I think that argument misses the point. Polls have their uses, particularly when they are used to assess broad questions about what’s on voters’ minds or which issues resonated the most. They seem much less reliable in predicting the future and that’s OK. It means that we have to treat politics more like other subjects, in which we draw on data, interviews, and our past experiences to shape coverage. The rise of the pollster as seer of all matters political is a relatively recent development, a corrective to an era when the “boys on the bus” covered politics largely by anecdote and gut instincts.

Certainly, reams will be written in the coming months and years about why the work of opinion research turned out to be way more art than science when it came to this year’s balloting. The widely predicted blue tsunami was more like a modest midsummer swell. Many state polls were remarkably wrong; Sen. Susan Collins trailed in every survey of Maine voters yet won handily. The estimated 8- to 10-point Biden victory will likely be more like 4% or 5% in the popular vote. The call on Saturday by the Associated Press and other major news organization is based on razor-thin margins in Pennsylvania and other states that were far closer than the pre-election polls had indicated.

To be fair, there were some seriously extenuating circumstances this year. Polls are based on models drawn from previous experience, and modern America has never voted during a pandemic. Nearly every state changed something about its voting process, from drive-thru voting in Texas to the states that widely expanded vote-by-mail. Although it’s not proven yet, it seems likely that the phenomenon of the shy (or perhaps defiant) Trump voter is real; it’s not hard to imagine supporters of a candidate whose main appeal is distrust of political elites misleading or hanging up on researchers calling to ask their presidential preference.

The debacle of 2016, in which opinion researchers universally agreed that a Hillary Clinton victory was the likeliest outcome, prompted a lot of explanation from statistics-savvy journalists. They explained how the FiveThirtyEight forecast, which gave Clinton a 2 out of 3 chance of winning, could be read as giving her a 1 in 3 chance of losing, less likely but certainly possible. In baseball, batters who get a hit 33% of the time wind up in the Hall of Fame. Unlikely events happen. People win lotteries.

Some of this year’s explanations will likely focus on what the polls got right — a comfortable popular vote edge for Biden, a solid win in the electoral college — while nodding to the ways in which 2020 smashed previous models for turnout and participation.

I take a different lesson from the foibles of polling. The most important thing we can do as we think about covering and investigating government and politics in election years is to not assume any outcome. That’s psychologically harder than it sounds, even for professional journalists. In the summer of 2016, news organizations, including ProPublica, did less than they should have on the intricacies of the Trump family businesses in part because his victory was deemed so implausible. Why devote months of hard slogging to a learning about a guy who’s headed to reality television by late November?

It’s difficult for reporters and editors to shut out the polls and work to put their heart into pursuing stories on two outcomes, only one of which will turn out to be true. (I have a friend who was assigned to write a “How Michael Dukakis Pulled Off a Stunning Victory” piece in 1988 that would run on the front page of the Times if the polls were wrong. He felt like he was wasting his time; George Bush’s victory proved him right.)

Still, shutting out the noise and focusing on the evidence is the best way to avoid being part of the group-think that characterizes the coverage of many nationally fascinating stories. I always try to keep in mind the events of April 19, 1995, when a truck bomber demolished the federal building in Oklahoma City. CNN and other networks were quoting experts who said the attack had all earmarks of Middle Eastern terrorism. They reminded viewers that an Islamic radical group had tried to blow up the World Trade Center in 1993 with an explosives-packed rental van. That speculation played in a nonstop loop in nearly every newsroom in America. I recall an angry argument at the Times’ Washington bureau between an editor in New York and a religion writer in Washington who adamantly refused to write a story about the ideology of the possibly Islamic terrorists.

One exception was a tiny organization, the Inter Press Service, which covered the developing world from a small room in the National Press Building and seldom turned on its television. (In fact, its reporters learned of the bombing when someone noticed footage of the smoking wreckage on a monitor in the building’s lobby.) Later that day, Inter Press was the first to focus on the possibility that the bombing had been done by far-right American extremists. One of the reporters, Jim Lobe, had been a lawyer in Seattle before becoming a journalist and was familiar with the militia movements of the Western United States. The story noted that the bombing had taken place on April 19 — the second anniversary of the deadly assault by federal authorities on the Waco, Texas, compound of a religious cult known as the Branch Davidians.

When the FBI arrested Timothy McVeigh and his motive for the attack was exactly as described, Lobe was asked how Inter Press got it right when so many others had not. It was simple, he replied. They knew generally what everyone was saying, but it didn’t make sense and so they did their own reporting.

I tracked Lobe down this week to check the details for this column. He chuckled at the memory. “We usually didn’t have CNN on at all,” he said. “I remember they had on all these commentators who said it was Islamic. It didn’t make any sense. We did the research, found out it was Patriot’s Day, talked to some people who were experts on the far right. Then, we put the story together.”

In an era when polls keep getting it wrong, it feels like good advice. It’s not possible to entirely shut out what they’re saying, but it’s best to do your own critical thinking.

Steve

More From Our Newsroom
If Trump Tries to Sue His Way to Election Victory, Here’s What Happens
People with Developmental Disabilities Were Promised Help. Instead, They Face Delays and Denials.
Judge Orders the Release of Data on Emergency Loans for Small Businesses
Permit for Controversial $9 Billion Plastics Plant in “Chemical Alley” to Be Put on Hold
For the Postal Service, a Frantic Election Day Turns to Finger-Pointing the Day After
Whether the GOP Can Stop Voters From Legally Fixing Rejected Mail-In Ballots Could Decide the Election
Close this screen
Close this screen

Republish This Story for Free

Creative Commons License (CC BY-NC-ND 3.0)

Thank you for your interest in republishing this story. You are are free to republish it so long as you do the following:

  • You have to credit ProPublica and any co-reporting partners. In the byline, we prefer “Author Name, Publication(s).” At the top of the text of your story, include a line that reads: “This story was originally published by ProPublica.” You must link the word “ProPublica” to the original URL of the story.
  • If you’re republishing online, you must link to the URL of this story on propublica.org, include all of the links from our story, including our newsletter sign up language and link, and use our PixelPing tag.
  • If you use canonical metadata, please use the ProPublica URL. For more information about canonical metadata, refer to this Google SEO link.
  • You can’t edit our material, except to reflect relative changes in time, location and editorial style. (For example, “yesterday” can be changed to “last week,” and “Portland, Ore.” to “Portland” or “here.”)
  • You cannot republish our photographs or illustrations without specific permission. Please contact [email protected].
  • It’s okay to put our stories on pages with ads, but not ads specifically sold against our stories. You can’t state or imply that donations to your organization support ProPublica’s work.
  • You can’t sell our material separately or syndicate it. This includes publishing or syndicating our work on platforms or apps such as Apple News, Google News, etc.
  • You can’t republish our material wholesale, or automatically; you need to select stories to be republished individually. (To inquire about syndication or licensing opportunities, contact [email protected].)
  • You can’t use our work to populate a website designed to improve rankings on search engines or solely to gain revenue from network-based advertisements.
  • We do not generally permit translation of our stories into another language.
  • Any website our stories appear on must include a prominent and effective way to contact you.

Copy and paste the following into your page to republish:

Close this menu
  • Graphics & Data
  • Topics
  • Series
  • Videos
  • Impact
  • ProPublica
  • Local Initiatives
  • Data Store

Follow Us:

  • Like us on Facebook
  • Follow us on Twitter

Stay informed with the Daily Digest.

Site Navigation

Sections

  • ProPublica
  • Local Reporting Network
  • Texas Tribune Partnership
  • The Data Store
  • Electionland

Browse by Type

  • Topics
  • Series
  • Videos
  • News Apps
  • Get Involved
  • The Nerd Blog
  • @ProPublica
  • Events

Info

  • About Us
  • Board and Advisors
  • Officers and Staff
  • Diversity
  • Jobs and Fellowships
  • Local Initiatives
  • Media Center
  • Reports
  • Impact
  • Awards
  • Corrections

Policies

  • Code of Ethics
  • Advertising Policy
  • Privacy Policy

Follow

  • Newsletters
  • Podcast
  • iOS and Android
  • RSS Feed

More

  • Send Us Tips
  • Steal Our Stories
  • Browse via Tor
  • Contact Us
  • Donate
  • More Ways to Give
ProPublica

Journalism in the Public Interest

© Copyright 2025 Pro Publica Inc.
Current site Current page