Today ProPublica is launching a new tool, created in partnership with the Google News Lab, that makes it easier for journalists, researchers and citizens to quickly find newsworthy information about the presidential race and congressional campaigns in their states. We’re calling it the Election DataBot because it collects huge amounts of data and reports the most interesting details, in real time — details about campaign finance filings, congressional votes, polls and Google Trends data, among other things.
Think of Election DataBot as your campaign tipsheet: It provides leads, giving you the opportunity to turn them into great stories. It builds on the resources in our other political databases, FEC Itemizer and Represent.
How to use the Election DataBot
These how-to videos will get you started.
The centerpiece of the Election DataBot is the Firehose, a stream of information pulled from a variety of sources that updates every 15 minutes. The Firehose lets you see a huge amount of data as it comes in. If you’re interested in just Arizona’s Senate race or New Hampshire’s 1st District race, you can get a specific feed of items for those individual races, too.
All of the pages in the DataBot are updated regularly and filtered to show information most useful to reporters and others who need real-time intelligence about the campaigns. It will tell you when a super PAC spends money in a race you care about, when an incumbent member of Congress is the only person to vote against a piece of legislation, when key polls change, and much more.
You can also sign up for email alerts for a particular candidate, committee or race, using a Google account.
We pull data from several sources, including:
- Campaign finance filings from the FEC, updated every 15 minutes using the ProPublica Campaign Finance API. This includes 48-hour notices of funds received, certain regular filings, independent expenditures, and correspondence between committees and the FEC.
- Search trends from Google Trends, including election opponent head-to-head comparisons and trend spike detection, updated every hour.
- Congressional voting data (for incumbents, of course), updated every 30 minutes, via the ProPublica Congress API.
- Race ratings for every House and Senate election from the Cook Political Report, updated daily.
- Polling data from the Huffington Post Pollster API, updated daily.
- Forecasting data for the presidential contest from FiveThirtyEight, updated daily.
We’re interested in what journalists and researchers find interesting or useful within the Election DataBot, and we’re working on adding additional items to the Firehose. Let us know, either in the comments or at email@example.com, what works, what doesn’t, and what else you’d like to see between now and Nov. 8.
Campaign Finance Data
- Comparing candidates. Using campaign finance data from the Federal Election Commission, the Election DataBot ranks House and Senate candidates in terms of money raised and spent, both within their states and nationally. How do House candidates in your state compare to all House candidates? Where does the incumbent senator in your state rank among Senate races this year?
- Comparing races. Election DataBot ranks the money raised and spent in each House and Senate race, so you can quickly see where a local or state race fits in the national picture.
- Seeing late shifts in campaign donations. In the final weeks before a primary or general election, House and Senate candidates are required to file reports to the FEC within 48 hours of receiving a contribution of $1,000 or more. They provide an early glimpse at donations that will be reported after the election. These “48-Hour” reports can show that a candidate’s campaign is seeking donations from congressional colleagues, family members and out-of-state donors and can be an indication that a campaign believes it has a more difficult race than expected. For example, the string of 48-Hour reports by Kansas Republican Tim Huelskamp before his competitive primary included contributions from his mother. The 48-Hour filings are alerted via the Election DataBot, which links to a committee’s page on FEC Itemizer.
- Knowing what organizations are spending money in a race you care about. The Election DataBot provides a way to track a race or candidate, so you get alerts when super PACs, other committees and nonprofit organizations spend money to influence the election. You also can track individual committees, seeing where and when they spend money in races. FEC Itemizer provides more details about specific committees, including a map of the states where they are spending money.
- Exploring regular filings of contributions and expenditures. Candidate committees, parties and PACs all file regular reports listing contributions and expenses, as well as amendments to those reports. The Election DataBot alerts you to when those filings, known as “F3” reports, have been submitted, and FEC Itemizer shows individual transactions in those filings, so you can look for familiar names (or ones that are new to you). You also can compare a report filed this year to the same one from previous elections. For example, Arizona Democrat Kyrsten Sinema raised more in her 2016 pre-primary filing than she did in 2014 or 2012.
- Seeing what campaigns are telling the FEC. When a campaign’s filing has some mistakes or issues, the FEC often will send a note. The campaign’s response, usually contained in an “F99” filing, can shed light on reporting oversights or other details that don’t get a lot of attention.
Some caveats: the Election DataBot attempts to bridge the worlds of congressional and campaign finance data, presenting important information in a timely manner. While congressional data comes directly from official sources, FEC electronic filings are considered unofficial when first submitted.Some committees make mistakes when filing, usually by omitting information or not identifying candidates or races correctly. Nonetheless, they are valuable sources of information for reporters, who rely on them for the latest details.
While Senate races often attract more attention, Senate candidates don’t file electronic campaign finance reports, instead submitting them on paper. The Election DataBot does keep track of those filings, but not in real time, only when they are posted online by the FEC. It does track most outside spending in Senate races in real time, however.
Congressional Voting Data
Part of covering politics includes helping readers understand an incumbent’s voting record. The DataBot makes that easier.
The DataBot pulls information from our congressional votes database and links out to Represent, our database of congressional activity. The Firehose is limited to rare events, like a lawmaker voting against his/her party, or missing a vote. Party voting doesn’t necessarily speak to ideology, but can show lawmakers who don’t always adhere to the party line.
If you want to delve more deeply into any incumbent’s voting record, our full Represent database is only a click away. There you’ll find every vote for all members in the current Congress, and the percentage of the time they vote against a majority of party colleagues. When lawmakers miss votes, they sometimes explain why, and you can find those explanations at our Personal Explanations app.
Horse Race Data
Polling, for all of its faults, is one way to learn about the status of an election. At the other end of the spectrum are political analysts who study the competitiveness of races. The Election DataBot tracks both, providing alerts on new polls in the presidential and congressional races, election forecasts based on poll analysis, and race ratings.
Using data from the Huffington Post Pollster API, we show the latest polls and link to documents detailing the results. FiveThirtyEight’s Election Forecast provides win probabilities for the presidential race in every state and nationally. When those percentages change, Election DataBot will let you know. Finally, the Cook Political Report produces independent, non-partisan analysis of elections and rates each race on a spectrum of “Solid” to “Tossup”. When a race rating changes in a House or Senate contest, Election DataBot has it, along with other information that can enable you to piece together why that might be, such as campaign filings.
Google Search Trends
What internet users search for can be a useful indicator. Search interest, while not a perfect predictor of electoral performance, has caught the attention of researchers. In the Republican presidential primary, Google search interest tracked closely with results in a number of states.
Election DataBot uses data from Google Trends to show when search interest - represented by a sample of Google searches - increases for a particular candidate within his or her home state. Many candidates see relatively little search interest, so we try not to surface small increases, instead looking for rising interest over a number of hours. When that happens, Election DataBot provides an alert.