ProPublica

Journalism in the Public Interest

Cancel

The ProPublica Nerd Blog

Announcing Simpler Tiles

When Are 190 Emails Like Six Emails?

On Tuesday we published a graphic that looks at six variations of a single email sent out by the Obama re-election campaign last Thursday night.

This all started when fellow news nerd Dan Sinker got an email from the campaign on the same night his wife did and noticed that although they were both apparently from the same person at the campaign — Julianna Smoot — the e-mails had subtle differences. So Dan set up a Google form and asked his Twitter followers to send in their own examples of the “Smoot Email.”

At ProPublica, we’d been wanting to dig more deeply into how “big campaign data” works so we struck a deal with Dan: He’d share his database with us and we’d help analyze and visualize it.

Of course, it was far from a valid sample, but we thought analyzing the data would yield interesting observations if not statistically significant conclusions.

What the Frack is in That Water?

Environmentalists have repeatedly pressed regulators to compel oil and gas companies to report what chemicals they use in the drilling and fracking process. No one knows the exact makeup of the frack mixture or drilling muds, but this list breaks down the main ingredients revealed so far.

Message Machine: Reverse Engineering an Obama Email Campaign

Campaigns are increasingly tailoring their messages—and their funding requests—using massive databases of personal information about potential voters. Here are six variations of a Thursday night message from the Obama campaign, based on emails submitted by 190 recipients across the country.

Ponying Up: How Much Have Big Banks Been Docked for the Financial Crisis?

Nearly four years after the financial crisis, settlements with the big players on Wall Street keep coming out, one after the other. It can be hard to keep track of it all. So who’s been hit, with what, and for how much in total?

Showing You the Money (Faster)

Over on The New York Times’ Open blog today is a post by Times Developer Derek Willis about the latest update to the Campaign Finance API, which we were excited to help out with.

When we started ramping up our investigation into money in politics late last year, super PACs had just started spending heavily in the early primary states. The Federal Election Commission required these committees to file reports within 24-48 hours of making expenditures, and we thought this data would be a good subject for a news application that would automatically keep up with the rapidly changing data.

Taking Stock of the Stock Act: A Side-by-Side Comparison

The Stop Trading on Congressional Knowledge Act, or Stock Act, recently passed in both chambers of Congress. We break down the main differences between the House and Senate versions, with a real-life scenarios that illustrate activities the bill targets.

Introducing Simple Tiles: Our New Mapping Library

Newsrooms have been publishing maps for a long time. However, mapping on the web is very difficult due to the sheer amount of data involved. Some of the largest datasets available to journalists are geospatial. The U.S. TIGER/Line roads database, for example, tips the scale at 9.2 gigabytes, and the USGS’s national hydrography data set is 17 gigabytes compressed.

The data needed to analyze legislative redistricting is similarly huge. In California alone there are 700,000 Census blocks. When we started covering redistricting last year, we knew we would need an easy way to present vast amounts data quickly. We took a serious look at both Mapnik and Mapserver — who are still leaders in the field — but the Ruby bindings support in both, at the time, weren’t as robust and stable as we needed them to be.

Updated Recovery Tracker

Once again, we’ve taken all the data used on the government’s stimulus Web site, Recovery.gov, spiffed it up and added thousands of other recovery spending records — the law doesn’t require all recipients to report to Recovery.gov.

Komen’s Contortions: a Timeline of the Charity’s Shifting Story on Planned Parenthood

The Komen foundation’s decision to cut funding to Planned Parenthood set off such an uproar that the charity quickly gave it back. We trace how their explanations changed along the way.

From Gung-Ho to Uh-Oh: Charting the Government’s Moves on Fracking

Fracking has only recently become a household word, but government involvement with the drilling technique goes back decades. We trace officials’ moves—and levels of caution—over time.

PAC Track: Now Includes Contributions

What and where are the super PACs spending?

Anatomy of a Stepper Graphic

Today we published a small graphic, “Anatomy of a Trade,” to accompany an article about Freddie Mac by ProPublica’s Jesse Eisinger and NPR News’s Chris Arnold. The article describes some “financial alchemy” performed by Freddie to protect one of their mortgage-backed investments. The stakes were high and the financial details enormously complex, so we wanted to walk readers through things step-by-step.

We were inspired to try stepper graphics, like this one, by Shan Carter of the New York Times, who gave a fantastic talk about them few months ago.

We wrote a small framework to handle the transitioning logic which is over on GitHub. At their base, stepper graphics are just slideshows, and slideshows are just linked lists, so our library simply uses a linked list to control navigation.

SOPA Opera Update: Opposition Surges

SOPA Opera

Well-funded interests on either side of SOPA and PIPA are lining up support among members of Congress. This database keeps track of where members of Congress stand.

SOPA Opera: Which Legislators Support SOPA and PIPA?

Announcing the ProPublica News Apps Fellowship

The ProPublica News Apps desk is looking for a smart, technically-savvy journalist to join our team for a pilot project we’re calling a News Applications Fellowship.

In this special internship, which is paid and will run until the end of the year, you’ll help us test a hypothesis: Can a smart, technical journalist with excellent and proven skills in other nerdy newsroom disciplines like graphics and CAR become a news app developer?

Rep. Jerry McNerney’s District

Democrats recognized that they could protect Jerry McNerney from being redistricted out of office by the Citizen’s Redistricting Commission.

Adaptive Design, Fixed Widths and Tablets

If you’re trying to make your fixed-width site adaptive, there are some things you need to know about the viewport tag.

Explore Sources: A New Feature to “Show Our Work”

We’re debuting a new feature today as part of Marshall Allen’s story about one woman’s fight with a Texas hospital to find out how her husband died.

In the course of reporting the piece, Marshall made over 500 annotations in 64 documents he uploaded to DocumentCloud, many of which were sources of facts in his story. He told us about this wealth of metadata, and wanted a way to present it to readers. We agreed that we didn’t want to show them in a separate graphic or interactive feature, but rather sprinkled throughout the story itself.

So we made a special feature we’re calling Explore Sources. To try it, click the “ON” button next to “Explore Sources” at the beginning of the article. Words and phrases throughout the piece will turn yellow. Click these yellow highlights to see the portion of the source document from which Marshall got that fact. Once the annotation is visible, click the document image inside of the popup to go to the full document in DocumentCloud, or anywhere else to dismiss it.

The News Apps Team

Hack With Us

ProPublica hosts newsroom developers -- or developers who want to see what it's like to work in news -- for 3-5 day job shadowing residencies called the ProPublica Pair Programming Project, or P5.

Download Our Data

Use ProPublica's data -- cleaned, categorized and often created from multiple sources -- in your reporting and research.

Use Our Code

Explore Our Work