Useful Code Snippets
Short but fierce. (Image: flickr/geckoam)
Open Source code doesn’t always come in big complex packages. At ProPublica we sometimes share small, simple snippets of using GitHub “Gists.” These Gists rarely have documentation and don’t even always have names, but they can be super-useful. Here are some we’ve shared over the past few years.
Untangling a Web of FEC Data
Our Tangled Web graphic shows the 200 biggest recipients of expenditure money from the five major presidential campaigns (Gingrich, Obama, Paul, Romney and Santorum), as well as from major super PACs, from around the middle of 2011 through February, 2012.
Some Thoughts on Timelines
Over the past few weeks we've done a lot of thinking about timelines. We made one that recounted the history of FDA fines levied on the Red Cross, another one that traced the changing government approach to fracking, and finally one that described the day-by-day unfolding of the Komen/Planned Parenthood debacle. Each was pretty different and went through multiple iterations before arriving at its final form. Here's the story behind each one -- the thinking, planning, and building of each timeline, complete with some of the approaches we rejected along the way.
When Are 190 Emails Like Six Emails?
On Tuesday we published a graphic that looks at six variations of a single email sent out by the Obama re-election campaign last Thursday night.
This all started when fellow news nerd Dan Sinker got an email from the campaign on the same night his wife did and noticed that although they were both apparently from the same person at the campaign — Julianna Smoot — the e-mails had subtle differences. So Dan set up a Google form and asked his Twitter followers to send in their own examples of the “Smoot Email.”
At ProPublica, we’d been wanting to dig more deeply into how “big campaign data” works so we struck a deal with Dan: He’d share his database with us and we’d help analyze and visualize it.
Of course, it was far from a valid sample, but we thought analyzing the data would yield interesting observations if not statistically significant conclusions.
Showing You the Money (Faster)
Over on The New York Times’ Open blog today is a post by Times Developer Derek Willis about the latest update to the Campaign Finance API, which we were excited to help out with.
When we started ramping up our investigation into money in politics late last year, super PACs had just started spending heavily in the early primary states. The Federal Election Commission required these committees to file reports within 24-48 hours of making expenditures, and we thought this data would be a good subject for a news application that would automatically keep up with the rapidly changing data.
Introducing Simple Tiles: Our New Mapping Library
Newsrooms have been publishing maps for a long time. However, mapping on the web is very difficult due to the sheer amount of data involved. Some of the largest datasets available to journalists are geospatial. The U.S. TIGER/Line roads database, for example, tips the scale at 9.2 gigabytes, and the USGS’s national hydrography data set is 17 gigabytes compressed.
The data needed to analyze legislative redistricting is similarly huge. In California alone there are 700,000 Census blocks. When we started covering redistricting last year, we knew we would need an easy way to present vast amounts data quickly. We took a serious look at both Mapnik and Mapserver — who are still leaders in the field — but the Ruby bindings support in both, at the time, weren’t as robust and stable as we needed them to be.
Anatomy of a Stepper Graphic
Today we published a small graphic, “Anatomy of a Trade,” to accompany an article about Freddie Mac by ProPublica’s Jesse Eisinger and NPR News’s Chris Arnold. The article describes some “financial alchemy” performed by Freddie to protect one of their mortgage-backed investments. The stakes were high and the financial details enormously complex, so we wanted to walk readers through things step-by-step.
We were inspired to try stepper graphics, like this one, by Shan Carter of the New York Times, who gave a fantastic talk about them few months ago.
We wrote a small framework to handle the transitioning logic which is over on GitHub. At their base, stepper graphics are just slideshows, and slideshows are just linked lists, so our library simply uses a linked list to control navigation.
The ProPublica News apps desk is:
Safeguard the public interest.
Support ProPublica’s award-winning investigative journalism.
Recent News Apps
News Nerds
Colophon
- Hosting: Rackspace and Amazon AWS (all praise to Varnish)
- Webserver: Apache (with Passenger)
- CMS: ExpressionEngine (via Solspace)
- Site Designers: Mule
- Headline Font: Meta Serif Web Medium (via Typekit)
- What We Code In: Ruby (Rails and Sinatra), JavaScript
- What We Code In but Pretend We Don't: PHP
- Languages We Miss: Perl, Latin
- Favorite HTTP Status Code: 417 Expectation Failed
Get Our Data and Reporting Tools
We frequently publish data, and reporting tools like tipsheets and guides.
Our Hottest Stories
- Donations to Scott Walker Flagged as Potential Fraud
- In Race For Better Cell Service, Men Who Climb Towers Pay With Their Lives
- Billion Dollar Bait & Switch: States Divert Foreclosure Deal Funds
- Pardon Attorney Torpedoes Plea for Presidential Mercy
- Patient Died at New York VA Hospital After Alarm Was Ignored
- Finding Oscar: Massacre, Memory and Justice in Guatemala
- Introducing the ProPublica Patient Harm Community on Facebook
- Built for a Simpler Era, OSHA Struggles When Tower Climbers Die
- Got Student Loans? Share Your Documents With Us
- Remember Stuxnet? Why the U.S. is Still Vulnerable
- Donations to Scott Walker Flagged as Potential Fraud
- Pardon Attorney Torpedoes Plea for Presidential Mercy
- In Race For Better Cell Service, Men Who Climb Towers Pay With Their Lives
- Air Force Pilots Balk at Flying the World’s Most Expensive Fighter Jet
- Patient Died at New York VA Hospital After Alarm Was Ignored
- Watchdog Group Calls for Probe of Lobbyists Behind Congressional Trip to Taiwan
- Billion Dollar Bait & Switch: States Divert Foreclosure Deal Funds
- Finding Oscar: Massacre, Memory and Justice in Guatemala
- Broadcasters Sue to...Block Transparency
- Happy Graduation! Here's The Best, Most Depressing Journalism on Student Debt



