Last month we took over five projects that were created by The Sunlight Foundation. Here’s an update on where those projects stand now and our future plans for them.
Sunlight Labs shuttered on Nov. 11. A momentous date (in fact, a federal holiday) and as you may recall a busy week for everybody in news. Sunlight staffers Kat Duffy and Bill Hunt worked tirelessly to find a good home for Labs projects. They had a hard deadline to get everything placed — on Nov. 11, Sunlight Labs’ servers were to be turned off and the staff laid off.
We’ve spent the past few weeks understanding the code and data that Kat and Bill sent us and making a plan for migrating everything to ProPublica URLs, designs and systems. Our plan is to get them all up and running first and then make any fixes, changes or improvements, including integration with existing ProPublica projects.
Three of the five projects are currently online, while the other two are not. Most of the projects are connected to Congress in some way, which means time is of the essence: The next House and Senate will convene on Jan. 3, 2017, and we want to have these projects as ready as possible by that date.
Here are specific updates on each, including some volunteer opportunities for coders looking to chip in:
Sunlight Congress API: The API is currently running at its original address and supports all of the calls it did before the move. What’s missing right now is the ability to sign up for an API key, so we’ve turned off the requirement to have one for the moment. If you are an existing API user, don’t change anything. If you are a new user, you can use the API right now without a key, but we will require keys once we have a solution in place. We will be moving the API to a propublica.org address, but none of the other parts of the URL will change, and we do not anticipate removing any existing responses.
Much of the legislative data that powers the API comes from the United States project. We’ll be contributing to the efforts there and we invite users of the API to get involved, too, by contributing code for new or existing sources of information, or by updating information about members of Congress.
ProPublica has its own Congress API (which started life as The New York Times Congress API), but Sunlight’s is more robust and has many more users, so our plan is to merge the ProPublica Congress API into the Sunlight API to that we can offer a single service. The process will begin this month and continue into early 2017. Once we’ve completed that transition we’ll have a single Congress API. If you are interested in helping us test things out during the transition, email us at [email protected].
Politwoops: This popular service that tracks deleted tweets by politicians and elected officials is running at a new URL, and we expect to move it again soon (don’t worry, we’ll redirect links). The only other changes involve the design of the site and the addition and removal of politicians. We’ll also look to integrate Politwoops data into Represent, our congressional news application.
House Expenditure Reports: We relaunched this project as part of Represent at the end of November and will continue to maintain it going forward. It’s also available as a free download in the ProPublica Data Store. We’ve updated the data to include office expenses from the third quarter of 2016, and we’ll be adding new features to Represent based on the data. We hope to have a public search interface in addition to providing data downloads.
House Staff Directory: We are working on integrating data on House staffers into Represent, and expect that it will be completed in mid-December. As with the expenditure reports, we’ll continue to update the data quarterly.
Capitol Words: This project that analyzes the words spoken by lawmakers has both an API and a public web application. It is not running yet, and our goal is to have it back up in time for the beginning of the new Congress in January at the same URL, and then to integrate parts of it into Represent. We will continue to support the API, though the URL will eventually change. We’re particularly interested in hearing from Capitol Words API users on how they use the service and what changes they might make. Email us at [email protected].
Although these applications will look different than they did a month ago, we remain committed to preserving their utility and to using them to make ProPublica’s own news applications better.