Late last month, a pioneer of the civic-tech and open-data movements announced they were changing direction in a big way. The Sunlight Foundation, founded a decade ago to advocate for a more “open and accountable government,” wrote on their website that they were planning to shutter Sunlight Labs, a division dedicated to building useful tools out of the often arcane data released by federal and state governments. The survival of the rest of the institution is also in doubt.
As editor of the team that builds ProPublica’s interactive databases — similar to the work of Sunlight Labs — the news came as a shock. While our first concern was with the colleagues and friends who would be laid off, we feared for the future of the important work that Sunlight Labs has done.
So when Kat Duffy, Sunlight Labs current and last director, asked publicly for organizations to adopt Sunlight Labs’ projects, we quickly raised our hands. (By the way, have I mentioned that Kat is on the market?)
Today we’re announcing that ProPublica is taking over five of Sunlight Labs’ projects:
- Politwoops, which tracks Tweets deleted by politicians.
- CapitolWords, which lets users search politicians statements in the Congressional Record, and track ideas over time.
- House Staff Directory, which includes information on congressional staffers available elsewhere only by expensive subscription.
- House Expenditure Reports, a repository of office expenditures made by members of the House of Representatives.
- Congress API, which provides data feeds programmers can use to create web and mobile applications to track the work of Congress.
The transition will begin immediately. Users of these products will see little change at first. ProPublica is taking ownership of the domain names, servers and github repositories and will keep them running as is while we focus our energy on preparing for Election Day.
In the weeks after Election Day, and over time, the Sunlight projects will begin to integrate elements of ProPublica’s look and feel, as well as our branding and domain name. We’ll be integrating the House Staff Directory and Expenditure projects into our Represent news application. We will move users of the Sunlight Congress API to our own Congress API, which uses many of the same data sources and has much of the Sunlight API’s functionality (and we’ll be adding the rest).
Sunlight’s Blog has details on the other projects that have been adopted.
It would be hard to overstate how important Sunlight Labs’s work has been to people who do the kind of work we do on the News Application Desk at ProPublica. Sunlight’s founders envisioned the organization a decade ago not as a traditional D.C. think-tank, but more like a Silicon Valley startup: entrepreneurial, experimental, and nimble. Their open-data advocacy work has always been amplified and protected by their status as interactive data practitioners. Far from bloodless white-paper authors, Sunlight has always been a smart bunch of nerds with real skin in the game.
Many in the civic tech movement see government bureaucrats, not regular people, as their key constituency. As advocates, they too often see “rewiring government” as an end in and of itself, and consider actually doing something with the data, as programmers say, “out of scope.”
Not so Sunlight, which has traditionally been comfortable taking stands and using data to provide real accountability. They hired real live investigative journalists to work alongside their open-data experts and civic-tech nerds, and they published stories that used open data to take bites out of injustice and corruption. Sunlight Labs was their way of demonstrating the promise of open data and civic tech.
What better way to help show the benefits of opening up state legislative data than with Open States, an unprecedented, exhaustive trove of state legislative data (which will become an independent project run by the remarkable community that has grown up around it). How better to help us all understand the amount of time representatives spend fundraising than through Political Party Time, a crowdsourced effort that has collected thousands of invitations to swanky D.C. fundraising receptions that happen while Congress is in session and while their top-billed representatives and senators should be doing the work of their constituents?
With Labs, Sunlight could answer the question “What is open data?” just as Miles Davis used to answer the question “What is Jazz?” — “I’ll play it first & tell you what it is later.”
The Sunlight Foundation was named for Justice Louis Brandeis’ maxim that “sunlight is the best disinfectant.” That sentiment, with an awfully similar metaphor, is at the very heart of ProPublica’s mission: “to spur reform through the sustained spotlighting of wrongdoing.”
As journalists we are big believers in government transparency. We count on it to do our work. For us, the story doesn’t end when transparency is achieved; it’s just getting to the good part.
Open data and civic tech are worthy notions, but without putting them to real-world use in the service of the needs of regular people, they never achieve their greatest potential: to empower all of us to hold powerful people and institutions to account, and to help us all make better decisions when we pick a doctor, take a vacation, elect a government, etc. Justice Brandeis’ second most famous quotation also very much applies: “the most important political office is that of private citizen.”
After Sunlight’s announcement that they were closing Labs, one commenter observed that “we’re in the open-data backlash phase now. Open data hasn’t lived up to its hype (how could it?) and everyone’s asking what the point is.”
To me, Labs always understood “what the point is,” and made that point clear through their great work.
We’re sad to see Sunlight Labs shut down, but we remain convinced of the power of data to help regular people scrutinize their government, live better lives, and to understand their world. We’re grateful to the Sunlight board for entrusting us with these projects, and we’re eager to help see that the mission that animated their creation continues.