Imagine that a parent responds to a ProPublica call for information about health insurance companies with a story about the financial burden their family faces in accessing specialized care. In their account, they mention that their child frequently misses school because the family has to travel hours to get to medical appointments.
That detail might not be central to the reporting project underway at the time, but it’s a big part of this family’s life. It’s representative of other issues facing their community, too: educational access problems, insufficient accommodations for people with disabilities, rural healthcare shortages and more — all topics worthy of accountability journalism in their own right.
For journalists, these may be different topics, projects and stories. But for the people living through them, they’re all a part of their reality.
Many of the sources who take time to write to a newsroom want the publication to continue covering their communities over the years. Some are frustrated when they don’t get the attention they hope for. But staying in touch relies on journalists’ ability to find, take stock of and sort through thousands of these personal accounts that may have accumulated over time.
While there are lots of tools on the market that help break down and synthesize messy data, we couldn’t find any that meet the security requirements and editorial needs of an investigative outlet.
So we spent the last year building one.
Re-Source is a search tool to help reporters use confidential machine learning to sift through and find unexpected connections in reader submissions, the sometimes-sensitive, idiosyncratic data that lives in multiple databases, often in different formats. The system allows reporters to reengage with any relevant source who had raised their hand to help — while maintaining their security. Today, we’re making the code for the beta version of this tool open-source so more newsrooms can try it with their overflowing inboxes.
Re-Source is for any newsroom that works with tabular source data. Whether your newsroom has an email inbox that receives story ideas or manages submissions over dozens of spreadsheets, as long as you can export your data as a CSV file, you can use Re-Source.
By making more of the material reporters receive from sources actionable, we hope that Re-Source can improve the relationships journalists build with their sources. Creating a tool that allows reporters to find more stories can help newsrooms connect with more people and make it more likely that the effort respondents make by writing in will pay off.
That meant our team had to treat Re-Source as more than a technical search problem. The system needed to:
- Protect sensitive information and tightly control how it can be accessed.
- Find relevant material even when it is described in language a reporter might not think to search for — or embedded in a response primarily about something else.
- Preserve enough of the original context for reporters to be able to evaluate a search result for the story’s needs and help them decide whether to follow up.
These goals dictated the technologies we could use to tackle the challenge.
Finding Responses We Couldn’t Before
Early in the development process, our team collected more than 40 examples of information that reporters thought would be useful for their projects and tested them against various search methods.
Keyword search can be a good approach when a reporter is looking for mentions of a particular company, hospital, government program or person. However, it relies on reporters and sources using the same words to describe what they’re talking about. For instance, it’s difficult to find a keyword that can accurately find multiple responses about the burden of the healthcare system on families when they may be written like, “We drove three hours each day to the doctor’s office.” Many newsrooms hear from people in different languages, and questionnaire and survey data employed by community engagement journalists can often be stored in the form of questions and answers on very specific topics. Creating a search system that would allow reporters to find things by meaning instead of specific terms was key to making material more useful.
Even exact keyword matches may not turn out to be as relevant as they initially seem. As our team gathered more examples of what reporters were looking for, we learned that many look for sources with a specific relationship to a company or organization, rather than anyone who simply mentions it. Depending on each story’s needs, a response about the poultry available in a local Kroger may be less helpful than a respondent who indicates they are a former employee of the retailer.
In order to address these problems, we created a hybrid search system that allows reporters to search by meaning, by keyword or by a source’s relationship to a topic. Using a machine-learning technique known as semantic search, we created mathematical representations of each written response. This is called an embedding. To make sure we didn’t lose any useful context, we joined each response to the callout question it was sent in response to, and we created one embedding per question-answer pair, allowing for higher precision.
Finding sources who have a specific relationship to a topic was trickier. A lot of times, reporters are just as or more interested in the person writing in as they are in the content of their submission. We used a verifiably confidential language model to pull out any parts of the submission that center on the source themselves, such as where they live and what kind of work they do. Then we created embeddings over these outputs.
Navigating New Technologies While Staying Secure
Lots of newsrooms wrestle with the question of how to adopt new technology while still protecting source confidentiality to the highest degree. We weren’t sure if there was even a way to use large language models while meeting the security needs of a newsroom, but through extensive testing with fake data, we found ways to keep newsroom data secure while taking advantage of the advanced logic and summarizing capabilities that LLMs provide.
Re-Source provides options to run local models on hardware you control, or to use a hosted model of your choice that serves an industry-standard OpenAI-compatible API. For hosted models, we suggest looking into providers that offer confidential, private inference backed by hardware attestation, such as Tinfoil. These providers run large open-source models in trusted execution environments that even they can’t see, ensuring sensitive data shared with newsrooms is never exposed to third parties. While your implementation of Re-Source can be end-to-end encrypted, you may still decide that not everything can, or should, go into the system.
Journalist-Centered by Design
Building Re-Source from the ground up meant we could center journalists’ needs at every stage of the product-development process — and test the system with real journalism. Most information retrieval algorithms (think: Google) focus on finding the absolute best hit. Investigative reporters, on the other hand, often want to talk to as many people as possible. Re-Source allows reporters to identify matches at scale for mass outreach.
For instance, could the system find multiple pediatric specialists who could review findings about Idaho children in a faith-healing sect who died of treatable conditions? Could it help us identify people who’ve engaged with us in the past and who live in neighborhoods we’re interested in deepening our connections to?
In these and many other cases, the answer was yes. The more searches we conducted for reporters, the more we learned about what people tend to tell us and what makes it useful to future journalism — or not. When we weren’t able to return any relevant results, that was helpful, too. Finding gaps in our sourcing helped us launch more callouts aimed at people we haven’t reached before.
While Re-Source can find relevant material that a newsroom has already collected, it can’t turn every response into infinitely reusable material. Experiences that were shared a long time ago can fall out of date. Responses always need to be verified for accuracy through thorough interviews and fact-checking. Information that sources share comes with a specific relationship and context, and it’s important to design a responsible search system that grows alongside the relationships journalists have with their sources.
Looking forward, a search system that allows newsrooms to continually maintain relationships with sources can fundamentally change the questions reporters ask and the way we treat these relationships. Rather than treating each project as a discrete unit, we can use each callout as an entry point into an ongoing conversation.
How to Use Re-Source for Your Newsroom
We’re excited to share Re-Source with other newsrooms, and we welcome any feedback as we continue to explore. The system is in beta and we’re still making changes.
The system runs on your own hardware, and ProPublica does not have access to or track any data you input into your newsroom’s instance. To install Re-Source and read our technical documentation, head to our public GitHub repository.
