Journalism in the Public Interest

The ProPublica Nerd Blog

Explore Sources: A New Feature to “Show Our Work”


Marshall Allen's story is annotated with our new Explore Sources feature.

Update, July 29: Our Explore Sources tool is also featured on a new series, Life and Death in Assisted Living. Read the first story.

We’re debuting a new feature today as part of Marshall Allen’s story about one woman’s fight with a Texas hospital to find out how her husband died.

In the course of reporting the piece, Marshall made over 500 annotations in 64 documents he uploaded to DocumentCloud, many of which were sources of facts in his story. He told us about this wealth of metadata, and wanted a way to present it to readers. We agreed that we didn’t want to show them in a separate graphic or interactive feature, but rather sprinkled throughout the story itself.

So we made a special feature we’re calling Explore Sources. To try it, click the “ON” button next to “Explore Sources” at the beginning of the article. Words and phrases throughout the piece will turn yellow. Click these yellow highlights to see the portion of the source document from which Marshall got that fact. Once the annotation is visible, click the document image inside of the popup to go to the full document in DocumentCloud, or anywhere else to dismiss it.

Behind the Scenes

To speed up the process of adding the links, we built a small web application which let Marshall select snippets of text in his story, just as he would in a word processor, and associate it with a DocumentCloud annotation via a point-and-click interface. The tool exports code that we can easily paste into our content management system.

Our newsroom web app helps reporters annotate their stories.

DocumentCloud already had a feature that allows people to embed annotations, and DocumentCloud developer Ted Han graciously added the ability to pass custom callbacks to the document embed function, which made it possible for our software to work.

While “Explore Sources” is just an experiment, we look forward to finding new ways to use it to make our reporting process more transparent and accountable, and when we can we’ll open source the code so other newsrooms can show their work, too.

Update: January 10, 2012: This post has been changed to clarify Marshall Allen's role in the project.

This is an excellent feature. One suggestion I would make is that the popup in the story also include the document’s title and source so that the reader has more context.

Seeing the source material is a huge leap, but I think it would go the final mile if the reader also knew the name and source of the document, like “Deposition by So and So” or “L.A. County Autopsy Report on John Doe”

Fred Magovern

Dec. 16, 2011, 9:37 a.m.

Couldn’t agree more w/ Ben.

Also, it appears that if you’re not part of a newsroom, you don’t have the ability to sign up for DocumentCloud. When I clicked on the pop-up, redirecting me to sign-up for DocCloud and it doesn’t appear I qualify.


I have some gripes w/ ProPublica but you guys are definitely pioneers, which I have to respect.

This is exactly the sort of thing I like to see, and in an era where news outlets are trying to figure out how to make money, this is how it should be done.

I’m not going to pay for press releases and mildly edited AP or Reuters stories.  I’m not going to pay to “see what people posted on our Facebook page” or “the latest YouTube sensation,” which I didn’t want to see in the first place.

If someone offered me even generic news with embedded sources, though, that would get my credit card out.  (Note:  I don’t consider donations here to count as paying for the news.)

Having now gone through that article with the annotations, one thing I don’t like about the system (purely an aesthetic/programming issue) is the auto-scrolling to place the highlighted area at the top of the window.  It’s not disorienting, per se, but it’s less convenient to me than leaving the page where I left it.

I get that it was (presumably) done to make sure that the source isn’t opened outside the window’s view, and it may well be the best tradeoff, but it bugs me, especially if something catches my eye in another paragraph when I click.

James McKinney

Dec. 19, 2011, 11:26 a.m.

Will you be posting the code to your GitHub including the “Behind the Scenes” part?

This is definitely an exciting step to make articles more useful.  I definitely would like to see the source identification.  If available a link to the full document would be way cool.  Sometimes context is the key.

Anyway, congratulations on an excellent idea.

Commenting is not available in this section entry.