ProPublica

Journalism in the Public Interest

Cancel

The ProPublica Nerd Blog

No Windows. One Exit. Free Drinks: Casino-Driven Design for Crowdsourcing

.

Why do we call it casino-driven design? Casinos are notorious for adopting an interior design that keeps people gambling. (Flickr: vorticeassurdo)

During the 2012 election, we created Free the Files, an interactive news application based on crowdsourced data, built in real time by thousands of volunteers. It was a collaborative effort to track TV ad spending by campaigns, super PACs and so-called “dark money” nonprofit groups in the country’s top swing markets.

Measured by participation rate, Free the Files was an astonishing success. More than 1,000 contributors submitted over 94,000 transcriptions to help turn messy invoices from local TV networks into clean data. One volunteer transcribed over 28,000 filings. Each transcription was “verified” after two or more users agreed on all of its data points. There are currently around 17,000 verified filings, and people are still working.

Much of this success came from the efforts of our expert engagement team, who motivated and interacted with our volunteers every day, and who hatched a clever campaign of social media, contests and promises of free T-shirts. Our users were, of course, also motivated by their own sense of civic responsibility, wanting to help build the first free database of political TV ad spending.

But design played a significant role as well. We kept our eye on optimizing each page for participation. We called the design we devised for participation-oriented areas of the site “Casino-Driven Design.” A variant of Behavior Design, Casino-Driven Design cuts away all distraction and drives the user’s attention toward staying focused on a single task.

No Windows. One Exit. Free Drinks.

Why do we call it casino-driven design? Casinos are notorious for adopting an interior design that keeps people gambling. There are no windows and no clocks so it’s easy to lose track of how long one has been gambling.

Casino-driven design creates an optimal atmosphere for task completion by actively discouraging cross-site exploration and page exits. And, like in a real casino, we keep the small rewards flowing, such as seeing your name on “freed file” pages and on a leaderboard.

There are no site-wide template elements on casino-driven pages — no section links, ads or even a link to the homepage. Casino-driven pages start out as a blank white page. We add only the elements necessary for users to understand where they are and the nature of the next task. The only way off the page is to complete the task or abandon it.

But there’s more to casino-driven design than just a clean page.

Glass Doors

There are two important goals in getting people to complete crowdsourcing tasks: Getting them to start volunteering, and getting them to keep going.

We tried out a bunch of strategies for getting people into the app for the first time, but our favorite was the “glass door”: We show a tantalizing taste of the activities that will become available after a user takes the first step — signing up:

What's behind the frosted glass door?

Behind the “frosted glass” is a view of what you’ll be able to do once you’ve logged in. It’s hard to get to that screen and not want to see what’s behind it.

We also helped our readers find an activity using yellow and blue wayfinding at the top of each page:

Wayfinding bars with real-time status data encouraged participation.

Yellow status bars across all of our apps mean “this stuff is about you.” In Free the Files, we used it to give people quick links to logging in, an at-a-glance look at how many files they’ve freed, and what’s coming up. Light blue boxes and bars across our apps mean “here’s the most important thing you can do” (we call this shade of blue “do-something blue”). When we launched an election-day contest to transcribe all the files in Las Vegas, we put a blue bar on top of the pages to let them know there was an important activity to take part in. By the end of the single-day Las Vegas challenge, we had transcribed every file in the market.

Low Friction

Once inside, we keep the click targets really big and bright, and the number of actions as low as we can get away with. We spent a long time editing down the number of data elements we asked our readers to help us transcribe. In the end we pared down to four elements — and after launch we found ways to cut that down even lower.

Once the user has transcribed them, he or she can mash the big green or red button. Quite satisfying!

A document "do" page.

To keep people around longer, we engineered the page never to fully reload. There was never a reason for people to leave the page — we provided pop-up instructions and autocomplete boxes for values we thought users may type in. For example, we preloaded the “Who bought it” autocomplete field with committee names from both the FEC’s database and the FCC’s file naming scheme so users could start typing and choose one rather than try to decode the often confusing forms.

Finally, we borrowed a trick from the gaming world by adding a leaderboard so that users could (justifiably) brag about how much they were contributing. They became super-competitive about their spot on the board. During the election-day challenge, we showed users their single-day count right in the yellow bar on each page to help them keep track of their score.

The leaderboard.

Users were also able to brag in social media about freeing files — in a popup, without leaving the casino.

Casino-driven design was resoundingly successful in our election-related apps. We later used it to help drive participation in our Message Machine project. We’re excited to keep evolving it when we build crowdsourcing apps.

Looking Ahead

Casino-driven Design is all about reducing friction to participation. We showed Free the Files to some MIT computer-science profs and they gave us some interesting ideas we’re eager to try in future casinos:

One of the things that made transcribing the ad contracts difficult was that there were a plethora of proprietary page layouts that each station or network of stations used. This meant that users had to hunt around for the same data points in different places on different filings. It’s possible to cluster similar-looking documents using a technology called computer vision and then only show one kind of page to each user. This ought to speed their work up.

We also plan to experiment with presenting users with a single task — say, transcribing just the date on a single form design — and repeating the task over and over again. This may let the unconscious mind take over and speed task completion enormously. Different users would see different elements to transcribe, so we’d still end up with the same data, just split up more atomically.

Beyond that, we could also ask users simply to draw boxes around the spot where each data point can be found in these various formats and then write software to look in those places and use OCR to transcribe what it finds. OCR isn’t a good fit for analyzing entire documents like in Free the Files (especially scanned or faxed pages), but if we have human help to guide the OCR on these boxes, we may eventually be able to grab data out of forms with only human verification.

Two suggestions come to mind, though I don’t have the clout of teaching at MIT.

First, I was probably scared away by the “casino,” since I sort of caught that I could end up spending all night going through.  So it might be worth finding some way of letting people work more casually.

Second, and sort of the reverse of my first idea, one thing I really wanted to see was, on the way out, being offered the option to double-check my work.  I knew a couple of my first runs through were very wrong, and very much wanted the chance to fix it, rather than just passing the burden on to the crowd-sourcing system outvoting me.

As a bonus, it probably would have gotten me to “just do a couple more.”

It’s not necessarily related to engagement, but another vague unease I had was the apparent obsession with the Ohio market.  I wasn’t worried about contributing to a market where I didn’t have a vested interest, but in experimenting with the interface, I only saw Ohio data no matter what I requested.

Until I had a conversation with someone at ProPublica about it, I assumed it might be a bug, which is never good for keeping people’s attention.

Don’t give up on OCR, though!  It might not “work” the same way it works for Project Gutenberg, but you should get pretty good results if you think about the output as an intermediate form fit into the two-dimensional space of the form.

Instead of looking for text, you’d want to look for each character with its position on the page.  Overlay that onto the grid of the form (all the forms I saw were on a grid), and it should be much simpler, at least, to guide the user guiding the recognition, so to speak.

Bonus, if you can identify the different fonts and sizes, since I remember that sometimes being useful in distinguishing between labels and content.

@John Good points. We were continually tweaking the market assignment algorithm. At first it was based on “swinginess” of the market, which weighted heavily toward Ohio. Later we made it more random, to show better spending trends across the country. We’re doing a bunch of experimenting with OCR and grabbing bits of data out of messy PDFs. More on that soon…

Al

Great work, folks!

I didn’t know about this effort.  Wish I had.  Probably best that I didn’t.
Reading the article now, two thoughts come to mind, both involving possible ways to capitalize on casino-driven design, not necessarily related to crowd-sourcing.  (I don’t know the IT design business or your interests, so these may be way off base.  If so, sorry.)
1.  Programs for those with ADHD, especially adults (because I am one), for personal or business use.  About 4%  or 8 million adults in the U.S. have ADHD.  Characteristics include :  forgetful, achieving below potential, stuck in a rut, time challenged, motivationally challenged, impulsive, novelty seeking, distractible, and scattered (“Fast Minds” by Craig Surman, M.D. and Tim Bilkey, M.D. with Karen Weintraub, c. Harvard University 2013).  There has to be a market for a program that could help someone do taxes instead of reading and responding to Pro Publica articles.
2.  Programs for businesses/organizations to keep their employees focused.  What a terrible idea.
2.  Programs for business use - -  probably exists already; how sad.

As far as I know, Lynn (which isn’t much—I’m not a interface/experience guy except when nobody else is willing to do it on my projects), that’s exactly where a lot of the research into this sort of design approach comes from.

I don’t know why it hasn’t hit the market, though, other than the “focus-oriented” word processor like FocusWriter.  It’s more or less the same principle.

Al, I didn’t mean to sound like I was condemning the choice of focusing on Ohio.  I meant more to suggest that, if the algorithm is swaying one way or another, it might be helpful to make it clear that this is the priority.

I don’t think any of us was chomping at the bit to look at a particular market, so just knowing that Ohio was the server’s primary target would have been fine.  Why it was that way would’ve been interesting, too, but that gets into distraction territory, especially to someone like myself who’s interested in the code and the math behind it…

difficult

I think being able to work on a particular market could entice people to do more work. I know if I was working on those, I would prefer to work on states I have lived in, some of which are swing states.

Some people don’t care, but others, especially those living in a swing state, might want to work on their state. You could add in a weight for the state the worker is in—maybe using their IP address or during an initial questionnaire if you have one.

My business relies on crowdsourcing (for audio transcription), and there are a number of interesting ideas in here I’m going to think about integrating into our site.

We also have a leaderboard, although now I want to check to see if it’s a daily leaderboard or just an all-time one. I like the idea of tweeting/FBing accomplishments. That works as a promotional tool as well. I wonder how many new users you picked up that way?

I think that badges (e.g. Foursquare) could be a nice addition. We have found that our workers enjoy earning a badge and it could be a way to entice people to do particular sorts of work or complete a couple more jobs. 

The lack of navigational links is interesting but not sure I can bring myself to do that on my own site. In terms of usability that is a bit of a no-no, but I can see the benefits. Of course, I find in real life that distractions are more than just the particular website I’m on—it’s easy to have, say, the NYT up in another tab, or check Facebook on my phone.

However, flashing updates about where they are on the leaderboard, or how many jobs they’ve completed (or $$ they’ve earned if applicable) could help keep them focused on the task at hand, and prevent them from clicking out to check on their status.

all in all, very interesting project and use of crowdsourcing.

Really interesting Al, thanks!

Given your interest, I think that you (and the other readers here) would be really interested in some recent research that I have come across that theorizes about crowds and such similar phenomena.

It’s called “The Theory of Crowd Capital” and you can download it here if you’re interested: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2193115

In my view it provides a powerful, yet simple model, getting to the heart of the matter. Enjoy!

Add a comment

Email me when someone responds to this article.