We’ve upgraded our FEC Itemizer interactive database to make it faster, particularly when displaying individual contribution and expenditure records.
It’s now integrated directly into the database behind the Campaign Finance API we launched last month. Before that change, FEC Itemizer parsed each electronic filing on the fly (using Fech, a Ruby library built for that task). For filings with a small number of transactions, things didn’t speed up much, but filings with hundreds or thousands of records will now load noticeably more quickly.
For instance, the FEC Itemizer compares the summaries of a filing that amends an earlier one much more efficiently than it used to. Previously, it loaded two electronic filings into memory and then compared their summary information. Now, because we store each filing’s summary data in a database, we can quickly retrieve just that information to make the comparison, even if the filings contain a lot of activity.
We’re also being smarter about not serving up fresh pages when the data itself hasn’t changed since the last time it was requested.
There are still some filings that are so large that database queries are still slow. For instance, ActBlue, which helps Democratic candidates raise money online by serving as a conduit for donations, recently posted a filing with 5.3 million contribution records. For ginormous filings like that, we’ve turned off displaying individual records from filings for now, and will be adding bulk per-filing downloads by Feb. 19, one day before the next filing deadline.
We’ve begun adding new features to FEC Itemizer, too. Now when you visit a committee’s page, you’ll get three new fields: summary totals for that committee for the election cycle, plus the last date covered by those figures. As the presidential primaries continue, we’ll be adding pages to track independent spending and other aspects of the 2016 campaign.
Since the launch of the Campaign Finance API, we’ve issued keys to more than 75 users (you can get yours by emailing firstname.lastname@example.org) and we have plans to add new responses and data, too. And we’d love to hear your suggestions and requests, either by email at email@example.com or in the comments below.