Congressional Data Mining: Coming Soon?

How a little-noticed provision in a House spending bill could revolutionize access to congressional information.

Photo by flickr user <a href="http://www.flickr.com/photos/ianalexandermartin/1846815435/" target="new">I am I.A.M.</a> used under a Creative Commons license.

Fight disinformation: Sign up for the free Mother Jones Daily newsletter and follow the news that matters.


By slipping a simple, three-sentence provision into the gargantuan spending bill passed by the House of Representatives last week, a congressman from Silicon Valley is trying to nudge Congress into the 21st Century. Rep. Mike Honda (D-Calif.) placed a measure in the bill directing Congress and its affiliated organs—including the Library of Congress and the Government Printing Office—to make its data available to the public in raw form. This will enable members of the public and watchdog groups to craft websites and databases showcasing government data that are more user-friendly than the government’s own.

If the Senate passes the bill with the provision intact, citizens seeking information about Congress’ activities—such as bill names and numbers, amendments, votes, and committee reports—won’t have to rely on government websites, which often filter information, are incomplete, or are difficult to use. Instead, the underlying data will be available to anyone who wants to build a superior site or tool to sift through it. “The language is groundbreaking in that it supports providing unfiltered legislative information to the public,” says Honda’s online communications director, Rob Pierson. “Instead of silo-ing the information, and only allowing access through a limited web form, access to the raw data will make it easier for people to learn what their government is doing.”

Successful, privately-created websites that provide the public with information about Congress’ actions already exist. OpenCongress.org, GovTrack.us, Legistorm.com, and MAPLight.org all make legislative data available to the public in ways that are easier to navigate than Congress’ primary web portal, a system called Thomas. Those sites currently get their data through techies who “scrape” Thomas and other government websites, which means they use bots to process the HTML and gather what is valuable. The process is labor-intensive and imprecise. “It’s difficult to keep the data up to date, in some cases impossible, and occasionally there are errors in the data,” says Josh Tauberer, the 26-year-old who runs GovTrack.us and does lots of the “scraping” that others use. “This could all be fixed by a bulk data download.”

Tauberer expects that the availability of additional and easier-to-use congressional data will spur innovation. “You can expect to see other sites spring up doing new and interesting things with the information.” He anticipates charts, graphs, and maps that represent congressional goings-on visually—”ways of visualizing the congressional process that we couldn’t yet imagine.” Honda, with his Silicon Valley roots, expects that developers and coders will quickly outpace the government’s efforts to date. “We hope that we can learn from the wisdom of crowds,” says Pierson.

There are government agencies that already provide massive amounts of data via databases. The Census Bureau provides huge amounts of information in raw form, allowing academics, statisticians, and think tank scholars to comb through it in any way they please. The Federal Elections Commission publishes unedited data on campaign contributions, giving rise to sites like OpenSecrets.org, which allows the public to see who is donating to whom, and allows journalists and watchdogs to investigate the influence of money in politics.

“In our Web 2.0 world, we can empower the public by providing them with raw data that they can remix and reuse in new and innovative ways,” Honda told Mother Jones in a statement. (Disclosure: In the summer of 2002, I briefly worked as an intern in Honda’s district office.) Honda’s provision, however, pertains only to legislative data. Federal departments like the Environmental Protection Agency, the Food and Drug Administration, and the Department of Energy have reams of data that political scientists, economists, and researchers of all stripes would love to get their hands on. Many who work at the intersection of technology, politics, and transparency believe that the key player in broadening Honda’s effort to include the executive branch will be Vivek Kundra, the former Chief Technology Officer of the District of the Columbia who was named Obama’s Chief Information Officer on Thursday. According to the National Journal‘s Tech Daily Dose, Kundra “told reporters Thursday he will launch data.gov, a Web site intended to ‘democratize data’ by giving the public raw feeds of information from a range of agencies.”

John Wonderlich, the policy director at the Sunlight Foundation, which has created or funded several tools that make government data easier to analyze, is holding out hope that the president’s Open Government Directive, which is due at the end of May, will further address the issue of data availability. He applauds Honda for putting Congress, at least, on the right track. “Without Honda’s attention to this issue, congressional level attention to bulk data access would be unlikely,” he says. “We’re happy to see this first step.”

AN IMPORTANT UPDATE

We’re falling behind our online fundraising goals and we can’t sustain coming up short on donations month after month. Perhaps you’ve heard? It is impossibly hard in the news business right now, with layoffs intensifying and fancy new startups and funding going kaput.

The crisis facing journalism and democracy isn’t going away anytime soon. And neither is Mother Jones, our readers, or our unique way of doing in-depth reporting that exists to bring about change.

Which is exactly why, despite the challenges we face, we just took a big gulp and joined forces with the Center for Investigative Reporting, a team of ace journalists who create the amazing podcast and public radio show Reveal.

If you can part with even just a few bucks, please help us pick up the pace of donations. We simply can’t afford to keep falling behind on our fundraising targets month after month.

Editor-in-Chief Clara Jeffery said it well to our team recently, and that team 100 percent includes readers like you who make it all possible: “This is a year to prove that we can pull off this merger, grow our audiences and impact, attract more funding and keep growing. More broadly, it’s a year when the very future of both journalism and democracy is on the line. We have to go for every important story, every reader/listener/viewer, and leave it all on the field. I’m very proud of all the hard work that’s gotten us to this moment, and confident that we can meet it.”

Let’s do this. If you can right now, please support Mother Jones and investigative journalism with an urgently needed donation today.

payment methods

AN IMPORTANT UPDATE

We’re falling behind our online fundraising goals and we can’t sustain coming up short on donations month after month. Perhaps you’ve heard? It is impossibly hard in the news business right now, with layoffs intensifying and fancy new startups and funding going kaput.

The crisis facing journalism and democracy isn’t going away anytime soon. And neither is Mother Jones, our readers, or our unique way of doing in-depth reporting that exists to bring about change.

Which is exactly why, despite the challenges we face, we just took a big gulp and joined forces with the Center for Investigative Reporting, a team of ace journalists who create the amazing podcast and public radio show Reveal.

If you can part with even just a few bucks, please help us pick up the pace of donations. We simply can’t afford to keep falling behind on our fundraising targets month after month.

Editor-in-Chief Clara Jeffery said it well to our team recently, and that team 100 percent includes readers like you who make it all possible: “This is a year to prove that we can pull off this merger, grow our audiences and impact, attract more funding and keep growing. More broadly, it’s a year when the very future of both journalism and democracy is on the line. We have to go for every important story, every reader/listener/viewer, and leave it all on the field. I’m very proud of all the hard work that’s gotten us to this moment, and confident that we can meet it.”

Let’s do this. If you can right now, please support Mother Jones and investigative journalism with an urgently needed donation today.

payment methods

We Recommend

Latest

Sign up for our free newsletter

Subscribe to the Mother Jones Daily to have our top stories delivered directly to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.

Subscribe

Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.

Donate