The open data movement is driven by an ethos to bring access to information into the 21st century by ensuring the government proactively releases information in a variety of useful formats, including those computers can easily read and categorize.
One of the central tenets of the open data movement is the need for governments to properly index their information holdings and make those indexes available to the public. Doing so allows both governments and the public to understand the full breadth of government data, enabling better data management and more informed discussions about how the government prioritizes releasing data.
But thanks to advances in the federal government’s open data practices and a recent FOIA victory by the Sunlight Foundation, we’re closer to understanding more about the government’s data holdings. And we’re closer to using FOIA more effectively for accountability.
One of the keys to effective use of the Freedom of Information Act, which enables the journalists, activists, researchers and others to access to information about government operations, procedures and activities, is to effectively target requests for information. But without comprehensive knowledge of the scope of information held by the government, it has been difficult carefully define the information sought.
Since taking office, President Barack Obama has made open data a priority. In 2013, the president issued his open data executive order which, among other things, required agencies to build indexes of their data holdings. In addition to naming datasets, these documents include valuable information about the data, ranging from the format, to a relevant contact person, to the reasons why certain datasets are withheld from public view. This was a positive step, allowing non-governmental stakeholders to see what data an agency both knew it held and was willing to release to the public.
Unfortunately, the executive order and subsequent guidance from the Office of Management and Budget required only a fraction of these indexes, and the information contained within them to be released to the public. Agencies were still permitted to withhold reams of information simply because they didn’t want to release it.
The Sunlight Foundation was particularly interested in seeing explanations and other information about the datasets that the government wouldn’t release to the public. With that in mind, we filed a FOIA request with OMB for full versions of these indexes.
After more than a year of back and forth, we are happy to say that the government recently began to comply with our request, opening up agency data inventories with only some redactions allowed under FOIA. With access to this information we now know that the Department of Transportation has proactively moved to release versions of some important datasets about vehicle safety. We now know that the Department of Labor has redacted quite a bit of information including some fields—like relevant contact point—that raises flags.
And thanks to Sunlight’s FOIA request, now agencies can only withhold information that falls under a small number of well-defined exemptions. Because of that, we now know that the Department of Defense hasn’t acknowledged holding any data that they won’t make public, which strongly indicates they either aren’t indexing such data at all or are intentionally hiding it.
This is only a small sample of the potentially invaluable information contained in the newly released data inventories. It is our hope that journalists, watchdogs, researchers and more will dig into the data, unearthing more stories.
Without FOIA, these data indexes may never have been made public, and this effort proved that agencies should continue to proactively release them in the future.
Matt Rumsey is the director of the Advisory Committee on Transparency at the Sunlight Foundation, a nonpartisan nonprofit that advocates for open government globally and uses technology to make government more accountable to all.