Clive Thompson tells me in the April 2011 edition of Wired that government should open its data catalogs in order to foster private-sector investment, businesses, creativity. He tells me a story where BrightScope builds a multi-million dollar business by getting the Department of Labor to “[cough] up the digital goods in bulk.” He then goes on to suggest that if private-sector companies lead the way, perhaps activists will get the goods, too.
I’d like to meet Clive Thompson. I’d like to tell him about the technical assistance work I was doing in Chicago in 1987 for local community organizations in Chicago with digital data on occupations, released on magnetic tape. What happened when the Illinois Department of Employment Security realized that we charged a small fee to these organizations to run and help interpret the data? They yanked the digital version—for everyone—and only disseminated hard copies.
So, I was there first, Clive. And I don’t say this only because I’m feeling left out. I’m saying this because I’m at the Association of Public Data Users (“PublicData”) conference at GWU in Washington DC, bemoaning the drop in funding for the federal statistical agencies, while the O’Reilly Strata Conference is happening in New York celebrating the era of Big Data, including the energetic Open Government data movement (“OpenGov”). And while it would seem that OpenGov might have the same agenda as PublicData, it doesn’t. And that makes me crabby.
See, the difference is that OpenGov isn’t necessarily interested in statistical data. Statistical data, the raison d’être of PublicData, are gathered in order to understand characteristics of the US population and economy (think the US Census), and mostly by federal statistical agencies (think Census Bureau; Bureau of Labor Statistics). Statistical data have been used for decades for drawing electoral districts, setting public policy and programming, disbursing federal funding, and planning infrastructure investments like highways. (Do I see an OpenGov Yawn?)
OpenGov primarily wants administrative data and operational data. Administrative data are data gathered as a result of governments administering programs or overseeing regulations – the 401(k) data used by BrightScope noted above; EPA data generated as a result of environmental regulations, etc. Operational data are records generated as a result of government going about its own business – the visitor’s log to the White House at the federal level; 311 calls at the local level. It’s not statistical data — data that surveyors and researchers collect through observation and experimentation.
So while the Census’ Bureau’s budget is set to be slashed by an amount that means the end of the quinquennial Economic Census, (http://www.entrepreneurship.org/en/Blogs/Policy-Forum-Blog/2011/September/The-Quality-of-Economic-Statistics-is-About-to-Erode.aspx), Clive Thompson is telling me that “members of the Obama administration intervened” so that BrightScope could have (government) administrative data. So while I won’t be able to use statistical data to help local workforce agencies better target job training more effectively by understanding growing and declining economic sectors, private sector businesses can be built on administrative and operational data.
I know it isn’t this stark. I know from my non-profit sector vantage point the loss of statistical data will shift me to other activities perhaps. I’ll be able to use 311 data to help target food pantry resources, for example, while a private-sector food delivery service might also use the same data to beef up deliveries to seniors. And I’m all for increased private sector economic activity.
But what is true is that I (and others) have been working with nonprofit organizations for decades to wedge out better data from governments, and the federal statistical system is currently under siege; yet the OpenGov movement seem to be flourishing with evident private sector support (e.g. Google and Yahoo’s sponsorship of the Open Government Working Group Meeting in Sebastopol, CA in 2007which resulted in the “8 Principles of Open Government Data”). And that makes me surly.
So yes to Clive. I’ll wait for “pushy start ups [to] pressure governments to release more info [so that] activists will get to use it too.” But in return, I ask that pushy starts ups understand that there are those who have gone before, and who are standing in line.