apologies for any typos, I’d like to look this over more, but I’ve got to get to other work.

Tom Slee has a very well written blog post with a critical perspective of open data. I encourage you to go and read it – but also to dive into the comments below it, which I think reflect one of the finer discussions I’ve seen below a blog post and articulate many of the critiques I would have had about Tom’s post, but in ways that are more articulate than I would have written (and frankly, how many times have you heard someone say that about comments).

I start with all this because a)  I think Tom and the team at Crooked Timber should be congratulated for fostering a fantastic online environment and b) before I dive in and disagree with Tom (which risks being confused with not liking or respecting him, which is definitely not the case – I don’t know him so can’t speak to his character but I sense nothing but honest and good intentions in his post and I definitely have a lot of respect for his mind). What I particularly respect is the calm and good natured way he responds to comments – especially when he concedes that a line of reasoning was flawed.

This is particularly refreshing given Tom’s original piece on this subject – the open data movement was a joke – of which this piece is a much more refined version – was fairly harsh and dismissive in its tone. That early piece lays bare some of the underlying assumptions that are also embedded in his newer piece. Tom has a very clear political perspective. He believes in a world where big companies are bad and smaller organizations are good even if that comes at the expense of a more effective or efficient service. Indeed, as one commenter notes (and I agree), Tom seems uncomfortable with the profit motive altogether. I don’t judge Tom’s for his perspective but they aren’t purely about open data – they are about a broader political agenda which open data may, or may not, end up serving. And to his credit, I think Tom is pretty clear about this. Open Data is not a problem in of itself to him, it is only a problem if it can be used to support an agenda that he believes trumps all others – enhancing freer and more open markets and/or putting the state’s role in certain activities at risk.

There is, I would like to note, however no discussion about the cost of closed data – or the powerful interests that mirror the open data free market doppelgänger’s – that like to keep it that way. No acknowledgement for the enormous inequalities embedded in the status quo where government data is controlled by government agents – and more often than not – sold to those who can pay for it. I have no doubt that open data will create new winners and losers – but let’s not pretend like the status quo doesn’t support many winners and create big losers either. Our starting point is not neutral. This sentiment came out with a rather unfortunantely terse statement from one commenter that, if one strips the emotion out of it, does raise a good point:

Um, so you don’t like open data, huh? At the very best, your analysis is radically incomplete for it doesn’t include any of the costs imposed by closed data. Anyone who has had to interact with the land record bureaucracy in places like India will tell you about the costs in time, bribes, lost work hours it takes to navigate a closed data environment. Also, there are plenty of stories in the Indian media of how farmers won disputes with authorities using GIS data and exposed land-grabs (because the data is open).

I remember Aneesh Chopra sharing a similar story (sorry I could find a better link). What I find frustrating is that open data advocates get accused of being techno-utopians, praising technology when things work and blaming officials when it goes wrong… but Slee seems to be doing the same in reverse. When people use bribery to hijack a GIS process to expropriate land, I blame corruption, not open data. And when the same GIS system allows a different set of poor farmers to securitize their land, get loans and stop using loan sharks – I praise the officials who implemented it and the legal system for being effective (and important per-requisite for open-data). Slee claims techno utopians fetishize open data, that’s true, some of them do (and I have been occasionally guilty). But he fetishizes the evils of private interests (and by extension, open data).

The more subtle and interesting approach Slee takes it to equate open data – and in particular standards – to culture creative products.

To maintain cultural diversity in the face of winner-take-all markets, governments in smaller countries have designed a toolbox of interventions. The contents include production subsidies, broadcast quotas, spending rules, national ownership, and competition policy. In general, such measures have received support from those with a left-leaning outlook.²⁰

The reason to do this is that it allows Slee to link open data to a topic that I sense he really dislikes – globalization – and explicitly connect Open Data to industrial policy which should, from what I can gather he believes, protect local developers from companies like Google. This leads to a fairly interesting ideas like “if Google is not going to pay or negotiate with all those transit agencies (#40) then that’s fine by me: perhaps that will let some of those apps being developed by hobbyists at hackathons gain some usage within their own transit area.” It is worth noting that a hobbyist doesn’t, by definition, make a sustaining wage from their work, defeating the cultural products/local economy link. I’m personally aware of this as I’ve had a number of hobby open data projects.

But what’s worse is the scenario he describes is unlikely to emerge. I’m not sure citizens, or their transit authorities, will want to rely on a hobbyist for a service I know I (and many others) see as critical. More likely is that one or two companies will become the app maker of choice for most transit authorities (so Google doesn’t win, just some other new big company) or many transit authorities will build something in house that is, frankly, of highly varied quality since this is not their core competency. This world isn’t hard to imagine since it was the world we lived in before Google maps starting sharing transit results.

Moreover, Slee’s vision will probably not reward local firms or punish large companies – it just  rewards different ones. Indeed, it is part of their strategy. Apple just announced that it won’t be support the General Transit Feed in its news iOS6 map and will instead push users to an app. This should be great news for hobbyist developers. The likely outcome however – as Clay Johnson notes – is that transit authorities have less incentive to do open data since the can force users to use “their” app. Hard to see how local hobbyist benefit. In any of the scenarios outlined above it is harder still to see how citizens benefit. Again, our transit future could look a lot like 2005. Indeed, one need only look at a city like Vienna to see the future. So yes, Google hurts, but to whose benefit? And it is worth noting that with smart phone penetration in the Western world increasing – and higher still among Blacks and Hispanics than whites and rising fastest among those making less than 30K – it is not clear to me that is the wealthy and privileged who are paying the price as they drive home.

But from my reading of Slee’s article this is still a good outcome since local creators and always better than those who live far away. Indeed, he considers the price small, calling it mere loss consumer efficiency and not a loss of civic improvement. I couldn’t disagree more. If figuring out how to catch the bus becomes harder it has a material impact on my sense of the civic infrastructure, and definitely my capacity to utilize it. Indeed, when it comes to real time data like that in GTFS version two, research tells us people it actually attracts more people to use the bus.

But what’s frustrating is that the choice Slee presents is false. Yes, open data can help foster bigger players. Google is a giant. But doing open data (and more importantly, standardizing it) has also enabled local and vibrant competitors to emerge much more easily. Some are hobbyists, others are companies. Indeed the total range of innovation that is possible because of the standard is amazing. I would strongly encourage you to read David Turner’s rebuttles (such as this one) in the comments are a strong rebuttal (especially around standardization) and don’t receive the attention I think they warrant.

Again, I strongly encourage you to go read Slee’s piece. It is filled with important insights that open data advocates need to digest. His opening story around the Map Kibera project and the Dalit’s claim is an important case studies in how not to engage in public policy or run a technology project in the developing world. The caveat however, is that these lessons would be true whether the data in those projects was going to be open or closed. His broader point, that there are corporate, free-market players seeking to exploit the open data movement should be understood and grappled with as well. However, I’d invite him to consider the reverse too. There are powerful corporate interests that benefit from closed data. He has his doppelgängers too.

I also want to note that I’m comfortable with many of Slee’s critique’s of open data because I both share and don’t share his political goals. I definitely want to live in a world where we strive for equality of opportunity and where monopolies are hard to form or stay entrenched. I’m also concerned about the internet’s capacity to foster big winners that could become uncontrollable. I’m less interested however, in a world that seeks to support middle men who extract value while contributing very little. So I’m looking for places where open data can help serve both these goals. That many mean that there is data than never gets made open – and I’m comfortable with that. But more importantly  my sense is that – in many circumstances – the right way to deal with these problems is to keep the barriers to entry for new entrants as low as possible.