Category Archives: open data

Misunderstanding and understanding the Open Data Hype

On Wednesday Gartner’s Andrea Dimaio wrote an interesting blog post entitled Open Data and Application Contests: Government 2.0 at the Peak of Inflated Expectations which Peter Smith nicely linked to the Gartner’s Hype Cycle graph from Wikipedia. I want to break his post down into three components. Two – the bad and the good, I’m going to talk about today – the third, which I’ll tackle on Monday involves some mapping and fun.

The Bad

As someone whose been thinking about and working on Open Data and Gov 2.0 for several years now three things struck me as problematic about Andrea’s post. Firstly, he misunderstands the point of open data. While many people – self-included- talk about how it can empower citizens, citizens will not be its primary beneficiary. The biggest user of open data portals is going to be government employees. Indeed, Tim Wilson reminded me the other day of our conversation with Jason Birch, the thought leader who made much of Nanaimo’s geo-data public, where he talked about how he wasn’t actually tasked with sharing data publicly – he was tasked with making the data available to other Nanaimo city employees. Sharing it with citizens was a (relatively) cost free addition. These projects aren’t about serving some techo-literati, it is about getting a city to first and foremost talk to itself – having it talk to its citizens is an important (and democracy expanding) benefit.

Second, was this unfortunate anecdote:

Yesterday I was discussing with a British client over lunch and he told me how the publication of data may lead to requests for more data (through the Freedom of Information Act), in a never-ending cycle of information gathering which is likely to cost a lot to both government and taxpayers. Another client observed (as I said in a previous post) that there is no way people will be able to tell to what extent a mash up on an application actually uses official, trusted government data.

Could government become swamped with data requests? Who knows, but in theory… it shouldn’t. Making data available should reduce the amount of time public servants spend responding to requests by diverting requests to open data portals. But let’s say Andrea’s concerns are valid and that, as a result of open data, citizens become more actively concerned and interested in how government works and thus Freedom of Information Act requests increase. The horror… citizens are interested in government! Citizens want to know how decisions are made! Remind me again… why is this a problem?

The real problem here isn’t access to data, it’s that the Freedom of Information Act process is itself broken. If open data creates a further demand for more transparent government and pushes us to foster better mechanisms for sharing government information, this is a good consequence. As for concerns that people might misrepresent public data, well a) people can already do this and we haven’t had a rash of bad applications, but even if they tried… people will stop using their service pretty quick.

Finally, another nice thing about public data is that it tends to get very clean, very quickly. My concern isn’t that government data will be misrepresented… I’m concerned that government data is already wrong and isn’t being verified. Knowing that someone might actually look at a data set is one of the most powerful incentives for organization to improve its collection. (Something Clay Shirky noted in a talk he made the other day at a Bioinformatics conference I’m at).

(There is of course, one group who may not see these a good consequences as it will change how they work: British public servant like Andrea’s client’s who raised the objections… but then they pay Gartner’s bills, not you.)

The Good

The end of Andrea Dimiao’s piece is where we find common ground. I agree that the Apps for Democracy competitions run the risk of limiting the definition of “the public” to citizen coders.  We want broader participation – particularly once more complex data sets like budgets, procurement and crime data are released – from academics, citizens groups and NGOs. Here in Vancouver we’ve talked about focusing any Apps competition on the themes of homelessness, housing and the environment, since these have been the dominant concerns of citizens in recent years.

More importantly, I agree (and love) Dimiao’s concept of employee-centric government. Indeed, my chapter for Tim O’Reilly’s upcoming book on Open Government makes a parallel argument, that namely we should stop trying to teach an analogue government to talk to a digital public and instead focus on making government digital (ie. getting it “open,” networked and using web 2.0 internally) first.

And perhaps most importantly, I agree that government 2.0 risks being over-hyped. I still believe in the potential, but know that getting there is going to be a painful process (mind the gap!). Government 2.0 advocates should expect lots of resistance and adoption problems ahead – but then change is painful.

More ways to make open data sexy: 5 Municipal Apps I'd love to see (what are yours?)

One of the big goals of the open data project is to get many citizens interested in different ways the data can be used. Many citizens lack the skills to code up an application and creating a website is intimidating, but they may have ideas that could improve the city or be useful to many citizens.

In the hopes of spurring more interest in the open data and getting those not tradition involved, well… involved, I’ve created an “Ideas for the Taking” page on the Vancouver Open Data wiki. I’ve seeded the page with some of the ideas I promised I would share at the Open Data Hackathon last week . Some use open data, others don’t. Mostly however, I hop they spurn others to think of what is possible and what interests them. (PS. If you are a reader and the wiki is too confusing, just email me your idea and I’ll add it to the wiki with (or without, if you prefer) you’re name attached.

So here are some ideas I’ve brainstormed:

1. Stolen Bike Tracker

Vancouver’s cycling community is huge, sadly however, the city is plagued by a serious problem: stolen bicycles. There is no solution to this problem but I think a well crafted app could help minimize the nuisance. I can imagine an app or website in which you take a photo of your bike and upload it along with some identifying information(like the serial number) to a website. The picture stays hidden, however, if your bike gets tragically stolen you load up the apps and press the “my bike was stolen button.” This marks the physical place where your bike was stolen and activates your bike photo and marks it as stolen. Now cyclists, bike shop owners and the police can check bikes to see if they are stolen before buying them (or return them to their owner if they are recovered). In addition, a street map of bike theft would also be created. This could be particularly relevant since I suspect a great deal of bike theft is not reported. Finally, for those worried about privacy, I could imagine the app using a Craigslist style contact system that would preserve the anonymity of the original owner.

2. A Downtown East Side Landlord wiki

There are a few data sets that might allow for someone to create a geo-wiki of the DTES. I think it would be interesting to have a wiki that – on a building by building level – outlined who owned which residential buildings, what they charged in rent, a list of the room amenities and comments about the property’s management. It might also be interesting to enable photos to be posted so people can show the living conditions. Such a wiki might give the public (and prospective renters) a window into the deplorable conditions and poor practices of the worst offenders. It might also help City Staff deploy resources for investigating code violations and other questionable practices.

3. Everyblock+

Obviously, I think an Everyblock app for Vancouver would be great. The one new layer I’d love to see added to it is a charity button. With this button you would see what charities are operating on the block/area you are standing on. This is harder to imagine realizing, but cooler still would be a button that would allow you to then donate to that charity.

4. Burrard Bridge Trial Website

While not located on the Open Data Portal, the city has been releasing weekly data sets on vehicle, pedestrian and cycle trip across the Burrard Bridge Trial on the Burrard Trial blog. The data is not particularly well organized (you’d have to scrape it and its only granular to the 24hr time block – so no hour by hour data sets) but it is a start. I’d be fascinating to have a site that does a deeper analysis of the data and maybe shows it in a more interesting format. Maybe a discussion on carbon emissions reduced… still more interesting would be an analysis of bicycle accidents at present versus before the trial (data that is, sadly, not obviously available).

5. City Services vs. Land Value Mashup

It would be interesting to see what impact city services have on land values. I’m not sure if land value data is available (anyone know?) but mashing it up against the location of parks, community centres, schools, firehalls, and other city amenities would be interesting. While potentially interesting to prospective home owners (maybe a real estate agency should develop – or pay to develop – this app) I think it might also be of interest to the electorate and politicians.

One last one: A Library-Amazon Greasemonkey script

A Library-Amazon Greasemonkey search script allows a user to see if a book being displayed on an Amazon.ca website is available in the Vancouver Public Library. This has two benefits. First, it is WAY easier to find books on the Amazon site then the library site, so you can leverage Amazon’s search engine to find books (or book recommendations) at the VPL. Second, it’s a great way to keep the book budget in check!

The Vancouver Public Library has said that it will share access to its database that would allow such an app to work. I believe I have the email address for the relevant person somewhere on my computer who can make this happen. (I can get the contact info for the right person if someone nudges me.) Better still the necessary Greasemonkey script is already available (scripts exist for Palo  Alto, Seattle and Ottawa), it would be great if someone tweaked the script so it worked with the VPL.

Of course, I’m hoping that others are already hatching plans about how they’d like to use the city’s data to create something they feel passionate about. And remember, if there is an app you’d like to create but the data set isn’t available – take the Open Data survey to let your voice be heard! If any of these ideas interest you, go for it. If I can help in any way, let me know, I’m keen to contribute.