Tag Archives: open data

The Three Laws of Open Government Data

Yesterday, at the Right To Know Week panel discussion – Conference for Parliamentarians: Transparency in the Digital Era – organized by the Office of the Information Commissioner I shared three laws for Open Government Data that I’d devised on the flight from Vancouver.

The Three Laws of Open Government Data:

If it can’t be spidered or indexed, it doesn’t exist

If it isn’t available in open and machine readable format, it can’t engage

If a legal framework doesn’t allow it to be repurposed, it doesn’t empower

To explain, (1) basically means: Can I find it? If Google (and/or other search engines) can’t find it, it essentially doesn’t exist for most citizens. So you’d better ensure that you are optimized to be crawled by all sorts of search engine spiders.

After I’ve found it, (2) notes that, to be useful, I need to be able to play with the data. Consequently, I need to be able to pull or download it in a useful format (e.g. an API, subscription feed, or a documented file). Citizens need data in a form that lets them mash it up with Google Maps or other data sets, or analyze in Excel. This is essentially the difference between VanMaps (look, but don’t play) and the Vancouver Data Portal, (look, take and play!). Citizens who can’t play with information are citizens who are disengaged/marginalized from the discussion.

Finally, even if I can find it and play with it, (3) highlights that I need a legal framework that allows me to share what I’ve created, to mobilize other citizens, provide a new service or just point out an interesting fact. This is the difference between Canada’s House of Parliament’s information (which, due to crown copyright, you can take, play with, but don’t you dare share or re-publish) and say, Whitehouse.gov which “pursuant to federal law, government-produced materials appearing on this site are not copyright protected.”

Find, Play and Share. That’s want we want.

Of course, a brief scan of the internet has revealed that others have also been thinking about this as well. There is this excellent 8 Principle of Open Government Data that are more detailed, and admittedly better, especially for a CIO level and lower conversation. But for talking to politicians (or Deputy Ministers or CEOs), like those in attendance during yesterday’s panel or, later that afternoon, the Speaker of the House, I found the simplicity of three resonated more strongly; it is a simpler list they can remember and demand.

Today: "right to know" panel for parliamentarians

4 Replies

Today from 10am-12am EST I’ll be a panelist for Conference for Parliamentarians: Transparency in the Digital Era a panel convened by the Office of the Information Commissioner as part of Right to Know Week. Apparently the Canadian School of Public Service will provide access to this conference as part of its Armchair Discussions (www.righttoknow.ca).

Misunderstanding and understanding the Open Data Hype

7 Replies

On Wednesday Gartner’s Andrea Dimaio wrote an interesting blog post entitled Open Data and Application Contests: Government 2.0 at the Peak of Inflated Expectations which Peter Smith nicely linked to the Gartner’s Hype Cycle graph from Wikipedia. I want to break his post down into three components. Two – the bad and the good, I’m going to talk about today – the third, which I’ll tackle on Monday involves some mapping and fun.

The Bad

As someone whose been thinking about and working on Open Data and Gov 2.0 for several years now three things struck me as problematic about Andrea’s post. Firstly, he misunderstands the point of open data. While many people – self-included- talk about how it can empower citizens, citizens will not be its primary beneficiary. The biggest user of open data portals is going to be government employees. Indeed, Tim Wilson reminded me the other day of our conversation with Jason Birch, the thought leader who made much of Nanaimo’s geo-data public, where he talked about how he wasn’t actually tasked with sharing data publicly – he was tasked with making the data available to other Nanaimo city employees. Sharing it with citizens was a (relatively) cost free addition. These projects aren’t about serving some techo-literati, it is about getting a city to first and foremost talk to itself – having it talk to its citizens is an important (and democracy expanding) benefit.

Second, was this unfortunate anecdote:

Yesterday I was discussing with a British client over lunch and he told me how the publication of data may lead to requests for more data (through the Freedom of Information Act), in a never-ending cycle of information gathering which is likely to cost a lot to both government and taxpayers. Another client observed (as I said in a previous post) that there is no way people will be able to tell to what extent a mash up on an application actually uses official, trusted government data.

Could government become swamped with data requests? Who knows, but in theory… it shouldn’t. Making data available should reduce the amount of time public servants spend responding to requests by diverting requests to open data portals. But let’s say Andrea’s concerns are valid and that, as a result of open data, citizens become more actively concerned and interested in how government works and thus Freedom of Information Act requests increase. The horror… citizens are interested in government! Citizens want to know how decisions are made! Remind me again… why is this a problem?

The real problem here isn’t access to data, it’s that the Freedom of Information Act process is itself broken. If open data creates a further demand for more transparent government and pushes us to foster better mechanisms for sharing government information, this is a good consequence. As for concerns that people might misrepresent public data, well a) people can already do this and we haven’t had a rash of bad applications, but even if they tried… people will stop using their service pretty quick.

Finally, another nice thing about public data is that it tends to get very clean, very quickly. My concern isn’t that government data will be misrepresented… I’m concerned that government data is already wrong and isn’t being verified. Knowing that someone might actually look at a data set is one of the most powerful incentives for organization to improve its collection. (Something Clay Shirky noted in a talk he made the other day at a Bioinformatics conference I’m at).

(There is of course, one group who may not see these a good consequences as it will change how they work: British public servant like Andrea’s client’s who raised the objections… but then they pay Gartner’s bills, not you.)

The Good

The end of Andrea Dimiao’s piece is where we find common ground. I agree that the Apps for Democracy competitions run the risk of limiting the definition of “the public” to citizen coders. We want broader participation – particularly once more complex data sets like budgets, procurement and crime data are released – from academics, citizens groups and NGOs. Here in Vancouver we’ve talked about focusing any Apps competition on the themes of homelessness, housing and the environment, since these have been the dominant concerns of citizens in recent years.

More importantly, I agree (and love) Dimiao’s concept of employee-centric government. Indeed, my chapter for Tim O’Reilly’s upcoming book on Open Government makes a parallel argument, that namely we should stop trying to teach an analogue government to talk to a digital public and instead focus on making government digital (ie. getting it “open,” networked and using web 2.0 internally) first.

And perhaps most importantly, I agree that government 2.0 risks being over-hyped. I still believe in the potential, but know that getting there is going to be a painful process (mind the gap!). Government 2.0 advocates should expect lots of resistance and adoption problems ahead – but then change is painful.

More ways to make open data sexy: 5 Municipal Apps I'd love to see (what are yours?)

13 Replies

One of the big goals of the open data project is to get many citizens interested in different ways the data can be used. Many citizens lack the skills to code up an application and creating a website is intimidating, but they may have ideas that could improve the city or be useful to many citizens.

In the hopes of spurring more interest in the open data and getting those not tradition involved, well… involved, I’ve created an “Ideas for the Taking” page on the Vancouver Open Data wiki. I’ve seeded the page with some of the ideas I promised I would share at the Open Data Hackathon last week . Some use open data, others don’t. Mostly however, I hop they spurn others to think of what is possible and what interests them. (PS. If you are a reader and the wiki is too confusing, just email me your idea and I’ll add it to the wiki with (or without, if you prefer) you’re name attached.

So here are some ideas I’ve brainstormed:

1. Stolen Bike Tracker

Vancouver’s cycling community is huge, sadly however, the city is plagued by a serious problem: stolen bicycles. There is no solution to this problem but I think a well crafted app could help minimize the nuisance. I can imagine an app or website in which you take a photo of your bike and upload it along with some identifying information(like the serial number) to a website. The picture stays hidden, however, if your bike gets tragically stolen you load up the apps and press the “my bike was stolen button.” This marks the physical place where your bike was stolen and activates your bike photo and marks it as stolen. Now cyclists, bike shop owners and the police can check bikes to see if they are stolen before buying them (or return them to their owner if they are recovered). In addition, a street map of bike theft would also be created. This could be particularly relevant since I suspect a great deal of bike theft is not reported. Finally, for those worried about privacy, I could imagine the app using a Craigslist style contact system that would preserve the anonymity of the original owner.

2. A Downtown East Side Landlord wiki

There are a few data sets that might allow for someone to create a geo-wiki of the DTES. I think it would be interesting to have a wiki that – on a building by building level – outlined who owned which residential buildings, what they charged in rent, a list of the room amenities and comments about the property’s management. It might also be interesting to enable photos to be posted so people can show the living conditions. Such a wiki might give the public (and prospective renters) a window into the deplorable conditions and poor practices of the worst offenders. It might also help City Staff deploy resources for investigating code violations and other questionable practices.

3. Everyblock+

Obviously, I think an Everyblock app for Vancouver would be great. The one new layer I’d love to see added to it is a charity button. With this button you would see what charities are operating on the block/area you are standing on. This is harder to imagine realizing, but cooler still would be a button that would allow you to then donate to that charity.

4. Burrard Bridge Trial Website

While not located on the Open Data Portal, the city has been releasing weekly data sets on vehicle, pedestrian and cycle trip across the Burrard Bridge Trial on the Burrard Trial blog. The data is not particularly well organized (you’d have to scrape it and its only granular to the 24hr time block – so no hour by hour data sets) but it is a start. I’d be fascinating to have a site that does a deeper analysis of the data and maybe shows it in a more interesting format. Maybe a discussion on carbon emissions reduced… still more interesting would be an analysis of bicycle accidents at present versus before the trial (data that is, sadly, not obviously available).

5. City Services vs. Land Value Mashup

It would be interesting to see what impact city services have on land values. I’m not sure if land value data is available (anyone know?) but mashing it up against the location of parks, community centres, schools, firehalls, and other city amenities would be interesting. While potentially interesting to prospective home owners (maybe a real estate agency should develop – or pay to develop – this app) I think it might also be of interest to the electorate and politicians.

One last one: A Library-Amazon Greasemonkey script

A Library-Amazon Greasemonkey search script allows a user to see if a book being displayed on an Amazon.ca website is available in the Vancouver Public Library. This has two benefits. First, it is WAY easier to find books on the Amazon site then the library site, so you can leverage Amazon’s search engine to find books (or book recommendations) at the VPL. Second, it’s a great way to keep the book budget in check!

The Vancouver Public Library has said that it will share access to its database that would allow such an app to work. I believe I have the email address for the relevant person somewhere on my computer who can make this happen. (I can get the contact info for the right person if someone nudges me.) Better still the necessary Greasemonkey script is already available (scripts exist for Palo Alto, Seattle and Ottawa), it would be great if someone tweaked the script so it worked with the VPL.

Of course, I’m hoping that others are already hatching plans about how they’d like to use the city’s data to create something they feel passionate about. And remember, if there is an app you’d like to create but the data set isn’t available – take the Open Data survey to let your voice be heard! If any of these ideas interest you, go for it. If I can help in any way, let me know, I’m keen to contribute.

Garbage Collection now IS sexy: Introducing VanTrash

A few months ago some of you will remember I blogged about How Open Data even makes Garbage collection sexier, easier and cheaper. I suggested that, with open data, coders could digitize the city’s garbage collection schedule and city maps and enable citizens to download it into their calendar or even set up a recurring email reminder.

The post went fairly viral being picked up places like here and here. As a result, two weeks later Luke Closs and Kevin Jones, two Vancouver based coders with a strong sense of fun and civic duty emailed me and said they’d actually scrapped the data and had created an alpha version of the site. I offered my (meagre) skills to help move the application forward and we began working on it.

Today, I’m pleased to say that VanTrash has been launched. If you live in Vancouver (or don’t) please do take a look at the website.

Our goal with VanTrash is twofold. First: we want a great service that leverages public data to helpmake our fellow citizens’ lives a little better or easier. Second: we’d like to sign up 3000 or more users.

Since there are about 260,000 households in Vancouver (although many have private contracted garbage pick up) 3000 users would represent between 1-2% of all households for whom the city collects their garbage. There are not that many services that citizens opt in for that get this market penetration – especially services created for virtually nothing. The more users we get, the stronger the message we send to government’s everywhere that government is a platform and that we need to let citizens built on top of it. More importantly, we demonstrate that great, and useful, things can be done for cheap, a lesson citizens and governments need to constantly relearn.

So if you live in Vancouver, and you think there service would be helpful to you (or perhaps to a forgetful or absent minded friend, family member or neighbour) please sign up or spread the word.

Vancouver's Open Data Portal: Use it, or Lose it.

As some of you saw yesterday via Twitter, Vancouver has launched a beta version of its open data portal. This is a major milestone for Vancouver on several levels. It is a testament to our politicians, who had the vision and foresight to embrace this idea. It is a tribute to the city’s staff who have worked unbelievably hard to make this project come alive so quickly. It is a triumph for those of us who advocate and have been working with the city to move us towards open government and government as platform. Finally, it represents an enormous opportunity for coders and citizens alike, and it is to this group that I’d like to address this blog post.

The Data Portal represents an opportunity for citizens, especially citizen coders, to help create a City that Thinks Like the Web: a city that enables citizens to create and access collective knowledge and information to create new services, suggest new ideas, and identify critical bugs in the infrastructure and services, among other a million other possibilities.

But the open data is only the part of the puzzle. Yes, our data is now beginning to be set free. But we have to use it.

If not, we’ll risk losing it.

I wish I could say that the city will share data no matter what and that political support will continue forever. But the fact is, municipal resources are limited. While the potential of open data is enormous, we need more than potential; we need some wins. More importantly, we need an active and engaged community working to make Vancouver better, more efficient, and more interesting because of our open data. We need to show politicians and public servants in Vancouver, but also in Edmonton, Calgary, Ottawa, Toronto, Montreal, Nanaimo, Moncton and other places across the country that citizens want access to data, and that if we get it, we will help their city (or province, or country) come alive in new and inventive ways.

Back in June, shortly after City Council passed Open3 (the nickname for the Open Motion), I gave this presentation to both City staff and at Open Web Vancouver. In it I described how “the bargain” Clay Shirky says exists on every successful web 2.0 site also exists in cities that want to think like the web.

In our case the bargain is simple: On one side, the city agrees to share as much data as it possibly can, in open formats, as quickly as it can. On the other side, the community – and in particular citizen coders – must make that data come alive in applications, websites and analysis. The city has taken the first step in fulfilling its side of the bargain. (And yes, we need to keep adding more data; there is work to be done.) Now it is time to activate the other half of the bargain. If we don’t, we put the deal at risk.

So what can you do?

First, you can code up an app, or find ways to help those who are. Indeed, there is going to be a Hackathon tomorrow evening at the Vancouver Archives to do just this. A number of projects are already underway that you can join – or start one yourself! I will be there myself, and I encourage you to swing by too.

Second, if you’d like to build an application, but the dataset you need is currently not available, then complete the city’s Open Data priority survey!

Third, come add ideas, resources and projects to the Vancouver Open Data Wiki.

I’m enormously excited to see what evolves next. As many of you know, I’ve been advising the Mayor’s Office on open data and open government for several months now – and through my work with them and with city staff, I’ve been deeply impressed by the energy and commitment that I’ve seen. As far as I know, only three major cities have created data portals such as this, and to do this in three months is incredible. Over the next few days I’m going to share some more thoughts on what the Open Data portal means for Vancouver. If you get a chance I hope you’ll send me your thoughts as well, or post some to your own blog if you have one.

How to Engage Citizens on a Municipal Website…

Sometimes, it’s nice to be small, the City of Nanaimo has been pushing the envelop on open data and open government for a number of years now.

Recently, I was directed to their new Council Agendas and Minutes webpage. I recommend you check it out.

Here’s why.

At first blush the site seems normal. There is the standard video of the council meeting (queue cheesy local cable access public service announcement), but them meeting minutes underneath are actually broken down by the second and by clicking on them you can jump straight to that moment in the meeting.

As anyone who’s ever attended a City Council meeting (or the legislature, or parliament) knows, the 80/20 rule is basically always in effect. About 80% of the time the proceedings are either dead boring and about 20% (often much less) of the time the proceedings are exciting, or more importantly, pertinent to you. One challenge with getting citizens engaged on the local level is that they often encounter a noise to signal problem. The ratio of “noise” (issues a given citizen doesn’t care about) drowns out the “signal” (the relatively fewer issues they do care about).

The City of Nanaimo’s website helps address this problem. It enables citizens to find what matters to them without having to watch or scroll through a long and dry council meeting. Better still, they are given a number of options by which to share that relevant moment with friends, neighbours, allies or colleagues via twitter, facebook, delicious or any other number of social media tools.

One might be wondering: can my city afford such a wizbang setup?

Excellent question.

Given Nanaimo’s modest size (it has 78,692 citizens) suggests they have a modest IT budget. So I asked Chris McLuckie, a City of Nanaimo public servant who worked on the project. He informed me that the system was built in-house by him and another city staff member, it uses off-the-shelf hardware and software and so cost under $2000 and it took 2 week to code up.

2 weeks?

No million dollar contract? No 8 month timeline? No expensive new software?

No. Instead, if you’re smart, you might find a couple of local creative citizen-hackers to put something together in no time at all.

You know what’s more, because Chris and the City of Nanaimo want to help more cities learn how to think like the web, I bet if the IT director from any city (or legislative body) asked nicely, they would just give them the code.

So how Open is your city? And if not, do they have $2000 lying around to change that?

Opendata & Opencities: Proposed panel for SXSWi

Over the past year I’ve been inspired by the fact that an increasing number of cities are thinking about how to more effectively share the data they generate with their citizens.

As most readers of this blog are probably aware, I’ve been engrossed advising the Mayor’s Office here in Vancouver on the subject and am excited about the progress being made on the City’s open data project.

Since there is so much energy around this topic across North America I thought there might be interest among SXSWers on the opportunities, challenges and benefits surrounding open data.

Here’s my proposed panel, and if you think it is a good idea I’d be elated if you took the time to head over to the panel picker website and voted for it!

Title:

OpenData: Creating Cities That Think Like the Web

Level:

Beginner

Category:

Community / Online Community, Government and Technology, Social Issues, User Generated Content, Web Apps / Widgets

Questions:

What is open data?

How can I effectively mobilize people to get my local government to share data?

How can open data be shared most effectively?

What are the benefits of open data?

What business models are emerging around municipal open data?

How can citizens/citizen coders help government bureaucracies share open data?

How do government bureaucracies centered on secrecy and security shift to being interested in open?

How is open data changing the role of government?

How is open data changing the relationship between citizens and government?

Description:

Across North America municipal governments are opening up their data and encouraging citizens to create online applications, mash-ups and tools to improve city services and foster engagement. Panelists from cities leading this open movement will discuss the challenges, lessons, benefits and opportunities of open data and open government.

Some of the people I’d love to have as panelists include:

Kelly Pretzer (@kellypretzer) Is a City of SF employee who has been working with a team on an open data initiative with the city of SF. You can track their work here.

Peter Corbett (@corbett3000) is CEO of iStrategyLabs. iStrategy Labs is the organization that ran the Apps for Democracy competition in Washington DC. If Peter can’t make it, we’d hope iStrategy could send a representative.

Ryan Merkley (@ryanmerkley) Political advisor to the Mayor of Toronto and helping oversee the open Toronto Initiative.

Myself! (@david_a_eaves) I’ve been advising the Mayor of Vancouver on open government and open data and co-drafted the Open Motion, passed by the City of Vancouver on May 21st.

It would, of course, be nice to have Vivek Kundra, but I’ll confess, I’m not sure I have that kind of pull…

Creating the Open Data Bargain in Cities

Embedded below is the talk I’ve given to both community hackers (at Open Web Vancouver) as well as City of Vancouver Staff regarding the opportunities and challenges around open data and the open motion. (Here’s an update on where Vancouver is at courtesy of some amazing work by city staff).

For those willing to brave through the presentation (or simply fast forward to the end) one piece I felt is most important is the talk’s last section which outlines what I term “The Bargain” in a reference to the informal contract Clay Shirky says exists between every Web 2.0 site and their users.

The bargain comes last, because it matters only if there is a promise (open and shared data) and a set of tools (applications languages) that are already working together. The bargain is also the most complex aspect of a functioning group, in part because it is the least explicit aspect and in part because it is the one the users have the biggest hand in creating, which means it can’t be completely determined in advance… A bargain helps clarify what you can expect of others and what they can expect of you.

– Clay Shirky in Here Comes Everybody (my italics, page 270)

I believe that in an open city, a similar bargain exists between a government and its citizens. To make open data a success and to engage the community a city must listen, engage, ask for help, and of course, fulfil its promise to open data as quickly as possible. But this bargain runs both ways. The city must to its part, but so, on the flip side, must the local tech community. They must participate, be patient (cities move slower than tech companies), offer help and, most importantly, make the data come alive for each other, policy makers and citizens through applications and shared analysis.

Creating a City That Thinks like the Web (Vancouver Remix)

View more documents from David Eaves.

Open Cities – the Counter Reaction

The Washington Monthly has an interesting piece about how some bureaucracies are having a reactionary (but albeit unsurprising) reaction to open data initiatives. The article focuses on how the data used by one application, Stumble Safely “helps you find the best bars and a safe path to stumble home on” by mashing together DC Crime Data, DC Road Polygons, DC Liquor Licenses, DC Water, DC Parks, and DC Metro Stations.

However, arming citizens with precise knowledge doesn’t appear to make one group of people happy: The Washington, D.C. police department. As the article notes:

But a funny thing has happened since Eric Gundersen launched his application: Stumble Safely has become less useful, rather than more so. When you click on the gray and red crime-indicating dots that have appeared on the map in the past few months, you don’t get much information about what exactly happened—all you get is a terse, one-word description of the category of the incident (”assault,” or “theft”) and a time, with no details of whether it was a shootout or just a couple of kids punching each other in an alley.

This isn’t Gundersen’s fault—it’s the cops’. Because while Kundra and the open-data community were fans of opening up the city’s books, it turned out that the Metropolitan Police Department was not. Earlier this year, as apps like Stumble Safely grew in number and quality, the police stopped releasing the detailed incident reports—investigating officers’ write-ups of what happened—into the city’s data feed. The official reason for the change is concern over victims’ and suspects’ privacy. But considering that before the clampdown the reports were already being released with names and addresses redacted, it’s hard to believe that’s the real one. More likely, the idea of information traveling more or less unedited from cops’ keyboards to citizens’ computer screens made the brass skittish, and the department reacted the way bureaucracies usually do: it made public information harder to get. The imperatives of Government 2.0 were thwarted by the instincts of Government 1.0.

This is just one in a long list of ways that old-style government (1.0) is reacting against technology. The end result sadly however is that the action taken by the police doesn’t reduce crime, it just reduces the public’s confidence in the police force. This is just a small example of the next big debate that will take place at all levels of government: Will your government try to control information and services or will it develop trust by being both accountable and open to others building on its work? You can’t have it both ways and I suspect citizens – particularly creatives – are going to strongly prefer the latter.

This is a crosspost from my Open Cities Blog at CreativeClass.com

eaves.ca

if writing is a muscle, this is my gym