Monthly Archives: September 2009

The Three Laws of Open Government Data

86 Replies

Yesterday, at the Right To Know Week panel discussion – Conference for Parliamentarians: Transparency in the Digital Era – organized by the Office of the Information Commissioner I shared three laws for Open Government Data that I’d devised on the flight from Vancouver.

The Three Laws of Open Government Data:

  1. If it can’t be spidered or indexed, it doesn’t exist
  2. If it isn’t available in open and machine readable format, it can’t engage
  3. If a legal framework doesn’t allow it to be repurposed, it doesn’t empower

To explain, (1) basically means: Can I find it? If Google (and/or other search engines) can’t find it, it essentially doesn’t exist for most citizens. So you’d better ensure that you are optimized to be crawled by all sorts of search engine spiders.

After I’ve found it, (2) notes that, to be useful, I need to be able to play with the data. Consequently, I need to be able to pull or download it in a useful format (e.g. an API, subscription feed, or a documented file). Citizens need data in a form that lets them mash it up with Google Maps or other data sets, or analyze in Excel. This is essentially the difference between VanMaps (look, but don’t play) and the Vancouver Data Portal, (look, take and play!). Citizens who can’t play with information are citizens who are disengaged/marginalized from the discussion.

Finally, even if I can find it and play with it, (3) highlights that I need a legal framework that allows me to share what I’ve created, to mobilize other citizens, provide a new service or just point out an interesting fact. This is the difference between Canada’s House of Parliament’s information (which, due to crown copyright, you can take, play with, but don’t you dare share or re-publish) and say, Whitehouse.gov which “pursuant to federal law, government-produced materials appearing on this site are not copyright protected.”

Find, Play and Share. That’s want we want.

Of course, a brief scan of the internet has revealed that others have also been thinking about this as well. There is this excellent 8 Principle of Open Government Data that are more detailed, and admittedly better, especially for a CIO level and lower conversation.  But for talking to politicians (or Deputy Ministers or CEOs), like those in attendance during yesterday’s panel or, later that afternoon, the Speaker of the House, I found the simplicity of three resonated more strongly; it is a simpler list they can remember and demand.

Today: "right to know" panel for parliamentarians

4 Replies

Today from 10am-12am EST I’ll be a panelist for Conference for Parliamentarians: Transparency in the Digital Era a panel convened by the Office of the Information Commissioner as part of Right to Know Week. Apparently the Canadian School of Public Service will provide access to this conference as part of its Armchair Discussions (www.righttoknow.ca).

More on the panel:

This conference aims to engage Parliamentarians in a debate and reflection on the new paradigm that the digital world has introduced for the right to know. Greater transparency in the digital era requires more than sound information management and the use of state-of-the-art information technology. It calls for a fundamental change of attitudes from disclosing information on a need-to-know basis to managing information with the presumption of disclosure as the default mode. How can public institutions trigger and accelerate this change of attitudes for the benefit of Canadians?

For those who are interested you can see my slides (sans audio, I’m afraid) below.

Government Transparency In A Digital Age

View more presentations from David Eaves.

Mapping Government 2.0 against the Hype Curve

15 Replies

Last week Andrea DiMaio wrote an interesting post on how Government 2.0 may be approaching the peak of the hype cycle. I’d never seen the hype cycle before and it looked fun, so I thought it might be interesting to try to map where I believe some current Canadian government 2.0 projects, a few older technologies, and a few web 2.0 technologies in general, are against this chart from a government perspective.

My suspicion is that we haven’t even begun to dip into the Trough of Disillusionment with most true Web 2.0 government projects (GCPEDIA & GCConnex). However, I think governments have overcome their paranoia about facebook, but are still very wary… that said I’ve noticed some government ministries have started to use facebook as a communication tool with the public. Blogs however (which are perfectly okay for public servants to create internally) are still viewed with suspicion – internally they are almost never used. But then, heck, given the cluttered nature of most government website (and the fact that finding info is hard), I think it is clear that we are still working our way up the “Slope of Enlightenment” on this web 1.0 technology.

Gov 20 hype curve 3

Those who’ve seen me speak know I much I love Arthur Schopenhauer‘s three stages of truth:

“All truth passes through three stages. First, it is ridiculed. Second, it is violently opposed. Third, it is accepted as being self-evident.”

So I’ve also included how I think the three stages map against this chart too.

So in sum… in  my view there is good news for Government 2.0 projects – visibility is still increasing. Unfortunately there is also some bad news: prepare for increased violent opposition…

Misunderstanding and understanding the Open Data Hype

7 Replies

On Wednesday Gartner’s Andrea Dimaio wrote an interesting blog post entitled Open Data and Application Contests: Government 2.0 at the Peak of Inflated Expectations which Peter Smith nicely linked to the Gartner’s Hype Cycle graph from Wikipedia. I want to break his post down into three components. Two – the bad and the good, I’m going to talk about today – the third, which I’ll tackle on Monday involves some mapping and fun.

The Bad

As someone whose been thinking about and working on Open Data and Gov 2.0 for several years now three things struck me as problematic about Andrea’s post. Firstly, he misunderstands the point of open data. While many people – self-included- talk about how it can empower citizens, citizens will not be its primary beneficiary. The biggest user of open data portals is going to be government employees. Indeed, Tim Wilson reminded me the other day of our conversation with Jason Birch, the thought leader who made much of Nanaimo’s geo-data public, where he talked about how he wasn’t actually tasked with sharing data publicly – he was tasked with making the data available to other Nanaimo city employees. Sharing it with citizens was a (relatively) cost free addition. These projects aren’t about serving some techo-literati, it is about getting a city to first and foremost talk to itself – having it talk to its citizens is an important (and democracy expanding) benefit.

Second, was this unfortunate anecdote:

Yesterday I was discussing with a British client over lunch and he told me how the publication of data may lead to requests for more data (through the Freedom of Information Act), in a never-ending cycle of information gathering which is likely to cost a lot to both government and taxpayers. Another client observed (as I said in a previous post) that there is no way people will be able to tell to what extent a mash up on an application actually uses official, trusted government data.

Could government become swamped with data requests? Who knows, but in theory… it shouldn’t. Making data available should reduce the amount of time public servants spend responding to requests by diverting requests to open data portals. But let’s say Andrea’s concerns are valid and that, as a result of open data, citizens become more actively concerned and interested in how government works and thus Freedom of Information Act requests increase. The horror… citizens are interested in government! Citizens want to know how decisions are made! Remind me again… why is this a problem?

The real problem here isn’t access to data, it’s that the Freedom of Information Act process is itself broken. If open data creates a further demand for more transparent government and pushes us to foster better mechanisms for sharing government information, this is a good consequence. As for concerns that people might misrepresent public data, well a) people can already do this and we haven’t had a rash of bad applications, but even if they tried… people will stop using their service pretty quick.

Finally, another nice thing about public data is that it tends to get very clean, very quickly. My concern isn’t that government data will be misrepresented… I’m concerned that government data is already wrong and isn’t being verified. Knowing that someone might actually look at a data set is one of the most powerful incentives for organization to improve its collection. (Something Clay Shirky noted in a talk he made the other day at a Bioinformatics conference I’m at).

(There is of course, one group who may not see these a good consequences as it will change how they work: British public servant like Andrea’s client’s who raised the objections… but then they pay Gartner’s bills, not you.)

The Good

The end of Andrea Dimiao’s piece is where we find common ground. I agree that the Apps for Democracy competitions run the risk of limiting the definition of “the public” to citizen coders.  We want broader participation – particularly once more complex data sets like budgets, procurement and crime data are released – from academics, citizens groups and NGOs. Here in Vancouver we’ve talked about focusing any Apps competition on the themes of homelessness, housing and the environment, since these have been the dominant concerns of citizens in recent years.

More importantly, I agree (and love) Dimiao’s concept of employee-centric government. Indeed, my chapter for Tim O’Reilly’s upcoming book on Open Government makes a parallel argument, that namely we should stop trying to teach an analogue government to talk to a digital public and instead focus on making government digital (ie. getting it “open,” networked and using web 2.0 internally) first.

And perhaps most importantly, I agree that government 2.0 risks being over-hyped. I still believe in the potential, but know that getting there is going to be a painful process (mind the gap!). Government 2.0 advocates should expect lots of resistance and adoption problems ahead – but then change is painful.

How bloggers can keep the internet healthy

19 Replies

I’m continuously trying to brainstorming ways that Mozilla can find the next million mozillians and figure out activities they can do. I think I’ve stumbled on to a new one but would need some help to make it happen.

As some of you may know, as part of Mozilla Service Week Mozillians around the world donated their time and helped perform numerous Internet Health Checks. The goal here is to help people migrate off of Internet Explorer 6 (which is vulnerable to attacks and therefor makes the web less safe).

In my case, I’d already helped move pretty much anyone I know who uses IE 6 to something newer and better. What I need is a way to help prod people I don’t know.

What I thought might be interesting is if someone could build a blogger & wordpress plugin. This plug would ascertain what browser a visitor to your blog is using and… if they are using IE6… then the blog widget would let the reader know that the author of the blog strongly encourages them to upgrade to IE8, Firefox, Safari or really anything newer and safer. With this (hopefully relatively simple plug-in) Mozilla can engage the blogging community, enable people to help advocate for a safer internet and, most importantly, encourage still more IE6 users to move to something newer and safer.

Yes, it isn’t the be all and end all, but its another small idea that allows another group of people to contribute is an easy, but tangible and important way. That said, if this widget can’t be created, or if there is something easier/better that can be done, please, send me an email or comment below!

More ways to make open data sexy: 5 Municipal Apps I'd love to see (what are yours?)

13 Replies

One of the big goals of the open data project is to get many citizens interested in different ways the data can be used. Many citizens lack the skills to code up an application and creating a website is intimidating, but they may have ideas that could improve the city or be useful to many citizens.

In the hopes of spurring more interest in the open data and getting those not tradition involved, well… involved, I’ve created an “Ideas for the Taking” page on the Vancouver Open Data wiki. I’ve seeded the page with some of the ideas I promised I would share at the Open Data Hackathon last week . Some use open data, others don’t. Mostly however, I hop they spurn others to think of what is possible and what interests them. (PS. If you are a reader and the wiki is too confusing, just email me your idea and I’ll add it to the wiki with (or without, if you prefer) you’re name attached.

So here are some ideas I’ve brainstormed:

1. Stolen Bike Tracker

Vancouver’s cycling community is huge, sadly however, the city is plagued by a serious problem: stolen bicycles. There is no solution to this problem but I think a well crafted app could help minimize the nuisance. I can imagine an app or website in which you take a photo of your bike and upload it along with some identifying information(like the serial number) to a website. The picture stays hidden, however, if your bike gets tragically stolen you load up the apps and press the “my bike was stolen button.” This marks the physical place where your bike was stolen and activates your bike photo and marks it as stolen. Now cyclists, bike shop owners and the police can check bikes to see if they are stolen before buying them (or return them to their owner if they are recovered). In addition, a street map of bike theft would also be created. This could be particularly relevant since I suspect a great deal of bike theft is not reported. Finally, for those worried about privacy, I could imagine the app using a Craigslist style contact system that would preserve the anonymity of the original owner.

2. A Downtown East Side Landlord wiki

There are a few data sets that might allow for someone to create a geo-wiki of the DTES. I think it would be interesting to have a wiki that – on a building by building level – outlined who owned which residential buildings, what they charged in rent, a list of the room amenities and comments about the property’s management. It might also be interesting to enable photos to be posted so people can show the living conditions. Such a wiki might give the public (and prospective renters) a window into the deplorable conditions and poor practices of the worst offenders. It might also help City Staff deploy resources for investigating code violations and other questionable practices.

3. Everyblock+

Obviously, I think an Everyblock app for Vancouver would be great. The one new layer I’d love to see added to it is a charity button. With this button you would see what charities are operating on the block/area you are standing on. This is harder to imagine realizing, but cooler still would be a button that would allow you to then donate to that charity.

4. Burrard Bridge Trial Website

While not located on the Open Data Portal, the city has been releasing weekly data sets on vehicle, pedestrian and cycle trip across the Burrard Bridge Trial on the Burrard Trial blog. The data is not particularly well organized (you’d have to scrape it and its only granular to the 24hr time block – so no hour by hour data sets) but it is a start. I’d be fascinating to have a site that does a deeper analysis of the data and maybe shows it in a more interesting format. Maybe a discussion on carbon emissions reduced… still more interesting would be an analysis of bicycle accidents at present versus before the trial (data that is, sadly, not obviously available).

5. City Services vs. Land Value Mashup

It would be interesting to see what impact city services have on land values. I’m not sure if land value data is available (anyone know?) but mashing it up against the location of parks, community centres, schools, firehalls, and other city amenities would be interesting. While potentially interesting to prospective home owners (maybe a real estate agency should develop – or pay to develop – this app) I think it might also be of interest to the electorate and politicians.

One last one: A Library-Amazon Greasemonkey script

A Library-Amazon Greasemonkey search script allows a user to see if a book being displayed on an Amazon.ca website is available in the Vancouver Public Library. This has two benefits. First, it is WAY easier to find books on the Amazon site then the library site, so you can leverage Amazon’s search engine to find books (or book recommendations) at the VPL. Second, it’s a great way to keep the book budget in check!

The Vancouver Public Library has said that it will share access to its database that would allow such an app to work. I believe I have the email address for the relevant person somewhere on my computer who can make this happen. (I can get the contact info for the right person if someone nudges me.) Better still the necessary Greasemonkey script is already available (scripts exist for Palo  Alto, Seattle and Ottawa), it would be great if someone tweaked the script so it worked with the VPL.

Of course, I’m hoping that others are already hatching plans about how they’d like to use the city’s data to create something they feel passionate about. And remember, if there is an app you’d like to create but the data set isn’t available – take the Open Data survey to let your voice be heard! If any of these ideas interest you, go for it. If I can help in any way, let me know, I’m keen to contribute.

Garbage Collection now IS sexy: Introducing VanTrash

garbage-can_rgbA few months ago some of you will remember I blogged about How Open Data even makes Garbage collection sexier, easier and cheaper. I suggested that, with open data, coders could digitize the city’s garbage collection schedule and city maps and enable citizens to download it into their calendar or even set up a recurring email reminder.

The post went fairly viral being picked up places like here and here. As a result, two weeks later Luke Closs and Kevin Jones, two Vancouver based coders with a strong sense of fun and civic duty emailed me and said they’d actually scrapped the data and had created an alpha version of the site. I offered my (meagre) skills to help move the application forward and we began working on it.

Today, I’m pleased to say that VanTrash has been launched. If you live in Vancouver (or don’t) please do take a look at the website.

Our goal with VanTrash is twofold. First: we want a great service that leverages public data to helpmake our fellow citizens’ lives a little better or easier. Second: we’d like to sign up 3000 or more users.

Since there are about 260,000 households in Vancouver (although many have private contracted garbage pick up) 3000 users would represent between 1-2% of all households for whom the city collects their garbage. There are not that many services that citizens opt in for that get this market penetration – especially services created for virtually nothing. The more users we get, the stronger the message we send to government’s everywhere that government is a platform and that we need to let citizens built on top of it. More importantly, we demonstrate that great, and useful, things can be done for cheap, a lesson citizens and governments need to constantly relearn.

So if you live in Vancouver, and you think there service would be helpful to you (or perhaps to a forgetful or absent minded friend, family member or neighbour) please sign up or spread the word.

Vancouver's Open Data Portal: Use it, or Lose it.

As some of you saw yesterday via Twitter, Vancouver has launched a beta version of its open data portal. This is a major milestone for Vancouver on several levels. It is a testament to our politicians, who had the vision and foresight to embrace this idea. It is a tribute to the city’s staff who have worked unbelievably hard to make this project come alive so quickly. It is a triumph for those of us who advocate and have been working with the city to move us towards open government and government as platform. Finally, it represents an enormous opportunity for coders and citizens alike, and it is to this group that I’d like to address this blog post.

The Data Portal represents an opportunity for citizens, especially citizen coders, to help create a City that Thinks Like the Web: a city that enables citizens to create and access collective knowledge and information to create new services, suggest new ideas, and identify critical bugs in the infrastructure and services, among other a million other possibilities.

But the open data is only the part of the puzzle. Yes, our data is now beginning to be set free. But we have to use it.

If not, we’ll risk losing it.

I wish I could say that the city will share data no matter what and that political support will continue forever. But the fact is, municipal resources are limited. While the potential of open data is enormous, we need more than potential; we need some wins. More importantly, we need an active and engaged community working to make Vancouver better, more efficient, and more interesting because of our open data. We need to show politicians and public servants in Vancouver, but also in Edmonton, Calgary, Ottawa, Toronto, Montreal, Nanaimo, Moncton and other places across the country that citizens want access to data, and that if we get it, we will help their city (or province, or country) come alive in new and inventive ways.

Back in June, shortly after City Council passed Open3 (the nickname for the Open Motion), I gave this presentation to both City staff and at Open Web Vancouver. In it I described how “the bargain” Clay Shirky says exists on every successful web 2.0 site also exists in cities that want to think like the web.

In our case the bargain is simple: On one side, the city agrees to share as much data as it possibly can, in open formats, as quickly as it can. On the other side, the community – and in particular citizen coders – must make that data come alive in applications, websites and analysis. The city has taken the first step in fulfilling its side of the bargain. (And yes, we need to keep adding more data; there is work to be done.) Now it is time to activate the other half of the bargain. If we don’t, we put the deal at risk.

So what can you do?

First, you can code up an app, or find ways to help those who are. Indeed, there is going to be a Hackathon tomorrow evening at the Vancouver Archives to do just this. A number of projects are already underway that you can join – or start one yourself! I will be there myself, and I encourage you to swing by too.

Second, if you’d like to build an application, but the dataset you need is currently not available, then complete the city’s Open Data priority survey!

Third, come add ideas, resources and projects to the Vancouver Open Data Wiki.

I’m enormously excited to see what evolves next. As many of you know, I’ve been advising the Mayor’s Office on open data and open government for several months now – and through my work with them and with city staff, I’ve been deeply impressed by the energy and commitment that I’ve seen. As far as I know, only three major cities have created data portals such as this, and to do this in three months is incredible. Over the next few days I’m going to share some more thoughts on what the Open Data portal means for Vancouver. If you get a chance I hope you’ll send me your thoughts as well, or post some to your own blog if you have one.

The Bottled Water Debate Redux (a public policy case)

I’m a big believer that public policy cannot be written in a vacuum. All too often people become obsessed that their issue or problem is the only problem and so design policies that address their concern while ignoring secondary effects that can be as, if not more, costly.

This has been my point in the debate on bottled water. An outright ban might decrease the number of bottles that head to the landfill every year, but at what cost? First, if someone doesn’t buy water, I’ve suggested there is a good chance they’ll by something else… say Coke… which has many more calories and is less healthy. What are the aggregate health costs of having people drink more high-fructose corn syrup? Secondarily, if people are buying other drinks, then they aren’t really reducing the amount of waste, they are simply substituting it with other waste. We need to understand the bottled water not as strictly and environmental issue, but as a systems problem that impacts a number of areas. (Hence, why I’ve argued for any, and preferably larger, deposits on bottled water).

When I’ve written about this is the past some people have understandably disagreed with my logic, claiming that those who buy bottled water would never (or only rarely) buy a different, higher calorie drink. Recently, however, I stumbled across this interesting article in The Telegraph. It discusses the decline consumption of bottled water in Britain. The quote I found particularly interesting was this one:

Last year the bottled water market suffered a significant fall, as consumers – swayed by an environmental campaign and the high cost – turned away from the once fashionable drink.

However, figures from the market research firm AC Nielsen, which tracks in detail the spending habits of tens of thousands of consumers each week, indicated that 71 per cent of the money saved was spent not on free tap water, but rather than sugary soft drinks…

…this equates to Britons consuming an extra 1,700 tonnes of sugar and 6.8 billion calories,

So it would appear that if we ban bottled water there is a cost – in terms of health – that we should be expected to pay as a society. Moreover, while there is some reduction in waste it is not absolute, indeed, since just over 70% of money is still spent on beverages it is quite significant. This of course is because (as I’ve been arguing) people aren’t buying the water, they are buying the convenience (something they are less willing to pay for during a recession and hence the decline). Indeed, this hypothesis has been confirmed by another British survey conducted by the University of Birmingham. Its researchers:

…found that convenience and taste — not health — were the main motivating factors for choosing bottled water.

“The majority of participants believed that bottled water has some health benefits, but that they were not necessarily significant or superior to the benefits provided by tap water. Convenience and taste were more influential factors for participants when deciding to buy a bottle of water”.

So let us stop trying to scare or guilt consumers out of buying bottled water and instead focus on solutions that address both health and environmental issues. I suspect that if we increased bottle deposits significantly on all beverage containers (and made sure water containers had deposits on them) we might discover that people a) buy fewer beverages, b) will be more inclined to recycle/re-use the containers and c) don’t create an incentive to purchase sugary high-fructose beverages over water.

Okay, barring some national emergency, I promise not to write on bottled water again for a while – it’s just that the topic makes for such a great policy case study on looking at an issue as a system versus seeing it as single issue problem.