Category Archives: cool links

My LRC Review of "When the Gods Changed" and other recommended weekend readings

This week, the Literary Review of Canada published my and Taylor Owen’s review of When the Gods Changed: The Death of Liberal Canada by Peter C. Newman. For non-Canadians Peter Newman is pretty much a legend when it comes to covering Canadian history and politics, he was editor of the country’s largest newspaper and main news magazine and has published over 35 books. I also think the review will be of interest to non-Canadians since I think the topic of the decline of Liberal Canada are also true for a number of other countries experiencing more polarized politics.

Some other articles I’ve been digesting that I recommend for some Friday or weekend reading:

Why China’s Political Model Is Superior

This one is a couple of months old, but it doesn’t matter. Fascinating read. For one it shows the type of timelines that the Chinese look at the world with. Hint. It is waaayyyy longer than ours. Take a whiff:

In Athens, ever-increasing popular participation in politics led to rule by demagogy. And in today’s America, money is now the great enabler of demagogy. As the Nobel-winning economist A. Michael Spence has put it, America has gone from “one propertied man, one vote; to one man, one vote; to one person, one vote; trending to one dollar, one vote.” By any measure, the United States is a constitutional republic in name only.

Unattractive Real Estate Agents Achieve Quicker Sales

Before getting serious on you again, here’s a lighter more interesting note. I often comment in talks I give that real estate agents rarely use data to attract clients – mostly just pictures of themselves. Turns out… there might be more data in that then I thought! Apparently less attractive agents sell homes faster and work harder. More attractive agents take longer, but get more money. Food for thought here.

Andrew Coyne: Question isn’t where conservatism is going, but where has it gone

Another oldie but a goody. Liberal Canada may be dead, but it appears that Conservative Canada isn’t in much better shape. I’ve always enjoyed Coyne and feel like he’s been sharper than usual of late (since moving back to the National Post). For Americans, there may be some interesting lessons in here for the Tea Party movement. Canada experienced a much, much lighter form of conservative rebellion with creation of the Reform Party in the late 80s/early 90s which split off from establishment conservatives. Today, that group is now in power (rebranded) but Coyne assesses that much of what they do has been watered down. But not everything… to the next two articles!

Environmental charities ‘laundering’ foreign funds, Kent says

Sadly, Canada’s “Environment” Minister is spending most of his time attacking environmental groups. The charge is that they use US money to engage in advocacy against a pipeline to be built in Canada. Of course “Laundering” is a serious charge (in infers illegal activity) and given how quick the Conservatives have been in suing opponents for libel Kent had better be careful the stakeholders will adopt this tactic. Of course, this is probably why he doesn’t name any groups in particular (clever!). My advice, is that all the groups named by the Senate committee should sue him, then, to avoid the lawsuit he’d have to either a) back down from the claim altogether, or b) be specific about which group he is referring to to have the other suits thrown out. Next headline… to the double standard!

Fraser Institute co-founder confirms ‘years and years’ of U.S. oil billionaires’ funding

Some nifty investigative work here by a local Vancouver reporter finds that while the Canadian government believes it is bad for environmental groups to receive US funds for advocacy, it is apparently, completely okay for Conservative groups to receive sums of up to $1.7M from US oil billionaires. Ethical Oil – another astro-turf pro-pipeline group does something similar. It receives money from Canadian law firms that represent benefiting American and Chinese oil interests. But that money is labelled “Canadian” because it is washed through Canadian law firms. Confused? You should be.

What retail is hired to do: Apple vs. IKEA

I love that Clay Christiansen is on twitter. The Innovator’s Dilemma is a top 5 book of all time for me. Here is a great break down of how IKEA and Apple stores work. Most intriguing is the unique value proposition/framing their stores make to consumers which explains their phenomenal success as why they are often not imitated.

NASA space hackathon (in Vancouver) this weekend

So, many, many things I’d like to blog upon at the moment. I’m in Brasilia at the Open Government Partnership meeting, so obviously lots to talk about there, and, of course, Canada Post has completely lost it and is suing a company over postal code data but it’s been twenty hour days and those post more thought (and rest).

In the mean time, if you are in Vancouver, the very cool people over at Steam Clock Software – and Angelina Fabbro in particular – are organizing the Vancouver edition of the NASA International Space Apps challenge THIS WEEKEND at the Network Hub.

You should feel free to show up as a team, or come on your own and join a team. It’s worth checking this out now, as there is some pretty rich data sets to explore so getting familiar with it before hand is… recommended.

Mostly I just love that NASA is thinking this way.

It’s also good that the Canadian government is not involved, since even getting them to acknowledge there participation would have proven too controversial (and sucked up a ton of public servants time).

Space geeks… unite!

Beautiful Maps – Open Street Map in Water Colours

You know, really never know what the web is going to throw at you next. The great people over at Stamen Design (if you’ve never heard of Stamen you are really missing out – they are probably the best data visualization company I know) have created a watercolor version of Open Street Maps.

Why?

Because they can.

It’s a wonderful example of how you, with the web, you can build on what others have done. Pictured below my home town of Vancouver – I suggest zooming out a little as the city really comes into focus when you can see more of its geography.

Some Bonus Awesomeness Facts about all this Stamen goodness:

  • Stamen has a number of Creative Commons licensed map templates that you can use here (and links to GitHub repos)
  • Stamen housed Code for America in its early days. So they don’t just make cool stuff. The pitch in and help out with cool stuff too.
  • Former Code for America fellow Michael Evans works there now.

 

Some upcoming talks

Sorry for the lack of posts this week – just some calm before gathering storm. April and May are going to be intense.

For those interested in these things I’ve a number of upcoming talks I’ll be giving and and conferences I’ll be attending. Many of these are open to the public in case you are in the neighborhood.

Conference: OECD High Level Meeting on E-Government: New ICT Solutions for Public Sector Agility
Topic: I’ll be on a panel discussing how governments can leverage technology to advance public policy objectives
Location: Mexico City
Date: Monday, March 26th

Conference: Creative Commons Salon on Open Data (hosted by CIPPIC and the Creative Law Society)
Topic: Open Data in Canada
Location: Ottawa
Date: Friday, March 30th

Conference: Democracy on Demand – hosted by the UBC International Relations Student Association
Topic: The Geopolitics of Open
Location: Vancouver
Date: Saturday, March 31st

Conference: Fostering a Critical Development Perspective on Open Government Data
Topic: Seminars on the implications of Open Data, particularly in southern economies
Location: Brasilia
Date: April 16-17

Conference: Open Government Partnership
Topic: (TBD) Panel on Access to Information and Open Data
Location: Brasilia
Date: April 18

Conference: Impacto del Derecho de Acceso a la Información en la participación ciudadana
Topic: Open Data, Transparency and the Future of Government
Location: Santiago, Chile
Date: April 19

Conference: Saskatchewan 3.0 Summit
Topic: The Possibilities and Challenges of Technology in Government
Location: Regina
Date: April 24

What I'm Digesting: Good Reads from the First Week of January

Government Procurement is Broken: Example #5,294,702 or “The Government’s $200,000 Useless Android Application” by Rich Jones

This post is actually a few months old, but I stumbled on it again the other day and could help but laugh and cry at the same time. Written by a freelance computer developer, the post traces the discovery of a simply iphone/android app the government paid $200,000 to develop that is both unusable from a user interface perspective and does not actually work.

It’s a classic example of how government procurement is deeply, deeply broken (a subject I promise to write more about soon). Many governments – and the bigger they are, the worse it gets – are incapable of spending small sums of money. Any project, in order to work in their system, must be of a minimum size, and so everything scales up. Indeed simply things are encouraged to become more expensive so that the system can process them. There is another wonderful (by which I mean terrifying) example of this in one of the first couple of chapter of Open Government.

How Governments Try to Block Tor by Roger Dingledine

For those who don’t know what Tor is, it’s “free software and an open network that helps you defend against a form of network surveillance that threatens personal freedom and privacy, confidential business activities and relationships, and state security known as traffic analysis.” Basically, if you are someone who doesn’t want anyone – particularly the government – seeing what websites you visit, you need Tor. I don’t think I need to say how essential this service is, if say, you live China, Iran or Syria or obviously Egypt, Libya, Tunisia or any of the other states still convulsing from the Arab Spring.

The hour and 10 minute long speech is a rip roaring romp through the world of government surveillance. It’s scary than you want to know and very, very real. People die. It’s not pretty but it is incredible. For those of you not technically inclined, don’t be afraid, there is techno-babble you won’t understand but don’t worry, it won’t diminish the experience.

The Coming War on General Computation by Cory Doctorow

Another video, also from the Chaos Communication Conference in Berlin (how did I not know about this conference? pretty much everything I’ve seen out of it has been phenomenal – big congrats to the organizers).

This video is Cory Doctorow basically giving everybody in the Tech World a solid reality check the state of politics and technology. If you are a policy wonk who cares about freedom of choice, industrial policy, copyright, the economy or individual liberty, this strikes video is a must view.

For those who don’t know Cory Doctorow (go follow him on Twitter right now) he is the guy who made Minister Moore look like a complete idiot on copyright reform (I also captured their twitter debate here).

Sadly, the lunacy of the copyright bill is only going to be the beginning of our problems. Watch it here:

Why is Finding a Post Box so Hard?

Sometimes it is the small things that show how government just gets it all so wrong.

Last Thursday The Daily Show’s Wyatt Cenac has a little bit on the US Post Office and its declining fortunes as people move away from mail. There is no doubt that the post offices days are numbered, but that doesn’t mean the decline has to be as steep as it is. Besides there are things they could be doing to make life a little easier to use them (and god knows they should be doing anything they can, to be more appealing).

Take, for example, the humble post office box. They can be frustratingly hard to locate. Consider Broadway and Cambie – one of the busiest intersections in Vancouver – and yet there is no post box at the intersection. (I eventually found it one block east on broadway) but I carried around a letter for 3 weeks before I eventually found one.

In short why is there not digital map (or for techies, and API) for post box locations? I could imagine all sorts of people that might make use of it. Would it be nice to just find out – where is the closest post box to where I’m standing? More importantly, it might actually help the post office attract a few extra customers. It certainly wouldn’t hurt customer service. I’ve wondered for a couple of years why it doesn’t publish this data set.

Turns out I’m not the only with this frustration. My friend Steven Tannock has channeled his frustration into a simple app called Wherepost.ca. It’s a simple website – optimized for mobile phone use – that allows users to add post boxes as well as find the one nearest to them. In short, Steven’s trying to create a public data set of post box locations by crowd sourcing the problem. If Canada Post won’t be helpful… we’ll help one another.

Launched on Thursday with 20 post office box locations, there are now over 400 boxes mapped (mostly in the Vancouver area) with several dozen users contributing. In addition, Steven tells me users in at least 2 other countries have asked for new icons so they can add post boxes where they live. It seems Canadians aren’t the only ones frustrated about not knowing where the nearest post box is.

The ideal, of course, would be for Canada Post to publish an API of all post box locations. I suspect however, that they either don’t actually know where they all are in a digital form (at which point they should really help Steven as he is doing them a huge service) or revealing their location will be seeing as sacrificing some important IP that people should pay for. Remember, this is an organization that refuses to make Postal Code data open, a critical data set for companies, non-profits and governments.

This isn’t the worlds fanciest app but its simplicity is what makes it so great, and so useful. Check it out at WherePost.ca and… of course, add a post box if you see one.

 

 

International Open Data Hackathon, Dec 3rd. It's coming together.

So a number of things have started to really come together for this Saturday Dec 3rd. I’ve noticed a number of new cities being tweeted about (hello Kuala Lumpur & Oakland!) and others adding themselves to the wiki. Indeed, we seem to be above 40 cities. It is hard to know how many people will be showing up in each given city, but in Vancouver I know that we already over 20 registered, while in Ottawa they are well above 40. If other cities have similar numbers it’s a great testament to the size of the community out there interested in playing with open government data.

A few thoughts to share with people as we get ready for the big day.

1. Leverage existing projects.

I’ve mentioned a few times that there are some great existing projects out there that can be easily leveraged.

In that vein I’ve noticed the good people at the Open Knowledge Foundation, who are behind OpenSpending (the project that powers WherDoesMyMoneyGo.org) have not only made their software easier to use but have put up some helpful instructions for creating your own instance up on the wiki. One hope I have for Saturday is that a number of different places might be able to visualize local budgets in much easier to understand ways. OpenSpending has the potential of being an enormously helpful tool for communities trying to understand their budget – hopefully we can provide some great examples and feedback for its creators.

In addition, the folks at MySociety have provided some helpful advice on the wiki for those interested in spinning up a version of MapIt for their country.

2. Get Data Now, Not on Saturday!

Here in Vancouver, my friend Luke C asked if we could get bicycle accident data for the city or province as he wanted to play around with it and maybe visualize it on December 3rd. It just so happened I had a contact at the Insurance Company of British Columbia (ICBC) which insures every vehicle in the province. I reached out and, after going through their request process, now have the data set to share with Luke.

The key piece here: now is the time to check and see if data you are interested in is available, see investigate what is out there, and request it from various stakeholders if it is not.

3. Share Your Code, Share your Data

Indeed, one advantage of having the BC bicycle accident data early is that I can start sharing it with people immediately. I’ve already uploaded the data set (all 6400 lines) onto BuzzData’s site here so others can download it, clone it, and share their own work on it. That way, even if Luke and I get separated, he’s still got something to hack on!

So please do let people know where they can find data you are hacking on, as well as project you’re hacking on. The Open Data Day Projects 2011 wiki page currently sits empty (as should be expected). But take a swing by the page 2010 project page, notice how it is quite full… I’d love to see us replicate this success. I’m hoping people link to not just their projects, but also Github repos, scraperwiki creations, BuzzData accounts and other places.

If you have a project and you think people in open data day hackathons in other cities might be interested, put it in the project page and tweet about it using the #odhd hashtag. You may discover there are people out there who feel as passionately about your project as you do!

4. Let’s Get Connected

Speaking of sharing, my friend Edward O-G, who is organizing the hackathon in Ottawa, did a great job last year setting up some infrastructure so people from different hackathons could video conference with one another. This year I think we’ll try using Google hangouts on google+. However, there is a non-trivial risk that this will not scale super well.

So…

Edward also suggested (brilliantly) that people create YouTube videos of whatever they create during the hackathon or in the days and weeks that follow. Please post those links to the Open Data Day Projects 2011 wiki page as well. There were a few projects last year that had youtube videos and they were very helpful, particularly when a project isn’t quite ready for prime time. It gives us a taste of what will be available. It also becomes something we can point people to.

5. Have Fun, Do What Is Interesting

Remember, Open Data Day is about meeting people, learning about open data, and working on something that you feel passionate about. This is all very decentralized and informal – no one is going to come and save your hackathon… it is up to you! So make sure you find something you think is worth caring about and work on it. Share your idea, and your passion, with others, that’s what makes this fun.

Can’t wait to hear what people are up to. Please feel free to email or tweet at me what you’re working on. I’d love to hear about it and blog about them.

 

Here in Vancouver, the open data hackathon will be happening at the offices of FoodTree, which has some of its own developers working on Open Food data.(If you haven’t signed up yet, definitely do so here).

 

 

Weaving Foreign Ministries into the Digital Era: Three ideas

Last week I was in Ottawa giving a talk at the Department of Foreign Affairs talking about how technology, new media and open innovation will impact the department’s it work internally, across Ottawa and around the world.

While there is lots to share, here are three ideas I’ve been stewing on:

Keep more citizens safe when abroad – better danger zone notification

Some people believe that open data isn’t relevant to departments like Foreign Affairs or the State Department. Nothing could be further than the truth.

One challenge the department has is getting Canadians to register with them when they visit or live in a country labeled by the department as problematic for traveling in its travel reports (sample here). As you can suspect, few Canadians register with the embassy as they are likely not aware of the program or travel a lot and simply don’t get around to  it.

There are other ways of tackling this problem that might yield broader participation.

Why not turn the Travel Report system into an open data with an API? I’d tackle this by approaching a company like TripIt. Every time I book an airplane ticket or a hotel I simply forward TripIt the reservation, which they scan and turn into events that then automatically appear my calendar. Since they scan my travel plans they also know which country, city and hotel I’m staying in… they also know where I live and could easily ask me for my citizenship. Working with companies like TripIt (or Travelocity, Expedia, etc…) DFAIT could co-design an API into the departments travel report data that would be useful to them. Specifically, I could imagine that if TripIt could query all my trips against those reports then any time they notice I’m traveling somewhere the Foreign Ministry has labelled “exercise a high-degree of caution” or worse trip TripIt could ask me if I’d be willing to let them forward my itinerary to the department. That way I could registry my travel automatically, making the service more convenient for me, and getting the department more information that it believes to be critical as well.

Of course, it might be wise to work with the State Department so that their travel advisories used a similarly structured API (since I can assume TripIt will be more interested in the larger US market than the Canadian market) But facilitating that conversation would be nothing but wins for the department.

More bang for buck in election monitoring

One question that arose during my talk came from an official interested in elections monitoring. In my mind, one thing the department should be considering is a fund to help local democracy groups spin up installations of Ushahidi in countries with fragile democracies that are gearing up for elections. For those unfamiliar with Ushahidi it is a platform developed after the disputed 2007 presidential election in Kenya that plotted eyewitness reports of violence sent in by email and text-message on a google map.

Today it is used to track a number of issues – but problems with elections remain one of its core purposes. The department should think about grants that would help spin up a Ushahidi install to enable citizens of the country register concerns and allegations around fraud, violence, intimidation, etc… It could then verify and inspect issues that are flagged by the countries citizens. This would allow the department to deploy its resources more effectively and ensure that its work was speaking to concerns raised by citizens.

A Developer version of DART?

One of the most popular programs the Canadian government has around international issues is the Disaster Assistance Response Team (DART). In particular, Canadians have often been big fans of DART’s work in purifying water after the boxing day tsunami in Asia as well as its work in Haiti. Maybe the department could have a digital DART team, a group of developers that, in an emergency could help spin up Ushahidi, Fixmystreet, or OpenMRS installations to provide some quick but critical shared infrastructure for Canadians, other countries’ response teams and for non-profits. During periods of non-crisis the team could work on these projects or supporting groups like CrisisCommons or OpenStreetMaps, helping contribute to open source projects that can be instrumental in a humanitarian crisis.

 

As Canada Searches for its Open Government Partnership Commitments: A Proposal

Just before its launch in New York on September 20th, the Canadian Government agreed to be a signatory of the Open Government Partnership (OGP). Composed of over 40 countries the OGP signatories are required to create a list of commitments they promise to implement. Because Canada signed on just before the deadline it has not – to date – submitted its commitments. As a result, there is a fantastic window for the government to do something interesting with this opportunity.

So what should we do? Here are the top 5 suggestions I propose for Canada’s OGP Commitments:

Brief Background on Criteria:

Before diving in, it is worth letting readers know that there are some criteria for making commitments. Specifically, any commitment must tackle at least one of the five “core” challenges: improve public services, increase public integrity, more effectively manage public resources, create safer communities, and increase corporate accountability.

In addition, each recommendation should reflect at least one of the core OGP principles, which are: transparency, citizen participation, accountability, and technology and innovation.

The Top Ten

Having reviewed several other countries commitments and being familiar with both what Canada has already done and what it could do, attached are 10 commitments I would like to see our government make to the OGP.

1. Be open about developing the commitments

Obviously there are a number of commitments the government is going to make since they are actions or programs that government was going to launch anyways. In addition, there will be some that will be new ideas that public servants or politicians have been looking for an opportunity to champion and now have an excuse. This is all fine and part of the traditional way government works.

But wouldn’t it be nice if – as part of the open government partnership – we asked citizens what they thought the commitments should be? That would make the process nicely consistent with the principles and goals of the OGP.

Thus the government should launch a two week crowd sourced idea generator, much like it did during the Digital Economy consultations. This is not suggestion that the ideas submitted must become part of the commitments, but they should inform the choices. This would be a wonderful opportunity to hear what Canadians have to say. In addition, the government could add some of its own proposal into the mix and see what type of response they get from Canadians.

2. Redefine Public as Digital: Pass an Online Information Act

At this year’s open government data camp in Warsaw, the always excellent Tom Steinberg noted that creating a transparent government and putting in place the information foundations of a digital economy will be impossible unless access to government data is not a gift from government (that can be taken away) but a right every citizen has. At the same time Andrew Rasiej of Tech President advocated that we must redefine public as digital. A paper print out in a small office in the middle of nowhere, does not make for  “public disclosure” in the 21st century. It’s bad for democracy, it’s bad for transparency, and it is grossly inefficient for government.

Thus, the government should agree to pass a Online Information Act, perhaps modeled on that proposed in the US Senate, that

a) Any document it produces should be available digitally, in a machine readable format. The sham that the government can produce 3000-10,000 printed pages about Afghan detainees or the F-35 and claim it is publicly disclosing information must end.

b) Any data collected for legislative reasons must be made available – in machine readable formats – via a government open data portal.

c) Any information that is ATIPable must be made available in a digital format. And that any excess costs of generating that information can be born by the requester, up until a certain date (say 2015) at which point the excess costs will be born by the ministry responsible. There is no reason why, in a digital world, there should be any cost to extracting information – indeed, I fear a world where the government can’t cheaply locate and copy its own information for an ATIP request as it would suggest it can’t get that information for its own operations.

3. Sign the Extractive Industries Transparency Initiative

As a leader in the field of resource extraction it is critical that Canada push for the highest standards in a sector that all too often sees money that should be destined for the public good get diverted into the hands of a few well connected individuals. Canada’s reputation internationally has suffered as our extractive resource sector is seen as engaging in a number of problematic practices such as bribing public officials – this runs counter to the Prime Minister’s efforts to promote democracy.

As a result, Canada should sign, with out delay, the Extractive Industries Transparency Initiative, much like the United States did in September. This can help signal our desire for a transparent extractive industry, one in which we play a significant role.

4. Sign on to the International Aid Transparency Initiative

Canada has already taken significant steps to publishing its aid data online, in machine readable formats. This should be applauded. The next step is to do so in a way that conforms with international standards so that this data can be assessed against the work of other donors.

The International Aid Transparency Initiative (IATI) offers an opportunity to increase transparency in foreign aid, better enable the public to understand its aid budget, compare the country’s effectiveness against others and identify duplication (and thus poorly used resources) among donors. Canada should agree to implement IATI immediately. In addition, it should request that the organizations it funds also disclose their work in ways that are compliant with IATI.

5. Use Open Data to drive efficiency in Government Services: Require the provinces to share health data – particularly hospital performance – as part of its next funding agreement within the Canada Health Act.

Comparing hospitals to one another is always a difficult task, and open data is not a panacea. However, more data about hospitals is rarely harmful and there are a number of issues on which it would be downright beneficial. The most obvious of these would be deaths caused by infection. The number of deaths that occur due to infections in Canadian hospitals is a growing problem (sigh, if only open data could help ban the antibacterial wipes that are helping propagate them). Having open data that allows for league tables to show the scope and location of the problem will likely cause many hospitals to rethink processes and, I suspect, save lives.

Open data can supply some of the competitive pressure that is often lacking in a public healthcare system. It could also better educate Canadians about their options within that system, as well as make them more aware of its benefits.

6. Reduce Fraud: Find Fraud by Creating a Death List

In an era where online identity is a problem it is surprising to me that I’m unable to locate a database of expired social insurance numbers. Being able to querry a list of social security numbers that belong to dead people might be a simple way to prevent fraud. Interestingly, the United States has just such a list available for free online. (Side fact: Known as the Social Security Death Index this database is also beloved by genealogist who use it to trace ancestry).

7. Save lives by publishing a API of recall data

The only time the public finds out about a product recall is after someone has died. This is a terribly tragic, not to mention grossly inefficient, outcome. Indeed, the current approach is a classic example of using 21st century technology to deliver a service in a 19th century manner. If the government is interested in using the OGP to improve government services it should stop just issuing recall press releases and also create an open data feed of recalled products. I expand on this idea here.

If the government were doubly smart it would work with major retailers – particularly in the food industry – to ensure that they regularly tap into this data. In an ideal world any time Save-on-Foods, Walmart, Safeway, or any other retailers scans product in their inventory it would immediately check it against the recall database, allowing bad food to be pulled out of production before it hits the shelves. In addition, customers who use loyalty cards could be called or emailed to be informed that they had bought a product that had been recalled. This would likely be much more effective than hoping the media picks the story up.

8. Open Budget and Actual Spending Data

For almost a year the UK government has published all spending data, month by month, for each government ministry (down to the £500 in some, £25,000 in others). More over, as an increasing number of local governments are required to share their spending data it has lead to savings, as government begin to learn what other ministries and governments are paying for similar services.

Another bonus is that it becomes possible to talk about the budget in new and interesting ways. This BEAUTIFUL graphic was published in the Guardian, while still complicated it is much easier to understand than any government document about the budget I have ever seen.

Public-spending-graphic-0051

9. Allow Government Scientists to speak directly to the media about their research.

It has become a reoccurring embarrassment. Scientists who work for Canada publish an internationally recognized ground break paper that provides some insight about the environment or geography of Canada and journalists must talk to government scientists from other countries in order to get the details. Why? Because the Canadian government blocks access. Canadians have a right to hear the perspectives of scientists their tax dollars paid for – and enjoy the opportunity to get as well informed as the government on these issues.

Thus, lift the ban that blocks government scientists from speaking with the media.

10. Create a steering group of leading Provincial and Municipal CIOs to create common schema for core data about the country.

While open data is good, open data organized the same way for different departments and provinces is even better. When data is organized the same way it makes it easier to citizens to compare one jurisdiction against another, and for software solutions and online services to emerge that use that data to enhance the lives of Canadians. The Federal Government should use its convening authority to bring together some of the countries leading government CIOs to establish common data schemas for things like crime, healthcare, procurement, and budget data. The list of what could be worked on is virtually endless, but those four areas all represent data sets that are frequently requested, so might make for a good starting point.

The State of Open Data 2011

What is the state of the open data movement? Yesterday, during my opening keynote at the Open Government Data Camp (held this year in Warsaw, Poland) I sought to follow up on my talk from last year’s conference. Here’s my take of where we are today (I’ll post/link to a video of the talk as soon as the Open Knowledge Foundation makes it available).

Successes of the Past Year: Crossing the Chasm

1. More Open Data Portals

One of the things that has been amazing to witness in 2011 is the veritable explosion of Open Data portals around the world. Today there are well over 50 government data catalogs with more and more being added. The most notable of these was probably the Kenyan Open Data catalog which shows how far, and wide, the open data movement has grown.

2. Better Understanding and More Demand

The things about all these portals is that they are the result of a larger shift. Specifically, more and more government officials are curious about what open data is. This is not to say that understanding has radically shifted, but many people in government (and in politics) now know the term, believe there is something interesting going on in this space, and want to learn more. Consequently, in a growing number of places there is less and less headwind against us. Rather than screaming from the rooftops, we are increasingly being invited in the front door.

3. More Experimentation

Finally, what’s also exciting is the increased experimentation in the open data space. The number of companies and organizations trying to engage open data users is growing. ScraperWiki, the DataHub, BuzzData, Socrata, Visua.ly, are some of the products and resources that have emerged out of the open data space. And the types of research and projects that are emerging – the tracking of the Icelandic volcano eruptions, the emergence of hacks and hackers, micro projects (like my own Recollect.net) and the research showing that open data could be generating savings of £8.5 million a year to governments in the Greater Manchester area, is deeply encouraging.

The Current State: An Inflection Point

The exciting thing about open data is that increasingly we are helping people – public servants, politicians, business owners and citizens imagine a different future, one that is more open, efficient and engaging. Our impact is still limited, but the journey is still in its early days. More importantly, thanks to success (number 2 above) our role is changing. So what does this mean for the movement right now?

Externally to the movement, the work we are doing is only getting more relevant. We are in an era of institution failure. From the Tea Party to Occupy Wall St. there is a recognition that our institutions no longer sufficiently serve us. Open data can’t solve this problem, but it is part of the solution. The challenge of the old order and the institutions it fostered is that its organizing principle is built around the management (control) of processes, it’s been about the application of the industrial production model to government services. This means it can only move so fast, and because of its strong control orientation, can only allow for so much creativity (and adaption). Open data is about putting the free flow of information at the heart of government – both internally and externally – with the goal of increasing government’s metabolism and decentralizing societies’ capacity to respond to problems. Our role is not obvious to the people in those movements, and we should make it clearer.

Internally to the movement, we have another big challenge. We are at a critical inflection point. For years we have been on the outside, yelling that open data matters. But now we are being invited inside. Some of us want to rush in, keen to make advances, others want to hold back, worried about being co-opted. To succeed, it is essential we must become more skilled at walking this difficult line: engaging with governments and helping them make the right decisions, while not being co-opted or sacrificing our principles. Choosing to not engage would, in my opinion, be to abscond from our responsibility as citizens and open data activists. This is a difficult transition, but it will be made easier if we at least acknowledge it, and support one another in it.

Our Core Challenges: What’s next

Looking across the open data space, my own feeling is that there are three core challenges that are facing the open data movement that threaten to compromise all the successes we’ve currently enjoyed.

1. The Compliance Trap

One key risk for open data is that all our work ends up being framed as a transparency initiative and thus making data available is reduced to being a compliance issue for government departments. If this is how our universe is framed I suspect in 5-10 years governments, eager to save money and cut some services, will choose to cut open data portals as a cost saving initiative.

Our goal is not to become a compliance issue. Our goal is to make governments understand that they are data management organizations and that they need to manage their data assets with the same rigour with which they manage physical assets like roads and bridges. We are as much about data governance as we are open data. This means we need to have a vision for government, one where data becomes a layer of the government architecture. Our goal is to make data platform one that not only citizens outside of government can build on, but one that government reconstructs its policy apparatus as well as its IT systems at top of. Achieving this will ensure that open data gets hardwired right into government and so cannot be easily shut down.

2. Data Schemas

This year, in the lead up to the Open Data Camp, the Open Knowledge Foundation created a map of open data portals from around the world. This was fun to look at, and I think should be the last time we do it.

We are getting to a point where the number of data portals is becoming less and less relevant. Getting more portals isn’t going to enable open data to scale more. What is going to allow us to scale is establishing common schemas for data sets that enable them to work across jurisdictions. The single most widely used open government data set is transit data, which because it has been standardized by the GTFS is available across hundreds of jurisdictions. This standardization has not only put the data into google maps (generating millions of uses everyday) but has also led to an explosion of transit apps around the world. Common standards will let us scale. We cannot forget this.

So let’s stop mapping open data portals, and start mapping datasets that adhere to common schemas. Given that open data is increasingly looked upon favourably by governments, creating these schemas is, I believe, now the central challenge to the open data movement.

3. Broadening the Movement

I’m impressed by the hundreds and hundreds of people here at the Open Data Camp in Warsaw. It is fun to be able to recognize so many of the faces here, the problem is that I can recognize too many of them. We need to grow this movement. There is a risk that we will become complacent, that we’ll enjoy the movement we’ve created and, more importantly, our roles within it. If that happens we are in trouble. Despite our successes we are far from reaching critical mass.

The simple question I have for us is: Where is the United Way, Google, Microsoft, the Salvation Army, Oxfam, and Greenpeace? We’ll know were are making progress when companies – large and small – as well as non-profits – start understanding how open government data can change their world for the better and so want to help us advance the cause.

Each of us needs to go out and start engaging these types of organizations and helping them see this new world and the potential it creates for them to make money or advance their own issues. The more we can embed ourselves into other’s networks, the more allies we will recruit and the stronger we will be.