Category Archives: open data

Requests for Endorsements: My Application to Attend The Open Government Partnership

Dear friends,

Below (first in english then in a rough french translation – my spoken is much better than my written so I’ve relied on Google translate) is my application letter to attend the April 16-18 Annual Open Government Partnership meeting in Brasilia as a Civil Society Representative.

The first reason I’m posting this is in an effort to make my application as transparent as possible. By posting it, if people have comments, additions, questions,I thought it might be helpful to post it. I can’t promise to engage every issue – I know some people have disagreed with positions I have taken – but I will do my best to engage what I believe is a broad community that cares about everything from open data, to access to information requests, to simply having more accessible MPs and websites.

Another – connected – reason I’m posting my application is I need your help. As part of the process I’m asked to submit organizations that endorse my application. For those of you who already have expressed this support… thank you. I will definitely add you to this letter. If you are willing to support my application and have not yet let me know, please do send me an email (or comment below). I need to add you organization (if any), your name and your email address.

The application deadline is February 6th (yes, it would have been better to get this up sooner) so ideally any feedback would be great to get today or this weekend. That said, I would still love to get feedback after the 6th in case I do go, I’d still like to be able to listen for and act on what you care about.

If you have other questions about or suggestions for the application, please let me know. Thank you!

Dear Open Government Partnership,

I am writing to express my desire to participate as a local civil society member at the 2012 OGP meeting in Brazil.

I have been active in Open Government for the past 7 years working as an advocate, adviser and chronicler of transparency in Canada and around the world.

Background and Engagement in OGP Issues

As an advocate, I’ve spoken about the challenges around Open Government across Canada at the municipal, provincial and federal levels. Internationally, I’ve given the opening keynote at the last two International Open Government Data Camps hosted by the Open Knowledge Foundation, was invited to address the 7^th International Conference of Information Commissioners in Ottawa and Mexico’s Semana Nacional de la Transparencia, and given talks at the Gov 2.0 Expo and Summit in Washington DC (at which the host, Tim O’Reilly, stated, “If you read only one blog in the Gov 2.0 space, you should read eaves.ca”).

In addition to speaking, I’ve tried to actively demonstrate ways open government can improve our communities. My belief is that we need a broad set of ways to engage citizens in open government – some will be motivated by accountability, but others will be engaged by simply having their lives made easier. Consequently, in pursuit of advancing accountability, I worked with a team of developers sponsored by Microsoft to create Emitter.ca, a website that mashes up pollution, politician, and company data to enable citizens to identify heavy polluters in their neighborhoods and region. While trying to find ways to show how open government can promote better services, I worked with friends to create Recollect.net, a simple service that uses open data to remind citizens to take out their recycling and garbage. And as an advocate, when the Federal Government lagged by years behind the US and the UK in creating an open data portal, I created http://www.datadotgc.ca which sought to track open data sets already being shared on various ministry website to demonstrate that, contrary to its position, the government already had a policy infrastructure to do open data. This site helped pressure the government into launching its own formal open data website.

I’ve also worked actively in the Open Government space by engaging with governments directly. In 2009, I co-drafted the open motion for the Mayor and Council of the City of Vancouver. This lead to the first city council motion in the world directing city staff to make open data part of their activities. As a result, Vancouver launched the world’s second municipal open data portal (after Washington DC). I also worked with the city’s IT staff to revise procurement rules to make open data a required specification as well make open source software a permissible option. At the federal level, I worked with NGO and educated key government players to shift Canada – which had been skeptical – into agreeing to participate in the OGP.

I have, however, also been critical when necessary. I’ve written pieces in newspapers and on my blog when governments have failed to be transparent or have taken steps in the wrong direction.

Meeting Contribution and Learnings

The Open Government Partnership provides civil society members with a rare carrot and stick for engaging their government on the issue of open government. Because it requires the government to set clear goals around transparency in an international forum, it provides civil society with leverage to hold the government to account.

While this leverage must be handled responsibly (factually incorrect critiques will erode the public’s confidence in civil society organizations), if properly used it can compel the government to move more aggressively on fixing problems in this area. This is of particular urgency in Canada, where government transparency has been in decline over the past several decades. Once considered cutting age, Canada’s access to information regime is wildly out of date. Access to information requests are handled more slowly than ever and access to government information – with the exception of a dramatic improvement in the area of open data – is becoming more restrictive. My goal at the Open Government Partnership will be to engage other government and NGOs to understand the transparency benchmarks being set by other governments that can be used as a way by which Canadians can judge the progress of their own government. I can also share my own experiences in moving open data policies through local and national governments, as well as some approaches for engaging non-traditional stakeholders in this space.

OGP Outreach Plan

Upon returning from the Open Government Partnership, I commit to aggregating feedback from various actors in an effort to have it directly inform the goals and actions of the Canadian Federal Government with whom I have a critical but cordial relationship. I will also, of course, blog about what I believe are the key benchmarks Canadian civil society actors should be using to pressure and measure the Canadian government against. Finally, I commit to get on the phone with any civil society actor that contacts me and discuss with them what I observed and how I believe it impacts their organization.

Funding

With regard to funding, my hope is that I will be able to find some alternative funding for travel. As a result, I’m looking to have my room and board covered along with some of my travel costs. My hope is that by doing so, it might be possible to use some of the OGP funds to support the travel of others.

The organizations, names and emails of the leaders endorsing my application

Organisation Leader Email

Thank you for considering my application. Please let me know if you have any questions.

Sincerely,

David Eaves

version française

Cher Partenariat gouvernement ouvert,

Je vous écris pour exprimer mon désir de participer en tant que membre de la société civile locale à la réunion de 2012 du OGP au Brésil.

J’ai été actif au sein du gouvernement ouvert pour les 7 dernières années de travail comme un défenseur, de conseiller et chroniqueur de la transparence au Canada et partout dans le monde.

Contexte et engagement dans les questions du OGP

En tant que défenseur, j’ai parlé des défis autour de la transparence du gouvernement partout au Canada aux niveaux municipal, provincial et fédéral. Au niveau international, je vous ai donné le discours d’ouverture lors des deux dernières International Open Camps de données hébergées par le gouvernement de l’Open Knowledge Foundation, a été invité à s’adresser à la 7e Conférence internationale des commissaires à l’information à Ottawa et du Mexique Semana Nacional de la Transparencia, et donné des conférences au l’Gov 2.0 Expo et au Sommet de Washington DC (à laquelle l’hôte, Tim O’Reilly, a déclaré: «Si vous ne voulez lire un blog dans l’espace Gov 2.0, vous devriez lire eaves.ca”).

En plus de parler, j’ai essayé de démontrer activement des façons un gouvernement ouvert pouvons améliorer nos collectivités. Ma conviction est que nous devons un large éventail de moyens pour engager les citoyens dans un gouvernement ouvert, dont certains seront motivés par la reddition de comptes, mais d’autres seront engagés en ayant simplement leur vie plus facile. Par conséquent, dans la poursuite de l’avancement de responsabilité, j’ai travaillé avec une équipe de développeurs parrainés par Microsoft pour créer Emitter.ca, un site Web qui écrase les données de la pollution, homme politique et la société pour permettre aux citoyens d’identifier les pollueurs dans leurs quartiers et de la région. Tout en essayant de trouver des façons de montrer le degré d’ouverture du gouvernement peut favoriser de meilleurs services, j’ai travaillé avec des amis pour créer Recollect.net, un service simple qui utilise les données ouvertes à rappeler aux citoyens de prendre leur recyclage et des ordures. Et en tant que défenseur, lorsque le gouvernement fédéral retardée par des années derrière les Etats-Unis et au Royaume-Uni dans la création d’un portail de données ouverte, j’ai créé http://www.datadotgc.ca qui a cherché à suivre les données ouvertes fixe d’ores et déjà partagé sur le site Web du ministère, pour montrer que, contrairement à sa position, le gouvernement avait déjà une infrastructure politique de faire de données ouvertes. Ce site a contribué pression sur le gouvernement en lançant son propre site web officiel de données ouvert.

J’ai aussi travaillé activement dans l’espace ouvert par gouvernement collaboration avec les gouvernements directement. En 2009, j’ai co-rédigé la motion ouverte pour le maire et le conseil de la ville de Vancouver. Cela a conduit à la motion du Conseil municipal première dans le monde de diriger le personnel municipal pour faire partie des données ouverte de leurs activités. En conséquence, Vancouver a lancé seconde municipale dans le monde Portail de données ouvertes (après Washington DC). J’ai également travaillé avec TI de la Ville de personnel pour réviser les règles de passation des marchés pour rendre les données ouvertes une spécification requise ainsi rendre les logiciels open source d’une option acceptable. Au niveau fédéral, j’ai travaillé avec des ONG et instruits joueurs clés du gouvernement de transférer au Canada – qui avait été sceptique – en acceptant de participer à l’OGP.

J’ai, cependant, a également joué un rôle crucial lorsque cela est nécessaire. J’ai écrit des pièces dans les journaux et sur mon blog où les gouvernements n’ont pas réussi à être transparents ou ont pris des mesures dans la mauvaise direction.

Comment vais-je contribuer à la réunion et ce, je veux apprendre de fréquenter;

Le Partenariat sur la transparence gouvernementale offre aux membres de la société civile avec une carotte et du bâton rare pour engager leur gouvernement sur la question d’un gouvernement ouvert. Parce qu’il oblige le gouvernement à fixer des objectifs clairs à la transparence dans un forum international, il offre à la société civile avec effet de levier pour obliger le gouvernement à rendre compte.

Bien que cet effet de levier doit être géré de façon responsable (dans les faits critiques incorrectes va éroder la confiance du public dans les organisations de la société civile), si elle est correctement utilisée, elle peut contraindre le gouvernement à agir de façon plus agressive sur la résolution des problèmes dans ce domaine. Ceci est d’une urgence particulière au Canada, où la transparence du gouvernement a été en déclin au cours des dernières décennies. Autrefois considéré comme la coupe d’âge, l’accès du Canada au système d’information est follement hors de date. Demandes d’accès à l’information sont traitées plus lentement que jamais et l’accès à l’information du gouvernement – à l’exception d’une amélioration spectaculaire dans la zone de données ouvertes – est de plus en plus restrictive. Mon objectif au sein du Partenariat sur la transparence gouvernementale sera d’engager les autres gouvernements et les ONG pour comprendre les repères de transparence étant fixés par les gouvernements d’autres qui peuvent être utilisés comme un moyen par lequel les Canadiens peuvent juger l’état d’avancement de leur propre gouvernement. Je peux aussi partager mes propres expériences dans le déplacement des politiques d’ouverture des données par les gouvernements locaux et nationaux, ainsi que quelques approches pour engager les acteurs non traditionnels dans cet espace.

Description de mon OGP plan de sensibilisation avec la société civile et d’autres où je rentre chez moi

Au retour de la société en commandite gouvernement ouvert, je m’engage à agréger les commentaires des différents acteurs dans un effort pour faire informer directement les objectifs et les actions du gouvernement fédéral canadien avec qui j’ai une relation critique, mais cordiale. Je vais aussi, bien sûr, blog sur ce que je crois sont les principaux critères canadiens acteurs de la société civile devraient être en utilisant à la pression et de mesurer le gouvernement canadien contre. Enfin, je m’engage à prendre le téléphone avec n’importe quel acteur de la société civile qui communique avec moi et discuter avec eux ce que j’ai observé et comment je crois que son impact sur leur organisation.

Financement

En ce qui concerne le financement, mon espoir est que je serai en mesure de trouver un financement alternatif pour les voyages. En conséquence, je suis à la recherche d’avoir ma chambre et pension comprise avec certains de mes frais de déplacement. Mon espoir est que, ce faisant, il pourrait être possible d’utiliser une partie des fonds du OGP à financer les déplacements des autres.

Les organisations, les noms et les courriels des dirigeants approuvant ma demande

Organisation Leader Courriel

Merci de considérer ma demande. S’il vous plaît laissez-moi savoir si vous avez des questions.

Sincèrement,

David Eaves

Use The Economist's Data to Find the Best City in the World

1 Reply

Yesterday The Economist Intelligence Unit and Buzzdata launched a $10,000 contest to help enhance The Economist’s “Best city in the world” index.

Yes. It’s a data and visualization competition to identify the best city in the world to live.

As part of the contest, The Economist Intelligence Unit has shared two data sets, its “liveability” and “cost of living” indices for 140 cities around the world. This is, in of itself, pretty cool. But the contest moves beyond their data. As the website outlines, the competition’s core objective is to not just use this data, but figure out what other data sets should be used.

Your mission: to create a new “liveability” index, using the 140 cities in the EIU’s datasets, that determines which is the best city in the world to live in, using these datasets PLUS any additional publicly available data sources that you wish to use (note: see the Contest Rules for information on using additional data). You are also required to create a visualization of the new index that you’ve created.

If you’ve always felt that some important factors in livability and quality of life have not been getting the attention they deserve, now is a chance to change (or add them to!) the debate.

You can check out the rules and judging criteria, as well as sign up, over at the contest’s webpage.

I, sadly, won’t be participating in the competition as… I’m pleased to share that I’ll be helping to judge the contest.

Algorithmic Regulation Spreading Across Government?

3 Replies

I was very, very excited to learn that the City of Vancouver is exploring implementing a program started in San Francisco in which “smart” parking meters adjust their price to reflect supply and demand (story is here in the Vancouver Sun).

For those unfamiliar with the program, here is a breakdown. In San Francisco, the city has the goal of ensuring at least one free parking spot is available on every block in the downtown core. As I learned during the San Fran’s presentation at the Code for America summit, such a goal has several important consequences. Specifically, it reduces the likelihood of people double parking, reduces smog and greenhouse gas emissions as people don’t troll for parking as long and because trolling time is reduced, people searching for parking don’t slow down other traffic and buses as they drive around slowly looking for a spot. In short, it has a very helpful impact on traffic more broadly.

So how does it work? The city’s smart parking meters are networked together and constantly assess how many spots on a given block are free. If, at the end of the week, it turns out that all the spaces are frequently in use, the cost of parking on that block is increased by 25 cents. Conversely if many of the spots were free, the price is reduced by 25 cents. Generally, each block finds an equilibrium point where the cost meets the demand but is also able to adjust in reaction to changing trends.

Technologist Tim O’Reilly has referred to these types of automated systems in the government context as “algorithmic regulation” – a phrase I think could become more popular over the coming decade. As software is deployed into more and more systems, the algorithms will be creating market places and resource allocation systems – in effect regulating us. A little over a year ago I said that contrary to what many open data advocates believe, open data will make data political – e.g. that open data wasn’t going to depoliticize public policy and make it purely evidenced base, quite the opposite, it will make the choices around what data we collect more contested (Canadians, think long form census). The same is also – and already – true of the algorithms, the code, that will increasingly regulate our lives. Code is political.

Personally I think the smart parking meter plan is exciting and hope the city will consider it seriously, but be prepared, I’m confident that much like smart electrical meters, an army of naysayers will emerge who simply don’t want a public resource (roads and parking spaces) to be efficiently used.

It’s like the Spirit of the West said: Everything is so political.

My Canadian Open Government Consultation Submission

2 Replies

Attached below is my submission to the Open Government Consultation conducted by Treasury Board over the last couple of weeks. There appear to be a remarkable number of submission that were made by citizens, which you can explore on the Treasury Board website. In addition, Tracey Lauriault has tracked some of the submissions on her website.

I actually wish the submissions on the Government website were both searchable and could be downloaded in there entirety. That way we could re-organize them, visualize them, search and parse them as well as play with the submissions so as to make the enormous number of answers easier to navigate and read. I can imagine a lot of creative ways people could re-format all that text and make it much more accessible and fun.

Finally, for reference, in addition to my submission I wrote this blog post a couple months ago suggesting goals the government set for itself as part of its Open Government Partnership commitments. Happily, since writing that post, the government has moved on a number of those recommendations.

So, below is my response to the government’s questions (in bold):

What could be done to make it easier for you to find and use government data provided online?

First, I want to recognize that a tremendous amount of work has been done to get the present website and number of data sets up online.

FINDING DATA:

My advice on making data easier to engage Socrata to create the front end. Socrata has an enormous amount of experience in how to share government data effectively. Consider http://data.oregon.gov here is a site that is clean, easy to navigate and offers a number of ways to access and engage the governments data.

More specifically, what works includes:

1. Effective search: a simple search mechanism returns all results
2. Good filters: Because the data is categorized by type (Internal vs. external, charts, maps, calendars, etc…) it is much easier to filter. One thing not seen on Socrata that would be helpful would be the ability to sort by ministry.
3. Preview: Once I choose a data set I’m given a preview of what it looks like, this enables me to assess whether or not it is useful
4. Social: Here there is a ton on offer
– I’m able to sort data sets by popularity – being able to see what others find interesting is, in of itself interesting.
– Being able to easily share data sets via email, or twitter and facebook means I’m more likely to find something interesting because friends will tell me about it
– Data sets can also be commented upon so I can see what others think of the data, if they think it is useful or not, and what for or not.
– Finally, it would be nice if citizens could add meta data, to make it easier for others to do keyword searches. If the government was worried about the wrong meta data being added, one could always offer a search with crowd sourced meta data included or excluded
5. Tools: Finally, there are a large number of tools that make it easier to quickly play with and make use of the data, regardless of one’s skills as a developer. This makes the data much more accessible to the general public.

USING DATA

Finding data is part of the problem, being able to USE the data is a much bigger issue.

Here the single most useful thing would be to offer API’s into government data. My own personal hope is that one day there will be a large number of systems both within and outside of government that will integrate government data right into their applications. For example, as I blogged about here – https://eaves.ca/2011/02/18/sharing-critical-information-with-public-lessons-for-governments/ – product recall data would be fantastic to have as an API so that major retailers could simply query the API every time they scan inventory in a warehouse or at the point of sale, any product that appears on the list could then be automatically removed. Internally, Borders and Customs could also query the API when scanning exports to ensure that nothing exported is recalled.

Second, if companies and non-profits are going to invest in using open data, they need assurances that both they are legally allowed to use the data and that the data isn’t going to suddenly disappear on them. This means, a robust license that is clear about reuse. The government would be wise to adopt the OGL or even improve on it. Better still helping establish a standardized open data license for Canada and ideally internationally could help reduce some legal uncertainty for more conservative actors.

More importantly, and missing from Socrata’s sites, would be a way of identifying data sets on the security of their longevity. For example, data sets that are required by legislation – such as the NPRI – are the least likely to disappear, whereas data sets the the long form census which have no legal protection could be seen as at higher risk.

How would you use or manipulate this data?

I’m already involved in a number of projects that use and share government data. Among those are Emitter.ca – which maps and shares NPRI pollution data and Recollect.net, which shares garbage calendar information.

While I’ve seen dramatically different uses of data, for me personally, I’m interested mostly in using data for thinking and writing about public policy issues. Indeed, much has been made of the use of data in “apps” but I think it is worth noting that the single biggest use of data will be in analysis – government officials, citizens, academics and others using the data to better understand the world around them and lobby for change.

This all said, there are some data sets that are of particular usefulness to people, these include:

1. Data sets on sensitive issues, this includes health, inspection and performance data (Say surgery outcomes for specific hospitals, or restaurant inspection data, crime and procurement data are often in great demand).
2. Dynamic real-time Data: Data that is frequently updated (such a border, passport renewal or emergency room wait times). This data is shared in the right way can often help people adjust schedules and plans or reallocate resources more effectively. Obviously this requires an API.
3.Geodata: Because GIS standards are very mature it is easy to “mashup” geo data to create new maps or offer new services. These common standards means that geo data from different sources will work together or can be easily compared. This is in sharp contrast to say budget data, where there are few common standards around naming and organizing the data, making it harder to share and compare.

What could be done to make it easier for you to find government information online?

It is absolutely essential that all government records be machine readable.

Some of the most deplorable moment in open government occur when the government shares documents with the press, citizens or parliamentary officers in paper form. The first and most important thing to make government information easier to find online is to ensure that it is machine readable and searchable by words. If it does not meet this criteria I increasingly question whether or not it can be declared open.

As part of the Open Government Partnership commitments it would be great for the government to commit to guarantee that every request for information made of it would include a digital version of the document that can be searched.

Second, the government should commit that every document it publishes be available online. For example, I remember in 2009 being told that if I wanted a copy of the Health Canada report “Human Health in a Changing Climate:A Canadian Assessment of Vulnerabilities and Adaptive Capacity” I had to request of CD, which was then mailed to me which had a PDF copy of the report on it. Why was the report not simply available for download? Because the Minister had ordered it not to appear on the website. Instead, I as a taxpayer and to see more of my tax dollars wasted for someone to receive my mail, process it, then mail me a custom printed cd. Enabling ministers to create barriers to access government information, simply because they do not like the contents, is an affront to the use of tax payer dollars and our right to access information.

Finally, Allow Government Scientists to speak directly to the media about their research.

It has become a reoccurring embarrassment. Scientists who work for Canada publish an internationally recognized ground break paper that provides some insight about the environment or geography of Canada and journalists must talk to government scientists from other countries in order to get the details. Why? Because the Canadian government blocks access. Canadians have a right to hear the perspectives of scientists their tax dollars paid for – and enjoy the opportunity to get as well informed as the government on these issues.

Thus, lift the ban that blocks government scientists from speaking with the media.

Do you have suggestions on how the Government of Canada could improve how it consults with Canadians?

1. Honour Consultation Processes that have started

The process of public consultation is insulted when the government itself intervenes to bring the process into disrepute. The first thing the government could do to improve how it consults is not sabotage processes that already ongoing. The recent letter from Natural Resources Minister Joe Oliver regarding the public consultation on the Northern Gateway Pipelines has damaged Canadians confidence in the governments willingness to engage in and make effective use of public consultations.

2. Focus on collecting and sharing relevant data

It would be excellent if the government shared relevant data from its data portal on the public consultation webpage. For example, in the United States, the government shares a data set with the number and location of spills generated by Enbridge pipelines, similar data for Canada would be ideal to share on a consultation. Also useful would be economic figures, job figures for the impacted regions, perhaps also data from nearby parks (visitations, acres of land, kml/shape boundary files). Indeed, data about the pipeline route itself that could be downloaded and viewed in Google earth would be interesting. In short, there are all sorts of ways in which open data could help power public consultations.

3. Consultations should be ongoing

It would be great to see a 311 like application for the federal government. Something that when loaded up, would use GPS to identify the services, infrastructure or other resources near the user that is operated by the federal government and allow the user to give feedback right then and there. Such “ongoing” public feedback could then be used as data when a formal public consultation process is kicked off.

Are there approaches used by other governments that you believe the Government of Canada could/should model?

1. The UK governments expense disclosure and release of the COINS database more generally is probably the most radical act of government transparency to date. Given the government’s interest in budget cuts this is one area that might be of great interest to pursue.

2. For critical data sets, those that are either required by legislation or essential to the operation of a ministry or the government generally, it would be best to model the city of Chicago or Washington DC and foster the creation of a data warehouse where this data could be easily shared both internally and externally (as privacy and security permits). These cities are leading governments in this space because they have tackled both the technical challenges (getting the data on a platform where it can be shared easily) and around governance (tackling the problem of managing data sets from various departments on a shared piece of infrastructure).

Are there any other comments or suggestions you would like to make pertaining to the Government of Canada’s Open Government initiative?

Some additional ideas:

Redefine Public as Digital: Pass an Online Information Act

a) Any document it produces should be available digitally, in a machine readable format. The sham that the government can produce 3000-10,000 printed pages about Afghan detainees or the F-35 and claim it is publicly disclosing information must end.

b) Any data collected for legislative reasons must be made available – in machine readable formats – via a government open data portal.

c) Any information that is ATIPable must be made available in a digital format. And that any excess costs of generating that information can be born by the requester, up until a certain date (say 2015) at which point the excess costs will be born by the ministry responsible. There is no reason why, in a digital world, there should be any cost to extracting information – indeed, I fear a world where the government can’t cheaply locate and copy its own information for an ATIP request as it would suggest it can’t get that information for its own operations.

Use Open Data to drive efficiency in Government Services: Require the provinces to share health data – particularly hospital performance – as part of its next funding agreement within the Canada Health Act.

Comparing hospitals to one another is always a difficult task, and open data is not a panacea. However, more data about hospitals is rarely harmful and there are a number of issues on which it would be downright beneficial. The most obvious of these would be deaths caused by infection. The number of deaths that occur due to infections in Canadian hospitals is a growing problem (sigh, if only open data could help ban the antibacterial wipes that are helping propagate them). Having open data that allows for league tables to show the scope and location of the problem will likely cause many hospitals to rethink processes and, I suspect, save lives.

Open data can supply some of the competitive pressure that is often lacking in a public healthcare system. It could also better educate Canadians about their options within that system, as well as make them more aware of its benefits.

Reduce Fraud: Creating a Death List

In an era where online identity is a problem it is surprising to me that I’m unable to locate a database of expired social insurance numbers. Being able to query a list of social security numbers that belong to dead people might be a simple way to prevent fraud. Interestingly, the United States has just such a list available for free online. (Side fact: Known as the Social Security Death Index this database is also beloved by genealogist who use it to trace ancestry).

Open Budget and Actual Spending Data

For almost a year the UK government has published all spending data, month by month, for each government ministry (down to the £500 in some, £25,000 in others). More over, as an increasing number of local governments are required to share their spending data it has lead to savings, as government begin to learn what other ministries and governments are paying for similar services.

Create a steering group of leading Provincial and Municipal CIOs to create common schema for core data about the country.

While open data is good, open data organized the same way for different departments and provinces is even better. When data is organized the same way it makes it easier to citizens to compare one jurisdiction against another, and for software solutions and online services to emerge that use that data to enhance the lives of Canadians. The Federal Government should use its convening authority to bring together some of the countries leading government CIOs to establish common data schemas for things like crime, healthcare, procurement, and budget data. The list of what could be worked on is virtually endless, but those four areas all represent data sets that are frequently requested, so might make for a good starting point.

Open Data in BC – Good & Bad Examples from Bikes to Libraries

2 Replies

Some small examples of open data use and public servants who do and don’t understand open data from the Province of British Columbia to the City of Vancouver.

Open Libraries?

For the past several years – ever since the open motion was passed in Vancouver – the city has been releasing more and more data sets. One data set I’ve encouraged them to proactively release was library data – the catalog, what books were popular, etc… Others have made the request and, in fact, some of the catalog data is available, if you know where to look – but it isn’t licensed. This hasn’t stopped people from creating cool things – like this awesome Firefox greasemonkey script that shows if a book you are looking at on Amazon’s site is available at your local VPL library – but it has driven these innovations underground, discouraged them, and made them difficult to maintain.

I’ve even had meetings with Vancouver Public Library (VPL) officials who ranged from deeply opposed to indifferent about sharing their data, usually on the grounds of privacy and security. How releasing the libraries catalog, or offering an API into the catalog or showing the number of times a book has been checked out threatens privacy is beyond me. Mostly I suspect it is driven by the fact that they don’t want anything competing with their website and software – pretty much the opposite approach to innovation than that taken by the leading cities and governments.

The reluctance of VPL to share its data given they are a) a community supported library and b) that City Council passed a motion explicitly directing city staff to make their data open, is all the more surprising (I mean even ICBC gave me bike accident data). This is why I was excited to see that the Provincial Government of British Columbian has taken the opposite view. Recently they released location and statistic for Public Libraries across BC for 2006-2009. It does not sadly, include the collections data or the number of check outs for each book (which would of course be awesome but it does provide lat/longs for every library and a great deal of data on each library system and sometimes individual branch such as staff levels, budget data and usage counts (again not by resource). It’s a good start and something I hope people will want to play with. Of course, getting an API into the actual catalog is the real idea – the things my friends talk about doing to enable them and their kids to better use the library…

Speaking of playing…

Bike Accident Data Keeps Generating Discussion

It is wonderful to see that blog posts and analysis as a result of Eric Promislow’s BC bike accident map continue to emerge. Eric created his map during the December 3rd Open Data Hackathon when he visualized bike accident data I managed to get from Insurance Company of British Columbia and uploaded it to Buzzdata. (Eric subsequently got automobile accident data and mapped that too). Another example appeared last week, when the map and data proved useful to Stephen Wehner who used it in a recent blog post to supplement some anecdotal data around accidents in his neighborhood.

It’s a wonderful example of how local citizens can begin to see the risks and problems in their neighborhoods, and arm themselves with real data when they want to complain to their councilperson, MLA, MP or other representative.

Solving the Common Standards problem in the Open Data Space

4 Replies

Last year during my Open Government Data Camp keynote speech on The State of Open Data 2011 I mentioned how I thought the central challenge for open data was shifting from getting data open (still a big issue, but a battle that is starting to be won) to getting all that open data in some common standards and schemas so that use (be it apps, analysis and other uses) can be scaled across jurisdictions.

Looks like someone out there is trying to turn that challenge in to a business opportunity.

Listpoint, a UK based company has launched a platform with the goal of creating translators between various established specs. As they point out in an email I saw from them:

“The Listpoint reference data management platform is a repository for data standards in the shape of code lists. Listpoint will help interpret open data by providing its underlying metadata and schema in machine readable format. E.g. mapping ISO country codes and Microsoft Country codes to provide a translation layer for systems to surface a single view of data.”

Interesting stuff… and exactly the types of challenges we need solved if we are going to scale the opendata revolution.

Open Government Advocacy: The Danger of Letting Narrative Trump Fact

3 Replies

So I loath making this the first post of the new year, but here we go.

Today Canada.com published a story “Tony Clement vows innovative new open government, but critics point to poor record.” In it, Jason Fekete the journalist responsible for the story, quotes a Democracy Watch spokesperson who sadly gets the facts completely wrong despite the fact that I warned Democracy Watch about their error a month ago after their press release caused similar errors to appear in a CBC story. I’ll outline why this is problem later in the post. Bur first the error.

In the article Fekete reports

Democracy Watch said it will appeal to the international open government group to reject Canada’s entry because the federal government failed to keep one of its required commitments to consult Canadians. Ottawa announced its online consultation one day after the watchdog complained about it.

This is, in fact, not true. To date, the government has not failed to meet its commitment. As I pointed out in an earlier blog post (to which Democracy Watch responded as is aware) Democracy Watch accuses the government of failing because it believed consultations needed to be conducted before a November OGP meeting in Brazil. Unfortunately, the meeting in which Governments will be sharing their plans (and thus need to complete their consultations) will be taking place in Brazil in April. The OPG clearly states this on their website (under section 5). There was a meeting in November, so the confusion was understandable.

Of course, just to be safe, I did what the CBC and Postmedia should have done. I emailed the OGP secretariat to check. Within minutes they confirmed to me that the April meeting is the deadline for consultations and developing plans. What is more interesting to me is the no one from Democracy Watch, the CBC or Post Media bothered to simply email or call the OGP secretariat and confirm these facts. For the CBC and Postmedia this is a matter of laziness. For Democracy Watch, I’m not sure what is driving this willed blindness. Ultimately, I suspect that once they went public with their narrative, backing down would be seen as weakness and then government secrecy would win!

This is dangerous for two reasons.

The first is, it isn’t true. Government secrecy doesn’t win if Democracy Watch got its facts mixed up. I agree that this government has a lot to answer for around its willingness to disclose government documents. Be it documents around the procurement of the F-35 or the treatment of Afghan prisoners there are many cases where the lack of transparency has been blatant and, I believe, counter to the principles of democracy and open government. Conceding that the Government is still on track for its Open Government Partnership objectives does not diminish that fact. The only thing that misrepresenting the facts does is cause conservative leaning voters who believe in government transparency (an important constituency) to tune out of the debate and believe that Democracy Watch is simply out to score points against the government, not fulfill its mission.

The bigger reason I think it is dangerous is that it undermines the very thing that makes the Open Government Partnership an effective tool of open government advocates. I want to be clear. The Open Government Partnership is, in part, designed to empower advocates and help them compel their government’s to be more open. Used correctly it could be powerful. The fact that Canadian government signed on to the International Aid Transparency Initiative (IATI) and the Extractive Industries Transparency Initiative is a result, in part, of the fact that other OGP countries were signing on. We were able to use peer pressure to create an upward spiral. We can also use the timelines of the OGP to ensure the government carries out the pledges that it has made. And of course, there is an Open Government consultation that is currently taking place (please participate) that the government will have to share the results of with its partners – potentially giving us a way to verify that our input is being taken seriously. Indeed, participating OPG countries may even try to out do one another to demonstrate how seriously they are taking this input.

But when this tool is misused it gives the government license to ignore and write off critics. As someone who wants to use the OGP commitments as a carrot and stick to hold our government to account, stories like those I linked to above hurt our capacity to be effective.

This government does not have a great record of transparency. At the same time, there is a legitimate effort to create open government goals as part of the OGP, let’s let the process run its course (and criticize when they actually violate the process) and use the tools that are at our disposal constructively to maximize impact, rather than try to snare a quick headline that in the long term, could damage the open government movements credibility.

I certainly wouldn’t encourage Democracy Watch to petition the OGP to ask Canada to leave the partnership. I suspect the secretariat would be confused by the request. The deadline has not passed, indeed, most OGP countries are in the middle of their consultations right now.

The Future of Academic Research

12 Replies

Yesterday, Nature – one of the worlds premier scientific journals recognized University of British Columbia scientist Rosie Redfield as one of the top 10 science newsmakers of 2011.

The reason?

After posting a scathing attack on her blog about a paper that appeared in the journal Science, Redfield decided to attempt to recreate the experiment and has been blogging about her effort over the past year. As Nature describes it:

…that month, Redfield took matters into her own hands: she began attempting to replicate the work in her lab at the University of British Columbia in Vancouver, and documenting her progress on her blog (http://rrresearch.fieldofscience.com).

The result has been a fascinating story of open science unfolding over the year. Redfield’s blog has become a virtual lab meeting, in which scientists from around the world help to troubleshoot her attempts to grow and study the GFAJ-1 bacteria — the strain isolated by Felisa Wolfe-Simon, lead author of the Science paper and a microbiologist who worked in the lab of Ronald Oremland at the US Geological Survey in Menlo Park, California.

While I’m excited about Redfields blog (more on that below) we should pause and note the above paragraph is a very, very sad reminder of the state of affairs in science. I find the term “open science” to be an oxymoron. The scientific process only works when it is, by definition, open. There is, quite arguably, no such thing as “closed science.” And yet it is a reflection of how 18th century the entire science apparatus remains that Redfields awesome experiment is just that – an experiment. We should celebrate her work, and ask ourselves, why is this not the norm?

So first, to celebrate her work… when I look at Redfields blog, I see exactly what I hope the future of scientific, and indeed all academic research, will look like. Here is someone who is constantly updating their results and sharing what they are doing with their peers, as well as getting input and feedback from colleagues and others around the world. Moreover, she plays to the mediums strengths. While rigorous, she remains inviting and, from my reading, creates a more honest and human view into the world of science. I suspect that this might be much more attractive (and inspiring) to potential scientists. Consider, these two lines from one of her recent posts:

So I’m pretty sure I screwed something up. But what? I used the same DNA stock tube I’ve used many times before, and I definitely remember putting 3 µl of DNA into each assay tube. I made fresh sBHI + novobiocin plates using pre-made BHI agar,, and I definitely remember adding the hemin (4 ml), NAD (80 µl) and novobiocin (40 µl) to the melted agar before I poured the plates.

and

UPDATE: My novobiocin plates had no NovR colonies because I had forgotten to add the required hemin supplement to the agar! How embarrassing – I haven’t made that mistake in years.

and then this blog post title:

Some control results! (Don’t get excited, it’s just a control…)

Here is someone literally walking through their thought processes in a thorough, readable way. Can you imagine anything more helpful for a student or young scientist? And the posts! Wonderfully detailed walk throughs of what has been tried, progress made and set backs uncovered. And what about the candor! The admission of error and the attempts to figure out what went wrong. It’s the type of thinking I see from great hackers as well. It’s also the type of dialogue and discussion you won’t see in a formal academic paper but is exactly what I believe every field (from science, to non-profit, to business) needs more of.

Reading it all, and I’m once again left wondering. Why is this the experiment? Why isn’t this the norm? Particularly at publicly funded universities?

Of course, the answer lies in another question, one I first ran into over a year ago reading this great blog post by Michael Clarke on Why Hasn’t Scientific Publishing Been Disrupted Already? As he so rightly points out:

When Tim Berners-Lee created the Web in 1991, it was with the aim of better facilitating scientific communication and the dissemination of scientific research. Put another way, the Web was designed to disrupt scientific publishing. It was not designed to disrupt bookstores, telecommunications, matchmaking services, newspapers, pornography, stock trading, music distribution, or a great many other industries…

…The one thing that one could have reasonably predicted in 1991, however, was that scientific communication—and the publishing industry that supports the dissemination of scientific research—would radically change over the next couple decades.

And yet it has not.

(Go read the whole article, it is great). Mathew Ingram also has a great piece on this published half a year later called So when does academic publishing get disrupted?

Clarke has a great breakdown on all of this, but my own opinion is that scientific journals survive not because they are an efficient means of transmitting knowledge (they are not – Redfield’s blog shows there are much, much faster ways to spread knowledge). Rather journals survive in their current form because they are the only rating system scientists (and more importantly) universities have to deduce effectiveness, and thus who should get hired, fired, promoted and, most importantly, funded. Indeed, I suspect journals actually impede (and definitely slow) scientific progress. In order to get published scientists regularly hold back sharing and disclosing discoveries and, more often still, data, until they can shape it in such a way that a leading journal will accept it. Indeed, try to get any scientists to publish their data in machine readable formats – even after they have published with it -it’s almost impossible… (notice there are no data catalogs on any major scientific journals websites…) The dirty secret is that this is because they don’t want others using it in case it contains some juicy insight they have so far missed.

Don’t believe me? Just consider this New York Times article on the break throughs in Alzheimer’s. The whole article is about a big break through in scientific research process. What was it? That the scientists agreed they would share their data:

The key to the Alzheimer’s project was an agreement as ambitious as its goal: not just to raise money, not just to do research on a vast scale, but also to share all the data, making every single finding public immediately, available to anyone with a computer anywhere in the world.

This is unprecedented? This is the state of science today? In an era where we could share everything, we opt to share as little as possible. This is the destructive side of the scientific publishing process that is linked to performance.

It is also the sad reason why it is a veteran, established researcher closer to the end of her career that is blogging this way and not a young, up and coming researcher trying to establish herself and get tenure. This type of blog is too risky to ones career. Today “open” science, is not a path forward. It actually hurts you in a system that prefers more inefficient methods at spreading insights, research and data, but is good at creating readily understood rankings.

I’m thrilled that Rosie Redfield has been recognized by Nature (which clearly enjoys the swipe at Science – its competitor). I’m just sad that the today’s culture of science and universities means there aren’t more like her.

Bonus material: If you want to read an opposite view, here is a seriously self-interested defensive of the scientific publishing industry that was totally stunning to read. It’s fascinating that this man and Michael Clarke share the same server. If you look in the comments of that post, there is a link to this excellent post by a researcher at a University in Cardiff that I think is a great counter point.

Why is Finding a Post Box so Hard?

9 Replies

Sometimes it is the small things that show how government just gets it all so wrong.

Last Thursday The Daily Show’s Wyatt Cenac has a little bit on the US Post Office and its declining fortunes as people move away from mail. There is no doubt that the post offices days are numbered, but that doesn’t mean the decline has to be as steep as it is. Besides there are things they could be doing to make life a little easier to use them (and god knows they should be doing anything they can, to be more appealing).

Take, for example, the humble post office box. They can be frustratingly hard to locate. Consider Broadway and Cambie – one of the busiest intersections in Vancouver – and yet there is no post box at the intersection. (I eventually found it one block east on broadway) but I carried around a letter for 3 weeks before I eventually found one.

In short why is there not digital map (or for techies, and API) for post box locations? I could imagine all sorts of people that might make use of it. Would it be nice to just find out – where is the closest post box to where I’m standing? More importantly, it might actually help the post office attract a few extra customers. It certainly wouldn’t hurt customer service. I’ve wondered for a couple of years why it doesn’t publish this data set.

Turns out I’m not the only with this frustration. My friend Steven Tannock has channeled his frustration into a simple app called Wherepost.ca. It’s a simple website – optimized for mobile phone use – that allows users to add post boxes as well as find the one nearest to them. In short, Steven’s trying to create a public data set of post box locations by crowd sourcing the problem. If Canada Post won’t be helpful… we’ll help one another.

Launched on Thursday with 20 post office box locations, there are now over 400 boxes mapped (mostly in the Vancouver area) with several dozen users contributing. In addition, Steven tells me users in at least 2 other countries have asked for new icons so they can add post boxes where they live. It seems Canadians aren’t the only ones frustrated about not knowing where the nearest post box is.

The ideal, of course, would be for Canada Post to publish an API of all post box locations. I suspect however, that they either don’t actually know where they all are in a digital form (at which point they should really help Steven as he is doing them a huge service) or revealing their location will be seeing as sacrificing some important IP that people should pay for. Remember, this is an organization that refuses to make Postal Code data open, a critical data set for companies, non-profits and governments.

This isn’t the worlds fanciest app but its simplicity is what makes it so great, and so useful. Check it out at WherePost.ca and… of course, add a post box if you see one.

Open Government Consultation, Twitter Townhalls & Doing Advocacy Wrong

3 Replies

Earlier this week the Canadian Federal Government launched its consultation process on Open Government. This is an opportunity for citizens to comment and make suggestions around what data the federal government should make open and what information it should share, and provide feedback on how it can consult more effectively with Canadians. The survey (which, handily, can be saved midway through completion) contains a few straightforward multiple choice questions and about eight open ended questions which I’ve appended to the end of this post so that readers can reflect upon them before starting to fill out the form.

In addition to the online consultations, Tony Clement – the Minister responsible for the Open Government file – will host a Twitter townhall on Open Government this Thursday (December 15). Note! The townhall will be hosted by the treasury board twitter account @TBS_Canada (English) and @SCT_Canada (French) not Minister Clement’s personal (and better known) twitter account. The townhall will first take place in French from 4-4:45pm EST using the hashtags #parlonsgouvert and then in English from 5-5:45 EST using the hashtag #opengovchat.

Some of you may have also noticed that Democracy Watch issued a strongly worded press release last week with the (somewhat long) headline “Federal Conservatives break all of their international Open Government Partnership commitments by failing to consult with Canadians about their draft action plan before meeting in Brazil this week.” This seems to have prompted the CBC to write this article.

Now, to be clear, I’m a strong advocate for Open Government, and there are plenty of things for which one could be critical about this government for not being open about. However, to be credible – especially around issues of transparency and disclosure – one must be factual. And Democracy Watch did more than just stretch the truth. The simple fact is, that while I too wish the government’s consultations had happened sooner, this does not mean it has broken all of its Open Government Partnership commitments. Indeed, it hasn’t broken any of its commitments. A careful read of the Open Government Partnership requirements would reveal that the recent December meeting was to share drafts plans (including the plans by which to consult). The deadline that Democracy Watch is screaming about does not occur until March of 2012.

It would have been fair to say the government has been slow in fulfilling its commitments, but to say it has broken any of them is flatly not true. Indeed the charge feels particularly odd given that in the past two weeks the government signed on greater aid transparency via IATI and released an additional 4000 data sets, including virtually all of StatsCan’s data, giving Canadian citizens, non profits, other levels of governments and companies access to important data sets relevant for social, economic and academic purposes.

Again, there are plenty of things one could talk about when it comes to transparency and the government. Yes, the consultation could have gotten off the ground faster. And yes, there is much more to done. But this screaming headline is somewhat off base. Publishing it damages both the credibility of the organization making the charge, and risk hurting the credible of open government advocates in general.

List of Open Ended Questions in the Open Government Consultation.

1. What could be done to make it easier for you to find and use government data provided online?

2. What types of open data sets would be of interest to you? Please pick up to three categories below and specify what data would be of interest to you.

3. How would you use or manipulate this data?

4. What could be done to make it easier for you to find government information online?

7. Do you have suggestions on how the Government of Canada could improve how it consults with Canadians?

8. Are there approaches used by other governments that you believe the Government of Canada could/should model?

9. Are there any other comments or suggestions you would like to make pertaining to the Government of Canada’s Open Government initiative?