Tag Archives: data

Canada ranks last in freedom of information

3 Replies

For those who missed it over the weekend it turns out Canada ranks last in freedom of information study that looked at the world’s western Parliamentary democracies. What makes it all the more astounding is that a decade ago Canada was considered a leader.

Consider two from the Information Commissioner, Suzanne Legault quotes pulled from the piece:

Only about 16 per cent of the 35,000 requests filed last year resulted in the full disclosure of information, compared with 40 per cent a decade ago, she noted.

And delays in the release of records continue to grow, with just 56 per cent of requests completed in the legislated 30-day period last year, compared with almost 70 per cent at the start of the decade.

These are appalling numbers.

The sad thing is… don’t expect things to get better. Why?

Firstly, the current government seems completely uninterested in access to information, transparency and proactive disclosure, despite these being core planks of its election platform and core values of the reform movement that re-launched Canadian conservatism. Indeed, reforming and improving access to information is the only unfulfilled original campaign promise of the Conservatives – and there appears to be no interest in touching it. Quite the opposite – that political staff now intervene to block and restrict Access to Information Requests – contravening the legislation and policy – is now a well known and documented fact.

Second, this issue is of secondary importance to the public. While everyone will say they care about access to information and open government, then number of people (while growing) still remains small. These types of reports and issues are of secondary importance. This isn’t to say they don’t matter. They do – but generally after something bigger and nastier has come to light and the public begins to smell rot. Then studies like this become the type of thing that hurts a government – it gives legitimacy and language to a sentiment people widely feel.

Third, the public seems confused about who they distrust more – the fact is, however bad the current government is on this issue, the Liberal brand is still badly tarnished on this issue of transparent government due to the scandals from almost a decade ago. Sadly, this means that there will be less burden on this government to act since – every time the issue of transparency and open government arise – rather than act, Government leaders simply point out the other parties failings.

So as the world moves on while Canada remains stuck, its government becoming more opaque, distant and less accountable to the people that elect it.

Interestingly , this also has a real cost to Canada’s influence in the world. It means something when the world turns to you as an expert – as we once were on access to information – minister’s are consulted by other world leaders, your public servants are given access to information loops they might otherwise not know about, there is a general respect, a soft power, that comes from being an acknowledged leader. Today, this is gone.

Indeed, it is worth noting that of the countries survey in the above mentioned study, only Canada and Ireland do not have open data portals which allow for proactive disclosure.

It’s a sign of the times.

Launching Emitter.ca: Open Data, Pollution and Your Community

11 Replies

This week, I’m pleased to announce the beta launch of Emitter.ca – a website for locating, exploring and assessing pollution in your community.

Why Emitter?

A few weeks ago, Nik Garkusha, Microsoft’s Open Source Strategy Lead and an open data advocate asked me: “are there any cool apps you could imagine developing using Canadian federal government open data?”

Having looked over the slim pickings of open federal data sets – most of which I saw while linking to them datadotgc.ca – I remembered one: Environment Canada’s National Pollutant Release Inventory (NPRI) that had real potential.

Emitter-screen-shot

With NPRI I felt we could build an application that allowed people and communities to more clearly see who is polluting, and how much, in their communities could be quite powerful. A 220 chemicals that NPRI tracks isn’t, on its own, a helpful or useful to most Canadians.

We agreed to do something and set for ourselves three goals:

  1. Create a powerful demonstration of how Canadian Federal open data can be used
  2. Develop an application that makes data accessible and engaging to everyday Canadians and provides communities with a tool to better  understand their immediate region or city
  3. Be open

With the help of a crew of volunteers with knew and who joined us along the way – Matthew Dance (Edmonton), Aaron McGowan (London, ON), Barranger Ridler (Toronto) and Mark Arteaga (Oakville) – Emitter began to come together.

Why a Beta?

For a few reasons.

  1. There are still bugs, we’d love to hear about them. Let us know.
  2. We’d like to refine our methodology. It would be great to have a methodology that was more sensitive to chemical types, combinations and other factors… Indeed, I know Matt would love to work with ENGOs or academics who might be able to help provide us with better score cards that can helps Canadians understand what the pollution near them means.
  3. More features – I’d love to be able to include more datasets… like data on where tumours or asthama rates or even employment rates.
  4. I’d LOVE to do mobile, to be able to show pollution data on a mobile app and even in using augmented reality.
  5. Trends… once we get 2009 and/or earlier data we could begin to show trends in pollution rates by facility
  6. plus much, much more…

Build on our work

Finally, we have made everything we’ve done open, our methodology is transparent, and anyone can access the data we used through an API that we share. Also, you can learn more about Emitter and how it came to be reading blog posts by the various developers involved.

Thank yous

Obviously the amazing group of people who made Emitter possible deserve an enormous thank you. I’d also like to thank the Open Lab at Microsoft Canada for contributing the resources that made this possible. We should also thank those who allowed us to build on their work, including Cory Horner’s Howdtheyvote.ca API for Electoral District boundaries we were able to use (why Elections Canada doesn’t offer this is beyond me and, frankly, is an embarrassment). Finally, it is important to acknowledge and thank the good people at Environment Canada who not only collected this data, but have the foresight and wisdom to share make it open. I hope we’ll see more of this.

In Sum

Will Emitter change the world? It’s hard to imagine. But hopefully it is a powerful example of what can happen when governments make their data open. That people will take that data and make it accessible in new and engaging ways.

I hope you’ll give it a spin and I look forward to sharing new features as they come out.

Update!

Since Yesterday Emitter.ca has picked up some media. Here are some of the links so far…

Hanneke Brooymans of the Edmonton Journal wrote this piece which was in turn picked up by the Ottawa Citizen, Calgary Herald, Canada.com, Leader Post, The Province, Times Columnist and Windsor Star.

Nestor Arellano of ITBusiness.ca wrote this piece

Burke Campbell, a freelance writer, wrote this piece on his site.

Kate Dubinski of the London Free Press writes a piece titled It’s Easy to Dig up Dirt Online about emitter.ca

Lots of great reading

3 Replies

So with summer having now sped by I haven’t done a reading update in quite some time… here’s a quickie:

1. The Ascent of Money by Niall Ferguson

For a subject that sounds like it should completely bore you – the history of finance – this book is brilliant and, frankly, fun. It’s also timely. The financial system is so old that we often forget that it actually emerged out of something. Money, bonds, stocks, all that good stuff, it hasn’t always been around. WE of course know this, but it is great to actually be walked through how it all emerged, especially when its so wonderful told. It’s also nice to take a look at an old system, like finance, which we are now as comfortable with as the air we breath (even, when at times, it turns toxic and crashes our economy) as so much of my time is spent looking at relatively newer systems – digital networks and the internet. Lots of lessons could be drawn, especially around trust networks (something here for Shirky while he’s at Berkman?).

One additional point. I initially started watching the PBS series by the same name which is based on the book and also hosted by Niall Ferguson but was not really riveted by it. It was somewhat slow moving and lacked the historical depth and arc the book has… so if you saw the TV documentaries and were turned off, not to worry, the book is definitely working picking up.

2. How Government HR Processes are Broken

Check out this fantastic post by an anonymous public servant in Gatineau. It’s a deadly piece about how broken hiring practices are in government and how it’s unsurprising some would be driven away. It would make you laugh if it weren’t making you cry. Got this from a public servant, then after tweeting it, a bunch more noted to me how painfully true it felt to them. Sigh.

3. David McCandless: The Beauty of Data Visualization

It’s hard(er) to do visualizations without open data. Here’s some beautiful ones from England.

We are going to make a better world, nudging and learning

4. An interview with David Mahfouda and Alex Pasternack, creators of a new app for booking/sharing rides in New York

A fantastic interview about how we can share resources to get around more quickly, cheaply and efficiently by using technology. A riff off of Robin’s Chases’ ZipCar idea but its not about sharing cars, but sharing rides. This is the future of urban transportation. That is, of course, if Ontario’s bus companies don’t try to outlaw it.

The Challenge of Open Data and Metrics

13 Replies

One promise of open data is its ability to inform citizens and consumers about the quality of local services. At the Gov 2.0 Summit yesterday the US Department of Health and Human Resources announced it was releasing data on hospitals, nursing homes and clinics in the hopes that developers will create applications that show citizens and consumers how their local hospitals stacks up against others. In short, how good, or even how safe, is their local hospital?

In Canada we already have some experience with this type of measuring. The Fraser Institute publishes an annual report card of schools performance in Alberta, BC, Ontario and Washington. (For those unfamiliar with the Fraser Institute it is a right-wing think tank based in Vancouver with, shall we say, dubious research credentials but strong ideological and fundraising goals.

Perhaps unsurprisingly, private schools do rather well in the Fraser Institute’s report card. Indeed it would appear (and I may be off by one here) that the t0p 18 schools on the list are all private. This does support a narrative that private schools are inherently better than state run schools that would be consistent with the Fraser Institute’s outlook. But, of course, that would be a difficult conclusion to sustain. Private schools tend to be populated with kids from wealthy families with better educated parents and have been given a blessed head start in life. Also, and not noted in the report card, is that many private schools are comfortable turfing out under-performing or unruly students. This means that the “delayed advancement rate,” one critical metric of a schools performance, is dramatically less impacted than a public school that cannot as easily send students packing.

Indeed, the Fraser Institute’s report card is rife with problems, something that teachers unions and, say,  equally ideological but left-oriented think tanks like the Centre for Policy Alternatives are all too happy to point out.

While I loath the Fraser Institute’s simplistic report card and think it is of dubious value to parents I do like that they are at least trying to give parents some tool by which to measure schools. The notion that schools, teachers and education quality can’t be measured, or are too complicated to measure is untenable. I suspect few parent – especially those in say, jobs where they are evaluated – believe it. Nor does such a position help parents assess the quality of education their child is receiving. While they understand, may be sympathetic to or even agree that this is a complicated issue it seems clear based on the success of Ontario’s school locator that many parents want and like these tools.

Ultimately the problem here isn’t the open data (despite what critics of the Ontario Government’s school comparison website would have you believe). Besides, are we now going to hide or suppress data so that parents can’t assess their kids schools? Nor is the problem school report cards per se. If anything is the problem it is that the Fraser Institute has had the field all to itself to play in. If teachers groups, other think tanks, or any other group believes that the Fraser Institute’s report cards are not too crude, why not design a better one? The data is available (and the government could easily be pressured to make more of it available). Why don’t teacher’s groups share with parents the metrics by which they believe parents should evaluate and compare schools? What this issue could use is some healthy competition and debate – one that generated more options and tools for parents.

The challenge for government is to make data more easily available. By making educational data more accessible, less time, IT skills and energy is needed to organize the data and precious resources can instead be focused on developing and visualizing the scoring methodology. This is certainly seems to be Health and Human Services approach: lower transaction costs, galvanize a variety of assessment applications and foster a healthy debate. It would be nice if ministries of education in Canada took a similar view.

But the second half of that challenge is also important, and groups outside of government need to recognize they can have a role, and the consequence of not participating. The mistake is to ask how to deal with groups like the Fraser Institute that use crude metrics, instead we need to encourage more groups and encourage our own organizations to contribute to the debate, to give it more nuance, and create better tools. Leaving the field to the Fraser Institute is a dangerous strategy, one that will serve few people. This is even more the case since in the future we are likely to have more, not less data about education, health and a myriad of other services and programs.

So, the challenge for readers is – will your organization participate?


How Science Is Rediscovering "Open" And What It Means For Government

4 Replies

Pretty much everybody in government should read this fantastic New York Times article Sharing of Data Leads to Progress on Alzheimer’s. On one hand the article is a window into what has gone wrong with science – about how all to frequently a process that used to be competitive but open, and problem focused has become a competitive but closed and intellectual property driven (one need only look at scientific journals to see how slow and challenging the process has become).

But strip away the talk about the challenges and opportunities for science. At its core, this is an article is about something more basic and universal. This is an article about open data.

Viewed through this lens it is a powerful case study for all of us. It is a story of how one scientific community’s (re)discovery of open principles can yield powerful lessons and analogies for the private sector and, more importantly the public sector.

Consider first, the similarities in problems. From the article:

Dr. Potter had recently left the National Institutes of Health and he had been thinking about how to speed the glacial progress of Alzheimer’s drug research.

“We wanted to get out of what I called 19th-century drug development — give a drug and hope it does something,” Dr. Potter recalled in an interview on Thursday. “What was needed was to find some way of seeing what was happening in the brain as Alzheimer’s progressed and asking if experimental drugs could alter that progression.”

Our government’s are struggling too. They are caught with a 20th-century organizational, decision-making and accountability structures. More to the point, they move at a glacial speed. On the one hand we should be worried about a government that moves too quickly, but a government that is too slow to be responsive to crises or to address structural problems is one that will lose the confidence of the public. Moreover, like in healthcare, many of the simpler problems have been addressed. citizens are looking for solutions to more complex problems. As with the scientists and Alzheimer’s we may need new models to speed the process up for understanding and testing solutions for these issues.

To overcome this 19th century approach – and achieve the success they currently enjoy – the scientists decided to do some radical.

The key to the Alzheimer’s project was an agreement as ambitious as its goal: not just to raise money, not just to do research on a vast scale, but also to share all the data, making every single finding public immediately, available to anyone with a computer anywhere in the world.

No one would own the data. No one could submit patent applications, though private companies would ultimately profit from any drugs or imaging tests developed as a result of the effort.

Consider this. Here a group of private sector companies recognize the intellectual property slows down innovation. The solution – dilute the intellectual property, focus on sharing data and knowledge, and understand that those who contribute most will be best positioned to capitalize on the gains at the end.

Sadly this is the same problem faced within governments. Sometimes it has to do with actual intellectual property (something I’ve recently argued our governments should abandon). However, the real challenge isn’t about about formal rules, it is more subtle. In complex siloed organizations where knowledge is power the incentives to maximize influence are to not share knowledge and data. Better to use the information you have strategically, in a limited fashion, to maximize influence. The result, data is kept as a scarce, but strategic asset. This is a theme I tackled both in my chapter in Open Government and in blog posts like this one.

In short, the real challenge is structural and cultural. Scientists had previously existed in a system where reputation (and career advancement) was built by hoarding data and publishing papers. While the individual incentives were okay, collectively this behavior was a disaster. The problem was not getting solved.

Today, it would appear that publishing is still important, but there are reputational effects from being the person or group to share data. Open data is itself a currency. This is hardly surprising. If you are sharing data it means you are doing lots of work, which means you are likely knowledgeable. As a result, those with a great deal of experience are respected but there remains the opportunity for those with radical ideas and new perspectives to test hypothesis and gain credibility by using the open data.

Unsurprisingly, this shift wasn’t easy:

At first, the collaboration struck many scientists as worrisome — they would be giving up ownership of data, and anyone could use it, publish papers, maybe even misinterpret it and publish information that was wrong.

Wow, does that sound familiar. This is invariably the first question government officials ask when you begin talking about open data. The answer, both in the scientific community and for government, is that you either believe in the peer-review process and public debate, or you don’t. Yes, people might misrepresent the data, or publish something that is wrong, but the bigger and more vibrant the community, the more likely people will find and point out the errors quickly. This is what innovation looks like… people try out ideas, sometimes they are right, sometimes they are wrong. But the more data you make available to people the more ideas can be tested and so the faster the cycle of innovation can proceed.

Whether it is behind the firewall or open to the public, open data is the core to accelerating the spread of ideas and the speed of innovation. These scientists are rediscovering that fact as our some governments. We’ve much to learn and do, but the case is becoming stronger and stronger that this is the right thing to do.

Which App for Climate Action do you like most?

5 Replies

Yesterday, at 5pm PST the Apps for Climate Action team at the Province of British Columbia released the list of 17 applications created using data from the Apps for Climate Action data catalog.

At the moment anyone can register and vote for the application that they think is the best. I’d encourage people to click over to the website and take a look.

The Apps for Climate Action is a demonstration of what can happen when we begin to make government held data freely available to the public: people can bring to life, even make fun, engaging and useful, what are often boring stats and numbers to bridge what Hans Rosling calls the last 6 inches (the distance from your eyes to your brain, a reference to the failure in design where we make data we can see, but not that captures our imagination).

In a month where our federal government cited imaginary data to justify policies on crime and has eliminated the gathering a huge swaths of effective data necessary for the efficient governing of our cities and rural communities as well as ensuring critical services will no longer reach innumerable Canadians, it is nice to see a province trying to do the opposite: not only understand that effective data is the cornerstone to good policy but to enable everyday, ordinary Canadians to leverage it so as to make smarter decisions, influence policy debates and empower themselves. It’s what a modern democracy, economy and civil society should look like.

The Apps for Climate action team and the government deserve a ton of praise fro striking out and trying something new and different. I hope they get worthwhile acknowledgement.

I for one am looking forward to the tough job of serving as a judge in the competition.

Minister Clement, privacy and (un)balanced views

8 Replies

Just moments ago Industry Minister Tony Clement, in response to growing criticism about his decision to end the mandatory long form census (now the Canadian Medical Association has come out in opposition), again cited “privacy concerns” from Canadians.

To quote Minister Clement via the Globe article:

“Just in the past 48 hours I’ve received dozens and dozens of [letters from] Canadians who despite the adverse publicity … have come forward and said, ‘We agree with your position,’” Mr. Clement said.

“I am not saying it’s every Canadian, but I am saying there are Canadians [who complained] and we should try to accommodate their concerns in a balanced way,” he said.

Of course, the minister made no reference to complaints made before he made his decision. That’s because it is rapidly becoming obvious their were none. The Privacy Commissioner registered just 3 complaints in the last decade. Statistics Canada’s survey about the last census generated no feedback regarding privacy. The minister’s claim about privacy concerns is a sham – he’s veering on the edge of having lied to Canadians. Now he’s trying to cover it up by citing complaints since his decision.

It is great to hear the Minister Clement is interested in Canadians concerns since he made his announcement because in contrast to the few dozen he’s received the rest of the country seems focused on a petition in opposition to his decision that has garnered 5800 Canadians names (and that grows at about 3 names a minute) in the last few days. Will he listen to their voices too?

I can’t say I’m confident. Even with conservatives like C.D. Howe Institute President William Robson speaking in opposition. Why? Two reasons.

First, let’s take a look at the last time the Minister consulted Canadians regarding a decision. How about last year on the issue of copyright, digital locks and circumvention. During this consultation 6641 speaking Canadians spoke against anti-circumvention provisions and a mere 46 Canadians spoke in favour. And yet, the desire of those 46 trumped the 6641. Or how about on Tuesday when Industry Canada suddenly pulled the second most popular discussion (about the census long form) from the Digital Economy Consultation.

It is the second, however, that is more important. I don’t think the Minster is the decision maker. Indeed, I don’t think he even wanted to do this. I don’t always agree with him but Minister Clement seems  smart and even fun. His twitter account is personable and engaging. More interestingly, the Minister allegedly spoke in opposition to this decision behind closed doors. The real decision maker was the Prime Minister. All the more reason why Canadians need to let their unhappiness with this decision known – they need to help Minister Clement reverse it.

The Census weak link: What the Liberals, Bloc & NDP should do

8 Replies

Public and the media condemnation of the government’s decision to end the long form census has been universal (see more fun quotes below). Now the political opposition has started to mobilize. The focus of the opposition has been on the secretive nature of the decision and the failure to consult any stakeholders. While this is problematic I’m not sure it is the most ironic and sensitive point. There is actually a much more juicy Achilles heel in this decision, one that might garner press attention.

In explaining the decision (such as in the quote to the Canadian Press below), Minister Clement has repeatedly claimed that MP offices has received numerous complaints about the long form census form:

“Every MP has had complaints like that so this year we decided to at least try another method that could be a sound method that would beat the issue of concern of degradation of data, and deal with the issue of coercion and too much intrusiveness”

The statement suggests broad base support but, ironically, anecdotal and, in theory, untestable.

The fun thing is, it doesn’t have to be that way. The opposition could bring more (and hard) information to this process and expose how the Conservatives are using a lack of information to at best mislead and at worse, lie, to ordinary Canadians. Better still, if the opposition parties have been organized, they should possess that data.

How is this? Well, I’d like to see the Liberals, Bloc and NDP ask each of their MPs to search and count every email and letter from constituents over the past 4 (or some sensible number) years that involved a complaint about the census.

My suspicion is that there are no more than 100 such letters (and, maybe even a lot less, like 10). You could then pick a couple of issues of your choice on which you have received 1000s of letters and which the conservatives have taken no action and put a simple chart up showing how they react to made up issues, but ignore real priorities of ordinary Canadians.

Of course, maybe there has been some massive, secret letter campaign targeted at Conservative only MPs but I think most Canadians will want to know how many letters the Minister and conservative MPs have actually received (sadly I don’t think you can FOI this) and most will suspect it is about as many as other MPs have.

Let’s show that better data leads to better decisions and that a little transparency can go a long way to exposing those who seek to mislead us. The minister says MPs get lots of letters – lets find out how many. And in comparison to what. Moreover, let’s show that those who seek to restrict the gathering of good information are generally those most inclined to use a lack of data to mislead the public and drive agendas that are not in the public interest. Canadians are sensible people, they want a smart government that makes good decisions. This, much more than secrecy, will rub them the wrong way.

The narrative isn’t as neat – but I suspect it could be turned into something more fun and more impactful.

Other fun articles…

Also, the universal condemnation of the Conservatives decision to end the mandatory long term census continues. My newest favourite is an article in the Globe where Derek Cook, Calgary’s the research and social planner at the City of Calgary and in the riding of the Prime Minister Harpers states that “If we don’t have that data at the neighbourhood level, we’re crippled.” Now, in addition to business groups, ngos, think tanks, cities, university researchers we are also seeing even more local and focused organizations such as those representing French Canadian communities, Inuit people and others starting to call for a reversal as well.