Yearly Archives: 2010

Census Update and other chuckles

4 Replies

Sorry for the lack of posts this week, blog was offline for a bit. (For geeks out there, I now have a company managing my blog for me and we we’re moving from a shared hosting service to a virtual private server – I should have less down time in the future – very excited).

Sadly, in that time there have been a bunch of fascinating developments on the census. As some of you may be aware a new poll by EKOS emerged today that has the Liberals and Conservatives dead even. More interesting however is how the census is playing a key role in the shift:

In seeking an explanation for these movements, we need look no further than the government’s ill-received decision to end the mandatory long form census. Not only does the shift of the highly educated support this conclusion, but a direct question on public approval for this decision provides compelling evidence that this move precipitated the current woes that the Conservative Party now faces.

When asked whether they felt that the privacy intrusion of the census justified a voluntary census or whether the lack of representativeness would cost us vital data, a clear majority of the public (56%) picked the latter (compared 26% who felt the mandatory long form was a violation of privacy). Even among Tory supporters, this appeal is not selling and there is an overwhelming lean to disapproval in the rest of the spectrum. Opposition to this decision is strongest among the university educated.

Of course, one of the retorts from pundits in favour of scrapping the long form census has been that only a few people care about this issue, it won’t matter in the medium term and it certainly won’t impact any election. For example:

Two things: I still standby my thesis that I believe that chucking mandatory nature of the long-form is a move to dismantle the welfare state (and that this is a move in the right direction). And two, nobody cares outside of the beehive. It’s the media that is pushing the story outside of the beehive walls propelled by the loud buzz of special interests.

Sigh, I suppose that 56% of  Canadians represent “a special interest.”

For me, both groups (56% and 26%) have legitimate concerns. As such, efforts by those in favour of this decision opposition as “special interest” driven are wrong and, frankly, disingenuous. Happily, they have failed. Indeed, the more these pundits try, the more they seem to make this a wedge issue in favour of those opposed to the decision. Mostly, I just think it would have been nice to have the issue debated before a decision was made.

More interesting has been another effort to defend trashing the long form census. I think Jack Mintz has thoroughly damaged his credibility with a terrible, contradictory and misleading op-ed in the Financial Post. Rather than dive into it, I encourage everyone to wander over to Aaron Wherry’s fantastic (and, unlike this post, short) dismantling of it. He’s already done all the heavy lifting.

Finally, just because I could help but notice the irony… I see that Conservative MP Garry Breitkreuz has an oped in the Mark in which he is worried about the role that the police is taking lobbying to keep the registry alive:

Taxpayers should be incensed at the CACP for co-opting the role of policy-maker. When law enforcement managers try to write the laws they enforce, history has taught us we risk becoming a state where police can dictate our personal freedoms.

I, of course, agree that it is dangerous for the police to get involved in policy debates. I now eagerly away for Garry Breitkreuz to demand that the RCMP own up to the funding of fake “research” in an effort to distort the debate on Insite and harm reduction policies. It would seem that someone at the RCMP, or higher up, doesn’t believe that should happen.

But on further review, maybe we shouldn’t get to excited. Looking at Garry’s website, and specifically, this PDF he’s made available for download, it seems like he’s actually quite keen to have police force members be outspoken about the gun registry as long as they agree with his view.

Ah, hypocrisy. If only he didn’t make it so easy.

Bugzilla – progress made and new thoughts

A few weeks ago I published a post entitled Some Thoughts on Improving Bugzilla. The post got a fair bit a traction and received a large number of supportive comments. But what was best, about the post, about open source, about Mozilla, is that it drew me into a serious of conversations with people who wanted to make some of it reality.

Specifically, I’d like to thank Guy Pyrzak over at Bugzilla and Clint Talbert at Mozilla both of whom spent hours entertaining and conversing about these ideas with me, problem solving them and, if we are really honest, basically doing all the heavy lifting to transform them from ideas on this blog into real changes.

So in this post I have two things to share. First is an update on progress from the ideas in the last post (which will be this post) as well as some new thoughts about how Mozilla instance of Bugzilla could be further improved (which will be my next post).

So here we go…

Update!

1. Simplifying Menus

First up I made some suggestions around simplifying the bugzilla landing page. These were pretty cosmetic, but they make the landing page a little less intimidating to a new user and, frankly, nicer for everyone. We are presently drafting up the small changes to the code that would require this change and getting ready to submit it as a proposal. Status – Yellow.

2. Gather more information about our users (and, while I’m at it, some more simplifying)

Second, I outlined some ideas for streamlining the process of joining bugzilla and on the data we collect about users.

On the first part, which is about the steamlined pages (designed to help ensure that true bug submitters end up in bugzilla and not those seeking support) here too we will be submitting some new proposed pages shortly. Status – Yellow

On the second part I suggested that we ask users if they English is their second language and that we mark new bugzilla accounts with a “new” symbol. Guy is coding up an extension to Bugzilla that will both of these. Once done, I’ll suggest to Mozilla that they include this extension in their instance. Status – Green.

3. Make Life Easier for Users and the Triage Guys

I thought we could make life more efficient for triage and users if we added a status where bugs could be declared “RESOLVED-SUPPORT.” There’s been some reception to this idea. However, the second part of this idea is that once a bug is tagged as such a script automatically should scan the support database, find articles with a strong word correlation to the bug description and email the bug submitter links to those pages. Once again, Guy has stepped forward to develop such an extension which hopefully will be working in the not to distant future. Status – Green.

4. Make Bugzilla Celebrate, enhance our brand and build community

But probably the most exciting part is the final suggestion. That we send (at least non-developers) much nicer emails celebrating that the bug they submitted has been patched. It turns out (hardly surprising) that I wasn’t the first person to think that Bugzilla should be able to send HTML emails. Indeed, that feature request was first made back in 2001 and, when I blogged about this the other week, had not be updated since 2006. Once again, Guy has proven to be unbelievably helpful. It turns out that due to some changes to bugzilla many of the blocks to patching this had disappeared and so he has been working on the code. Status – Green.

Lots here for many people to be proud of. Hopefully some of these ideas will go live in the not too distant future. That said, still many hurdles to clear and if you are a decision maker on any of these and would like to talk about these ideas, please do not hesitate to contact me.

Are you a Public Servant? What are your Open Data Challenges?

3 Replies

A number of governments have begun to initiate open data and open government strategies. With more governments moving in this direction a growing number of public servants are beginning to understand the issues, obstacles, challenges and opportunities surrounding open data and open government.

Indeed, these challenges are why many of these public servants frequent this blog.

This is precisely why I’m excited to share that, along with the Sunlight Foundation, the Personal Democracy Forum, Code for America, and GovLoop, I am helping Socrata in a recently launched survey aimed at government employees at the national, regional and local levels in the US and abroad about the progress of Open Data initiatives within their organization.

If you are a government employee please consider taking time to help us understand the state of Open Data in government. The survey is comprehensive, but given how quickly this field and the policy questions that come with it, is expanding, I think the collective result of our work could be useful. So, with that all said, I know you’re busy, but hope you’ll consider taking 10 minutes to fill out the survey. You can find it at: http://www.socrata.com/benchmark-study.

Breaking: A private members bill for reinstating long-form census to be announced today at 11am

4 Replies

This is an Eaves.ca exclusive. On the matter of a issue that just won’t go away, today the Liberal’s  will be tabling a private members bill to reinstate the Long-Form Census today at 11am EST.

Post below is a copy of the press release and the proposed bill.

I suspect there will be more on this pretty much… everywhere.

For immediate release

August 26, 2010

Liberals introduce bill to reinstate mandatory long-form census

OTTAWA – In a move to bring ‘sense back to the census,’ Liberals today introduced legislation to amend the Statistics Act to bring back the mandatory long-form census questionnaire.

“Liberals believe that sound information helps make sound decisions,” said Liberal Finance Critic John McCallum. “That’s why we will fix the mess Stephen Harper created by reinstating the mandatory long-form census and ensuring the threat of jail time is removed in the Act.”

The Bill – which will be tabled upon the September 20th return of Parliament – clarifies that 20% of the Canadian population will receive a mandatory long-form questionnaire during the period in which the Government of Canada conducts a census. It will also remove the controversial threat of jail time for not completing the census.

“As one business person put it recently, the first rule of business is ‘you can’t manage what you can’t measure,’ said Mr. McCallum. “It’s also the first rule of government, yet the Conservatives have chosen to blindfold themselves by gutting the census, which means our annual $250 billion budget is more likely to be mismanaged.”

The long-form census plays a major role in improving health care, education, employment equity, immigrant settlement, retirement security, public transportation, and support for veterans and Aboriginals, to name a few.

“Provincial, territorial and municipal governments, community groups, medical associations, religious groups and countless organizations who spoke out against the Conservative decision understand the census is a vital tool that helps inform their decisions,” said Liberal Industry Critic Marc Garneau.

The Conservative decision to make the long-form census voluntary will mean that the results can’t be compared to previous years, and will skew the picture of what Canada looks like, as lower income, minority Canadians will be less likely to fill out a voluntary survey.

“The Conservatives have shown zero interest in listening to Canadians but were forced to add language questions only when they faced legal threat, effectively admitting the voluntary form is inadequate,” added Mr. Garneau. “Now that the Conservatives are the last party to advocate removing the threat of jail time, it’s time to make the long-form census mandatory again, so we can get an accurate picture of Canadians.”

“Canadians rely on the federal government for a multitude of services, and they expect our federal government to make wise decisions based on sound information. This bill will make sure decision makers at all levels of society can do just that,” he concluded.

-30-

Contact:

Office of the Hon. John McCallum, MP: 613-996-3374

Office of Marc Garneau, MP: 613-796-0103

Background:

An Act to amend the Statistics Act (mandatory long-form census)

SUMMARY

This enactment amends the Statistics Act to provide that the census of population taken under section 19 the Act must be taken using a long-form census questionnaire that conforms substantially, in length and substantive scope, to the census questionnaire used to take that census starting in 1971 and at intervals thereafter to meet the requirements of that section. This enactment also removes the punishment of imprisonment for a person convicted for the offence of providing false of unlawful information.

Her Majesty, by and with the advice and consent of the Senate and House of Commons of Canada, enacts as follows:

1. Section 19 of the Statistics Act is amended by adding the following after subsection (3):

Mandatory long-form census questionnaire

(4) The census taken at the time and in the manner set out in this section must include the use of a long-form census questionnaire and the distribution of that questionnaire to at least 20% of all households or whatever percentage of households is determined to be necessary by the Chief Statistician to ensure an accurate statistical representation of the Canadian population and its constituent groups.

“long-form census questionnaire”

(5) In this section, the term “long-form census questionnaire” refers to a census questionnaire that conforms substantially, in length and substantive scope, to the long-form census used to take the census in 1971 and at intervals thereafter to meet the requirements of this section.

2. The portion of section 31 of the Act after paragraph (b) is replaced by the following:

is, for every refusal or neglect, or false answer or deception, guilty of an offence and liable on summary conviction to a fine not exceeding five hundred dollars.

Creating effective open government portals

7 Replies

In the past few years a number of governments have launched open data portals. These sites, like www.data.gov or data.vancouver.ca share data – in machine readable formats (e.g. that you can play with on your computer) that government agencies collect.

Increasingly, people approach me and ask: what makes for a good open data portal? Great question. And now that we have a number of sites out there we are starting to learn what makes a site more or less effective. A good starting point for any of this is 8 Open Government principles, and for those newer to this discussion, there are the 3 laws of open data (also available in German Japanese, Chinese, Spanish, Dutch and Russian).

But beyond that, I think there are some pretty tactical things, data portal owners should be thinking about. So here are some issues I’ve noticed and thought might be helpful.

1. It’s all about automating the back end

Probably the single greatest mistake I’ve seen governments make is, in the rush to get some PR or meet an artificial deadline, they create a data portal in which the data must be updated manually. This means that a public servant must run around copying the data out of one system, converting (and possibly scrubbing it of personal and security information) and then posting it to the data portal.

There are a few interrelated problems with this approach. Yes, it allows you to get a site up quickly but… it isn’t sustainable. Most government IT departments don’t have a spare body that can do this work part time, even less so if the data site were to grow to include 100s or 1000s of data sets.

Consequently, this approach is likely to generate ill-will towards the government, especially from the very community of people who could and should be your largest supporters: local tech advocates and developers.

Consider New York, here is a site where – from I can tell – the data is not regularly updated and grumblings are getting louder. I’ve heard similar grumblings out of some developers and citizens in Canadians cities where open data portals get trumpeted despite infrequent updates and having few data sets available.

If you are going to launch an open data portal, make sure you’ve figured out how to automate the data updates first. It is harder to do, but essential. In the early days open data sites often live and die based on the engagement of a relatively small community or early adopters – the people who will initially make the data come alive and build broader awareness. Frustrate the community and the initiative will have a harder time gaining traction.

2. Keep the barriers low

Both the 8 principles and 3 laws talk a lot about licensing. Obviously there are those who would like the licenses on many existing portals to be more open, but in most cases the licenses are pretty good.

What you shouldn’t do is require users to register. If the data is open, you don’t care who is using it and indeed, as a government, you don’t want the hassle of tracking them. Also, don’t call your data open if members must belong to a educational institution or a non-profit. That is by definition not data that is open (I’m looking at you StatsCan, its not liberated data if only a handful of people can look at it, sadly, you’re not the only site to do this). Worst is one website that, in order to access the online catalogue you have to fax in a form outlining who you are.

This is the antithesis of how an open data portal should work.

3. Think like (or get help from) good librarians and designers

The real problem is when sites demand too much of users to even gain access to the data. Readers of this blog know about my feelings regarding Statistics Canada’s website, the data always seems to be one click away. Of course, that’s if you even think you are able to locate the data you are interested in, which usually seems impossible to find.

And yes, I know that Statistics Canada’s phone operators are very helpful and can help you locate datasets quickly – but I submit to you that this is a symptom of a problem. If every time I went to Amazon.com I had to call a help desk to find the book I was interested in I don’t think we’d be talking about how great Amazon’s help desk was. We’d be talking about how crappy their website is.

The point here is that an open data site is likely to grow. Indeed, looking at data.gov and data.gov.uk these sites now have thousands of data sets on them. In order to be navigable they need to have excellent design. More importantly, you need to have a new breed of librarian – one capable of thinking in the online space – to help create a system where data sets can be easily and quickly located.

This is rarely a problem early on (Vancouver has 140 data sets up, Washington DC, around 250, these can still be trolled through without a sophisticated system). But you may want to sit down with a designer and a librarian during these early stages to think about how the site might evolve so that you don’t create problems in the future.

4. Feedback

Finally, I think good open data portals want, and even encourage feedback. I like that data.vancouver.ca has a survey on the site which asks people what data sets they would be interested in seeing made open.

But more importantly, this is an area where governments can benefit. No data set is perfect. Most have a typo here or there. Once people start using your data they are going to find mistakes.

The best approach is not to pretend like the information is perfect (it isn’t, and the public will have less confidence in you if you pretend this is true). Instead, ask to be notified about errors. Remember, you are using this data internally, so any errors are negatively impacting your own planning and analysis. By harnessing the eyes of the public you will be able to identify and fix problems more quickly.

And, while I’m sure we all agree this is probably not the case, maybe the face that the data us public, there will be a small added incentive to fixing it quickly. Maybe.


Interview on Open Source, Open Gov & Open Data withe CSEDEV

2 Replies

The other week – in the midst of boarding a plane(!) – I did an interview with the CSEDEV on some thoughts around open data, open government and open source.

The kind people at CSEDEV have written up the interview in a kind of paraphrased way and published it as three short blog posts here, part 2 here and part 3 here.

Part of what makes this interesting to me is how a broader set of people are becoming interested in open government. Take CSEDEV for example. Here is an Ottawa based software firm focused on enterprise solutions. It’s part of an increasing number of software companies and IT consulting firms are taking note of the open government and open data meme. Indeed, another concrete example of this is Lagan, a large supplier of 311 systems, announced the other week that they would support the open311 standard. This dramatically alters the benefits of a 311 system and the capacity for it to serve as a platform and innovation driver for a city.

But, even more exciting, the meme is starting to spread beyond IT and software. I was recently asked to write an article on what open data and open government means for business more generally, here in BC. (Will link to it, when published)

These moments represent an important shift in the open data and open government debate. With vendors and consultants taking notice governments can more easily push for, and expect, off the shelf solutions that support open government initiatives. Not only could this reduce cost to government and improve access for public servants and citizens, it could also be a huge boost for open standards which prove to be transformative to the management of information in the public sector.

Exciting times. Watch the open government space – now that it’s linked to IT, it’s beginning to gain speed.

The week in review (or… why I blog and a thank you)

3 Replies

Here’s a few snippets of comments, emails and other communications I’ve had this week in response to specific posts or just the blog in general. Each one touches on why I love blogging and my readers and why this blog has come to mean so much to me.

Venting, and finding out your not alone…

So, yesterday I got a little bit into a hate-on for Statistics Canada’s website. It wasn’t the first time and pretty much every time I do it I find another soul out there whose had their soul crushed by the website as well. Take this comment from last week:

Re: Stats Canada’s website being unusable. I completely frickin agree. God. Has anyone in government actually tried to use that website? An econ professor gave our class an assigment last year that involved looking stuff up on Statscan. Half of our class failed the assignment because they gave up and the other half had the wrong data, but got the marks anyways for trying. I think he actually took that assigment off of the grading at the end. It’s a bloody gong show…

Sometimes it makes me feel more human knowing that others are out there struggling with the same thing. StatsCan does great work… I just wish they made it accessible.

…and then having some kind souls find some solutions for you.

But as nice as knowing you’re not alone… even better is how often the internet connects you to others who just happen to have that esoteric piece of knowledge that saves the day.

I agree, Stats Can is one of the worst government websites out there (specifically those stupid CANSIM tables), one that, as a policy analyst with XXXXXXXX Canada, i frequently have to use to get data. I had the data for XXXXXXX and it wasn’t hard to get it for the country.

This kind soul led me straight to a completely different page on statscan that happened to have the data I was looking for. (for those interested, it was here).

And they weren’t the only one. Another reader posted a link to the data over twitter…

Thank god there is an army good natured amateur and professional experts experienced in navigating the byzantine structure of the statscan website!

So… thank you! I’m going to try to grind out an updated pan-North American version of the Fatness Index this weekend.

Impacting Policy

But this week also had that other rewarding ingredient I love to get: hearing about a post helped, incrementally, foster better public policy. This came in via email from a public servant about yesterday’s blog post:

Your blog today provided a good example in a meeting with government colleagues about the benefits of opening data. It illustrates the implications of not releasing data to the public (e.g. stifling innovation)… It resonated well with them.

This is a huge part of why I blog. Part of it is to explore ideas, part of it is to introduce ideas and thoughts, but a big piece of it is to enable public servants and do just this, helps small internal government meeting (on subjects like open data) go a little more smoothly.

So to everyone out there, be it policy wonks, students, public servants, politicians or ordinary, engaged citizens. Thank you. It was a good week. We wrote some good posts, some good comments, had an original story on the stupidity of the census, and maintained sanity in the face of the StatsCan website. Thank you everyone for making it so fun. Hope you all have a great weekend. – Dave

Fatness Index 2 years on: the good, the bad, the ugly

4 Replies

Two years ago I saw that Richard Florida and Andrew Sullivan had re-posted a map created by calorielab that color-coded US states by weight.

As I found it interesting I created a North America wide map the included Canadian data (knowing that it probably would be a perfect apple to apple comparisons). The map and subsequent blog post turned into one of my best viewed pages with well over 20,000 pageviews.

The very cool people over at Calorie Labs informed me that they have released an updated version of the American map (posted below, you can see the original at their site here). Not too much has changed, but after looking at the map I’ve a few comments.

Calorie lab’s release of an updated version of the map has triggered a few thoughts and some lessons that I think should matter to policy makers, health-care professionals and citizens in general. Here they are:

The Good

The amazing people at Calorie Lab. When I created the map 2 years ago I didn’t even check to see if their work was copyrighted. Although the data was public domain, I copied Calorielab’s colour palette as I was trying to create a “mash-up” of their work with Canadian data. I wanted the maps to look similar. My map was a derivative work.

Did the people at Calorielab freak out? No. Quite the opposite. They reached out, said thank you and asked if I needed help.

It seems this year they’ve gotten even cooler. I don’t remember if the original map’s license but with the publishing of their 2010 update they wrote:

CalorieLab’s United States of Obesity 2010 map is licensed for use by anyone in any media and can be downloaded in various formats (small GIF, large GIF, SVG, EPS).

There’s a line directed specifically at people like me. It says, please, use this map! Not only is the license open but they’ve provided it in lots of formats (Which is great cause two years I had to recreate the thing from scratch and it took hours).

So naturally you are wondering, where is David’s 2010 mashup-Northern American Fatness Index.

The Bad

The bad is that trying to find the Canadian data is a pain. A couple of times a year I get a cool idea for a visual or graph that Statistics Canada data might help me create. In minutes I’m on their webpage and, within 5 minutes, I’m walking away from my computer fearing I might throw it out the window.

StatCans website may be the worst, most inaccessible government website in the western world. Whatever data you are looking for always seems to be at least one more click away.

It spent an hour trying to find data that StatsCan allegedly wants me to find. (This in an era of google where I generally find data people don’t want me to find, in minutes). Ultimately, I think I found the relevant data on overwieight/obesity figures by province (but who knows! Should I be choosing peer group A, or B, or C, D, E, F, G? None of which have labels explaining what they mean!).

The Ugly

Sadly, it gets worse. Even if you a) locate the data on Statscan’s website and b) it is free, it will probably still be inaccessible.  The only way the data can be viewed is with a Beyond 20/20 Professional Browser. You need to learn a new software package, one 99.9% of Canadians have never heard of, and that only works on a PC (I’m on a mac). The data I want is pretty simple, a CSV file, or even an Excel spreadsheet would be sufficient, something the average Canadian could access. But I guess it is not to be.

So I give up.

You win StatsCan. There are 10s of thousands of Canadians like me who would love to do interesting things with the data our tax dollars paid to collect, but even when your data is free and “open,” it isn’t. You’ve enjoyed tremendous support in the last month from those Canadians who understand why you are important (including me) but many Canadians have had to go up a steeper learning curve around why they should care. I might suggest they’d have gotten up that curve faster if they too could have used your data.

Myself, healthcare professionals, students and countless others could paint innumerable stories explaining Canadians and Canada to one another – helping us grasp our history, our social and health challenges, as well simply who we are. But we can’t.

In the end I’m still one of your biggest supporters, but frankly even I feel alienated.

Note: If someone wants to help me get this data, I’ll take a cut at recreating the map again, otherwise, as I said before. I give up.

How Science Is Rediscovering "Open" And What It Means For Government

4 Replies

Pretty much everybody in government should read this fantastic New York Times article Sharing of Data Leads to Progress on Alzheimer’s. On one hand the article is a window into what has gone wrong with science – about how all to frequently a process that used to be competitive but open, and problem focused has become a competitive but closed and intellectual property driven (one need only look at scientific journals to see how slow and challenging the process has become).

But strip away the talk about the challenges and opportunities for science. At its core, this is an article is about something more basic and universal. This is an article about open data.

Viewed through this lens it is a powerful case study for all of us. It is a story of how one scientific community’s (re)discovery of open principles can yield powerful lessons and analogies for the private sector and, more importantly the public sector.

Consider first, the similarities in problems. From the article:

Dr. Potter had recently left the National Institutes of Health and he had been thinking about how to speed the glacial progress of Alzheimer’s drug research.

“We wanted to get out of what I called 19th-century drug development — give a drug and hope it does something,” Dr. Potter recalled in an interview on Thursday. “What was needed was to find some way of seeing what was happening in the brain as Alzheimer’s progressed and asking if experimental drugs could alter that progression.”

Our government’s are struggling too. They are caught with a 20th-century organizational, decision-making and accountability structures. More to the point, they move at a glacial speed. On the one hand we should be worried about a government that moves too quickly, but a government that is too slow to be responsive to crises or to address structural problems is one that will lose the confidence of the public. Moreover, like in healthcare, many of the simpler problems have been addressed. citizens are looking for solutions to more complex problems. As with the scientists and Alzheimer’s we may need new models to speed the process up for understanding and testing solutions for these issues.

To overcome this 19th century approach – and achieve the success they currently enjoy – the scientists decided to do some radical.

The key to the Alzheimer’s project was an agreement as ambitious as its goal: not just to raise money, not just to do research on a vast scale, but also to share all the data, making every single finding public immediately, available to anyone with a computer anywhere in the world.

No one would own the data. No one could submit patent applications, though private companies would ultimately profit from any drugs or imaging tests developed as a result of the effort.

Consider this. Here a group of private sector companies recognize the intellectual property slows down innovation. The solution – dilute the intellectual property, focus on sharing data and knowledge, and understand that those who contribute most will be best positioned to capitalize on the gains at the end.

Sadly this is the same problem faced within governments. Sometimes it has to do with actual intellectual property (something I’ve recently argued our governments should abandon). However, the real challenge isn’t about about formal rules, it is more subtle. In complex siloed organizations where knowledge is power the incentives to maximize influence are to not share knowledge and data. Better to use the information you have strategically, in a limited fashion, to maximize influence. The result, data is kept as a scarce, but strategic asset. This is a theme I tackled both in my chapter in Open Government and in blog posts like this one.

In short, the real challenge is structural and cultural. Scientists had previously existed in a system where reputation (and career advancement) was built by hoarding data and publishing papers. While the individual incentives were okay, collectively this behavior was a disaster. The problem was not getting solved.

Today, it would appear that publishing is still important, but there are reputational effects from being the person or group to share data. Open data is itself a currency. This is hardly surprising. If you are sharing data it means you are doing lots of work, which means you are likely knowledgeable. As a result, those with a great deal of experience are respected but there remains the opportunity for those with radical ideas and new perspectives to test hypothesis and gain credibility by using the open data.

Unsurprisingly, this shift wasn’t easy:

At first, the collaboration struck many scientists as worrisome — they would be giving up ownership of data, and anyone could use it, publish papers, maybe even misinterpret it and publish information that was wrong.

Wow, does that sound familiar. This is invariably the first question government officials ask when you begin talking about open data. The answer, both in the scientific community and for government, is that you either believe in the peer-review process and public debate, or you don’t. Yes, people might misrepresent the data, or publish something that is wrong, but the bigger and more vibrant the community, the more likely people will find and point out the errors quickly. This is what innovation looks like… people try out ideas, sometimes they are right, sometimes they are wrong. But the more data you make available to people the more ideas can be tested and so the faster the cycle of innovation can proceed.

Whether it is behind the firewall or open to the public, open data is the core to accelerating the spread of ideas and the speed of innovation. These scientists are rediscovering that fact as our some governments. We’ve much to learn and do, but the case is becoming stronger and stronger that this is the right thing to do.

Good Statistical Data: We fund it in Africa, but not in Canada

15 Replies

It turns out that the Canadian government is a supporter of collecting good statistical data – especially data that can be used to alleviate poverty and address disease. There’s only one catch. It can’t help Canadians.

As the fall out from the canceling of the mandatory long form census continues to grow – today the head of Alberta Health Services spoke out, saying the the census decision will hamper the province’s ability to deliver health care efficiently – we  now learn that the very arguments the government dismisses here in Canada, it supports on the international stage.

As it happens, the Canadian International Development Agency contributes to the Trust Fund for Statistical Capacity Building (TFSCB) an international fund designed to support the Marrakech Action Plan for Statistics. And what, you should legitimately ask, is the Marrakech plan? It is a general agreement by international actors to support building developing countries statistical capacity. It has, specifically, as a primary objective, the goal of developing countries capacity to perform censuses. More interestingly, it has a secondary goal, to: “Set up an international Household Survey Network.” the very same part of the census the government just gutted here in Canada.

Both the Trust Fund and the Marrakech Action plan websites explain this in detail. But so to does the CIDA website, where the government acknowledges that this work is essential as:

“The projects supported aim to improve in the collection, processing, analysis, storage, dissemination, and use of quality statistics to support poverty reduction and economic and social development. Developing countries can submit funding proposals to the Trust Fund. The proposals are ideally based on a national strategy for the development of statistics. By implementing such a strategy, countries can improve their statistical capacities to measure development progress and results, notably with regard to the Millennium Development Goals, and to better plan and utilize scarce resources.”

In short, our government accepts that the Household Survey is essential to helping marginalized people. It recognizes that such a survey will help other governments tackle poverty, health care and other social development issues. Indeed, it believes it so strongly, we will spend millions of dollars a year funding the development of statistical capacity abroad to ensure that other governments don’t do what we just did to the long form census.

I’m grateful that our government believes that good statistics and the types of questions found on the long form are essential to developing good policy – I’m just sad they don’t believe it to be true for Canada citizens.