How Science Is Rediscovering "Open" And What It Means For Government

4 Replies

Pretty much everybody in government should read this fantastic New York Times article Sharing of Data Leads to Progress on Alzheimer’s. On one hand the article is a window into what has gone wrong with science – about how all to frequently a process that used to be competitive but open, and problem focused has become a competitive but closed and intellectual property driven (one need only look at scientific journals to see how slow and challenging the process has become).

But strip away the talk about the challenges and opportunities for science. At its core, this is an article is about something more basic and universal. This is an article about open data.

Viewed through this lens it is a powerful case study for all of us. It is a story of how one scientific community’s (re)discovery of open principles can yield powerful lessons and analogies for the private sector and, more importantly the public sector.

Consider first, the similarities in problems. From the article:

Dr. Potter had recently left the National Institutes of Health and he had been thinking about how to speed the glacial progress of Alzheimer’s drug research.

“We wanted to get out of what I called 19th-century drug development — give a drug and hope it does something,” Dr. Potter recalled in an interview on Thursday. “What was needed was to find some way of seeing what was happening in the brain as Alzheimer’s progressed and asking if experimental drugs could alter that progression.”

Our government’s are struggling too. They are caught with a 20th-century organizational, decision-making and accountability structures. More to the point, they move at a glacial speed. On the one hand we should be worried about a government that moves too quickly, but a government that is too slow to be responsive to crises or to address structural problems is one that will lose the confidence of the public. Moreover, like in healthcare, many of the simpler problems have been addressed. citizens are looking for solutions to more complex problems. As with the scientists and Alzheimer’s we may need new models to speed the process up for understanding and testing solutions for these issues.

To overcome this 19th century approach – and achieve the success they currently enjoy – the scientists decided to do some radical.

The key to the Alzheimer’s project was an agreement as ambitious as its goal: not just to raise money, not just to do research on a vast scale, but also to share all the data, making every single finding public immediately, available to anyone with a computer anywhere in the world.

No one would own the data. No one could submit patent applications, though private companies would ultimately profit from any drugs or imaging tests developed as a result of the effort.

Consider this. Here a group of private sector companies recognize the intellectual property slows down innovation. The solution – dilute the intellectual property, focus on sharing data and knowledge, and understand that those who contribute most will be best positioned to capitalize on the gains at the end.

Sadly this is the same problem faced within governments. Sometimes it has to do with actual intellectual property (something I’ve recently argued our governments should abandon). However, the real challenge isn’t about about formal rules, it is more subtle. In complex siloed organizations where knowledge is power the incentives to maximize influence are to not share knowledge and data. Better to use the information you have strategically, in a limited fashion, to maximize influence. The result, data is kept as a scarce, but strategic asset. This is a theme I tackled both in my chapter in Open Government and in blog posts like this one.

In short, the real challenge is structural and cultural. Scientists had previously existed in a system where reputation (and career advancement) was built by hoarding data and publishing papers. While the individual incentives were okay, collectively this behavior was a disaster. The problem was not getting solved.

Today, it would appear that publishing is still important, but there are reputational effects from being the person or group to share data. Open data is itself a currency. This is hardly surprising. If you are sharing data it means you are doing lots of work, which means you are likely knowledgeable. As a result, those with a great deal of experience are respected but there remains the opportunity for those with radical ideas and new perspectives to test hypothesis and gain credibility by using the open data.

Unsurprisingly, this shift wasn’t easy:

At first, the collaboration struck many scientists as worrisome — they would be giving up ownership of data, and anyone could use it, publish papers, maybe even misinterpret it and publish information that was wrong.

Wow, does that sound familiar. This is invariably the first question government officials ask when you begin talking about open data. The answer, both in the scientific community and for government, is that you either believe in the peer-review process and public debate, or you don’t. Yes, people might misrepresent the data, or publish something that is wrong, but the bigger and more vibrant the community, the more likely people will find and point out the errors quickly. This is what innovation looks like… people try out ideas, sometimes they are right, sometimes they are wrong. But the more data you make available to people the more ideas can be tested and so the faster the cycle of innovation can proceed.

Whether it is behind the firewall or open to the public, open data is the core to accelerating the spread of ideas and the speed of innovation. These scientists are rediscovering that fact as our some governments. We’ve much to learn and do, but the case is becoming stronger and stronger that this is the right thing to do.

Good Statistical Data: We fund it in Africa, but not in Canada

15 Replies

It turns out that the Canadian government is a supporter of collecting good statistical data – especially data that can be used to alleviate poverty and address disease. There’s only one catch. It can’t help Canadians.

As the fall out from the canceling of the mandatory long form census continues to grow – today the head of Alberta Health Services spoke out, saying the the census decision will hamper the province’s ability to deliver health care efficiently – we  now learn that the very arguments the government dismisses here in Canada, it supports on the international stage.

As it happens, the Canadian International Development Agency contributes to the Trust Fund for Statistical Capacity Building (TFSCB) an international fund designed to support the Marrakech Action Plan for Statistics. And what, you should legitimately ask, is the Marrakech plan? It is a general agreement by international actors to support building developing countries statistical capacity. It has, specifically, as a primary objective, the goal of developing countries capacity to perform censuses. More interestingly, it has a secondary goal, to: “Set up an international Household Survey Network.” the very same part of the census the government just gutted here in Canada.

Both the Trust Fund and the Marrakech Action plan websites explain this in detail. But so to does the CIDA website, where the government acknowledges that this work is essential as:

“The projects supported aim to improve in the collection, processing, analysis, storage, dissemination, and use of quality statistics to support poverty reduction and economic and social development. Developing countries can submit funding proposals to the Trust Fund. The proposals are ideally based on a national strategy for the development of statistics. By implementing such a strategy, countries can improve their statistical capacities to measure development progress and results, notably with regard to the Millennium Development Goals, and to better plan and utilize scarce resources.”

In short, our government accepts that the Household Survey is essential to helping marginalized people. It recognizes that such a survey will help other governments tackle poverty, health care and other social development issues. Indeed, it believes it so strongly, we will spend millions of dollars a year funding the development of statistical capacity abroad to ensure that other governments don’t do what we just did to the long form census.

I’m grateful that our government believes that good statistics and the types of questions found on the long form are essential to developing good policy – I’m just sad they don’t believe it to be true for Canada citizens.

Canada's emerging opendata mashups (plus some ideas)

2 Replies

Over at IT World Canada, Jennifer Kavur has put together a list of 25 sites and apps for Open Government. What’s fantastic about this list is it demonstrates to government officials and politicians that there is a desire, here in Canada, to take government data and do interesting things with it.

Whether driven by developers like Michael Mulley or Morgan Peers who just want to improve democracy and have fun, or whether it is by those like Jeff Aramini who want to start a business and make money, the appetite to do something is real, and it is growing. Indeed, the number of apps and sites is far greater than 25 including simple mashups like CSEDEV’s environment Canada pollution data display or the 17 apps recently created as part of the Apps For Climate Action competition.

What is all the more remarkable is that this growth is happening even as there is little government data available. Yes, a number of cities have made data available, but provincially and especially federally there is almost no concerted effort to make data easy to use. Indeed, many of the sites cited by Kavur have to “scrape” the data of government websites, a laborious process that can easily break if the government website changes structure. It begs the question, what would happen if the data were accessible?

As an aside, two data sets I’m surprised no one has done much with are both located on the Toronto website: Road Restriction data and DineSafe data. Given how poor the city’s beta road restriction website is and the generally high interest in traffic news, I’d have thought that one of the local papers or media companies would have paid someone to develop an iPhone app or a widget for their website using this data. It is one thing commuters and consumers want to know more about.

As for DineSafe, I’m also surprised that no one in Toronto has approached the eatsure developers and asked them if they can port the site to Toronto. I’m still more surprised that a local restaurant review website has developed a widget that shows you to the DineSafe rating of a restaurant on its review page. Or that an company like urbanspoon or yelp hasn’t hired an iPhone app developer to integrate this data into their app…

Good times for Open Data in Canada. But if the feds and provinces were on board it could be much, much better…

The Government admits the voluntary Long Form is bunk

8 Replies

Yesterday, in response to a legal challenge from the Federation of Francophone and Acadian Communities of Canada Minister Clement announced the government would shift questions regarding the French language from the voluntary long form to the mandatory short form of the census.

Specifically these questions would move:

1) Can this person speak English or French well enough to conduct a conversation?

2a) What language does this person speak most often at home?

2b) Does this person speak any other languages on a regular basis at home?

This was done allegedly after receiving “new” advice from the new Statscan head which advised that having these questions in the voluntary form would not satisfy the government’s obligations under the Official Languages Act.

Of course, my friend points out that by moving these question the government is admitting that data collected by the voluntary form is essentially useless. It certainly isn’t good enough for the government otherwise… why not leave it on the voluntary form? In short, the minister just admitted that the long form, for which we are going to spend over $130M dollars to collect data, is for intent and purposes, useless.

This of course runs counter to the claims the Minister has been making all along that the data would still be sound.

It also begs the question of what other obligations the government has that might not be met. Legislation around disabled Canadians comes to mind, as of course do immigrants and first nations, all groups that we will know significantly less about. Once again, good public policy is based on having sound data. It would be nice if the government at least acknowledged this.

Update: Turns out the National Statistical Council (who normally advises StatsCan on these issues but which has been cut out of the loop on the issue of the long form) has come to the same conclusion.

2nd Update: In the ongoing – we are an international laughing stock – part of the census story, Nature, one of the two most preeminent science and research peer review journals in the world has published a scathing editorial piece about the Long Form Census decision. This is equivalent to the international research community pointing at Canada and asking “why do you want to go back to the dark ages?” It’s a question that – as a passionate believer in effective public policy – I’ve been asking myself as well.

Which App for Climate Action do you like most?

5 Replies

Yesterday, at 5pm PST the Apps for Climate Action team at the Province of British Columbia released the list of 17 applications created using data from the Apps for Climate Action data catalog.

At the moment anyone can register and vote for the application that they think is the best. I’d encourage people to click over to the website and take a look.

The Apps for Climate Action is a demonstration of what can happen when we begin to make government held data freely available to the public: people can bring to life, even make fun, engaging and useful, what are often boring stats and numbers to bridge what Hans Rosling calls the last 6 inches (the distance from your eyes to your brain, a reference to the failure in design where we make data we can see, but not that captures our imagination).

In a month where our federal government cited imaginary data to justify policies on crime and has eliminated the gathering a huge swaths of effective data necessary for the efficient governing of our cities and rural communities as well as ensuring critical services will no longer reach innumerable Canadians, it is nice to see a province trying to do the opposite: not only understand that effective data is the cornerstone to good policy but to enable everyday, ordinary Canadians to leverage it so as to make smarter decisions, influence policy debates and empower themselves. It’s what a modern democracy, economy and civil society should look like.

The Apps for Climate action team and the government deserve a ton of praise fro striking out and trying something new and different. I hope they get worthwhile acknowledgement.

I for one am looking forward to the tough job of serving as a judge in the competition.

How not to fix a mistake: How Clement's just dug a deeper hole

4 Replies

Faced with a lawsuit from a Francophone group opposed to the changes to the long form of the census the government has decided to shift questions on francophone skill from the now voluntary long form to the mandatory short form of the census.

The government hopes that this will placate angry francophone voters. The reality is that it will probably make many Canadians more upset.

Now the public gets to see that:

a) It is completely obvious that the Government did not think this policy through – so much so that it must now make changes on the fly

b) That the only way to be heard by this government is to take legal action

c) That the government is willingly ignoring the innumerably other stakeholders like the federal ministries, the provinces, cities, plus 300+ NGOs, business groups, religious organizations, etc… that are negatively impacted by this decision

d) That their goal is to destroy the census and that the actions today were about accelerating that process, not consulting or listening to Canadians

Of course, the repercussions continue to pile in. Even some hard core conservative commentators think that Minister Clement has embarrassed himself and seriously damaged his career, and polls continue to show the Conservatives have slumped significantly. The most damning piece in those polls however is hidden in the find print, not only have the conservatives slid double digits, but they are not the second choice for most Canadians, so even if people do change their minds, decisions like this make it harder to imagine they’ll go conservative.

More evidence that StatsCan disagreed with Clement (aka Helping @kady out)

5 Replies

Over at the CBC the ever resourceful Kady O’Malley has posted documents from Statistics Canada surrounding the decision to make the long form of the census voluntary.

She’s starting to notice some interesting bits, here’s two I saw that she might want to add to the list.

First, there are two lines written by public servants that seems to run counter to Minister Clement’s defense of eliminating the long form last week

There is this statement referring to both the short and long form of the census:

Census information is used in planning schools, community health services, housing needs, daycare and emergency services and other important services for our communities.

Worse still, this next one refers explicitly to the long form (renamed the National Household Survey or NHS in this memo):

The NHS questionnaire will cover topics such as language, immigration, Aboriginal peoples, mobility, ethnicity, education, labour, income and housing.

The information in the NHS will provide data to support government programs directed at target populations. Information from the NHS will also support provincial/territorial and local government planning and program delivery.

So here is an official government memo that appears to run counter to Minister Clement’s argument last week that the only special interests were benefiting (and were thus opposed) to changes in the census. As we see here (and as I argued last week) the biggest users of this data are government who use it to ensure that programs (say for the elderly) are targeted effectively and so tax dollars used efficiently.

More importantly, the document seems to recognize that the provinces/territories and municipalities are huge stakeholders in this process – wouldn’t that suggest they should have been consulted before hand?

Indeed, Kady points this out in her piece by highlighting a comment that vainly tries to raise the flag that “stakeholders” (read other ministries, the provinces/territories, municipal governments, the bank of Canada, the list could go on for about 300+ organizations) should be consulted.

The other interesting piece from the documents that I noted was this hilarious comment (comment number 7) of which I’ve taken a screen shot.

I love the comment! “If this that important why not mandatory?

Ah the lonely voice of reason, hidden in a comment bubble of a MS Word document.

Huge credit to Kady O’Malley for doing the hard work of getting these documents and for being a grade journalist and posting them online. If you do link to this post, please also link to hers (again found here).

On Governments and Intellectual Property (or why we move slowly)

4 Replies

David H. sent me this short and fantastic article from Wired magazine last week.

The article discusses the travails of Mathew Burton, a former analyst and software programmer at the Department of Defense who spent years trying to get the software he wrote into the hands of those who desperately needed it. But alas, no one could figure out the licensing rights for the software it was supposed to work with… so it never went anywhere. Today Mathew has (unsurprisingly) left Defense and has open sourced the code so that anyone can use it. The lesson? The tangled mess of navigating all the license agreements isn’t protecting anyone and certainly not the public. It’s just preventing interesting new and derivative works from being used to render American safer.

In short, the crises here doesn’t have to do with size of government, but in a misplaced desire by many governments to protect “intellectual property.”

Now I understand the need of government to protect physical property. A forest, for example, can only be logged once every few generations, so allocating that resource efficiently matters. But intellectual property? Things like documents, data, and software code? It’s use is not diminished when someone uses it. Indeed, often its value increases when numerous people start to use it.

But rather than give to tax payers the intellectual property their tax dollars already paid for, our governments lock them down. Today, under the false belief that they are protecting themselves and potential revenue streams (that have never materialized) our governments copyright, patent and license all sorts of intellectual property our tax dollars paid for. In short, we treat ideas like we treat forests, something that only a handful of people can use and benefit from.

This has three happy consequences.

First, ideas and innovations are more expensive and spread more slowly. Remember the goal of innovation is not to license technology, its to use technology to enable us to be happier, safer or more productive (or ideally all three!). When our governments license technology that accomplishes one or all of these things they are, in fact, restricting the number of people who can benefit by giving a single actor a monopoly to sell this service (again, one tax payers funded to develop!) to tax payers or (worse) back to the government.

Second, we end up wasting a colossal amount of money on lawyers. With our governments pretending to be a corporation, managing all this intellectual property tax payers funded to develop, we naturally require an army of lawyers to protect and license it!

Finally, many governments are locked out of open source projects and communities. Since, by policy, many governments require that they own any code they, or their contractors develop, they cannot contribute to open source projects (in which the code is by definition, not owned but shared). This means free, scalable and customizable software and products that small companies like Google are forbidden within government. Instead they (and by they, I mean us) have to pay for proprietary solutions.

At some point I’d love to read more about how government got into the intellectual property businesses. I imagine it is a history paved with good intentions. However, the more I reflect on it, the more I wonder why the first order question of “why do governments have intellectual property” never gets asked. The costs are high and the benefits seem quite low. Maybe it’s time we radically rethink this.

The New Tall Tale of Tony Clement

16 Replies

“Yeah, there are groups that are upset. … Hey, listen, they had a good deal going,” Mr. Clement said Thursday following a meeting with fellow MPs in Ottawa.

“They got good, quality data and the government of Canada was the heavy. We were the ones who were coercing Canadians on behalf of these private businesses, or other social institutions, or other governments and provinces, for this data. We were the ones threatening Canadians with jail times or with large fines.”

– Minister Clement yesterday

Yes, Clement’s story about the census has changed again (here are the previous stories).

So which “groups” are we talking about. Yes, you’re mind may go to the hundreds of charities and non-profits, religious organizations, businesses that have spoken out. That is certainly who Minister Clement wants you to think about.

But, as I’ve said repeatedly, the biggest user of census data is not these groups (although Canadians will miss their services too).

The biggest user is… Government. And the group that will suffer the most will be Government (and by proxy, Canadians, who can expect a less effective government as a result).

So, think of the biggest user is groups like Human Resources and Skills Development Canada (the kind of people you want to be effective when your country is still shedding jobs from a recession), Environment Canada, Health Canada, etc…

And of course, the biggest second users of this data are the Provinces.

The third? The cities!

Also note that in each of these cases the Federal Government charges for this data so no one is getting a free ride financially. This is in contrast to many other governments, like the United States federal government, which makes its census data available for free.

So let’s be clear, this isn’t about the minister taking on every religious organization in the country, most of the non-profits and many, many businesses. It’s about blinding government.

Your Government *did* just get dumber… (that was fast)

17 Replies

Want to know who the biggest user of census data is? Government. To understand what services are needed, where problems or opportunities arise, or how a region is changing depends on having accurate data. The federal government, but also the provincial and, most importantly, local governments use Statistics Canada’s data every day to find ways to save taxpayers money, improve services and make plans. Now, at the very moment that governments are finding new ways to use this information more effectively than ever before, it is being cut off.

This is a direct attack on the ability of government to make smart decisions. It is an attack on evidence-based public policy. Moreover, it was a political decision – it came from the minister’s office and does not appear to reflect what Statistics Canada either wants or recommends. Of course, some governments prefer not to have information; all that data and evidence gets in the way of legislation and policies that are ineffective, costly and that reward vested interests (I’m looking at you, tough-on-crime agenda).

I wrote this on July 6th at the very beginning of the census scandal. What’s amazing is the short period of time it took for it to take on reality.

This week in a trainwreck of a press conference that pretty much every media outlet (save the ever loyal National Post) has mocked, Stockwell Day showed what the world of post-evidence based public policy will look like.

And what does it look like? Like a $5.1-billion a year increase in spending on prisons for a country with a declining crime rate in which 94% of Canadians survey feel safe.

Here is a scheme that only becomes defensible once you get rid of the evidence. Why? Because once you do that you can just make stuff up. Which is pretty much what the minister did. Take a look at John Geddes beautiful article which outlines how the Minister mislead the public about jail terms for criminals who conduct home invasions (they’ve gotten longer, not shorter).

Of course, for Conservatives the whole reason for getting rid of the census was that it was supposed to curtail big government. Stephen Taylor – Conservative blogger and cheerleader – says as much in his National Post Column. The beginning of the end of the Canadian welfare state. What was his line? “If it can’t be measured, future governments can’t pander.” It took about 9 days to disprove that thesis. A $5.1-billion dollar a year increase to create prison capacity for a falling crime rate is the case in point. Turns out even if you can’t measure it you can still do something about it. Just badly.

This isn’t the end of big government. It isn’t even the end of pandering governments. It’s just the beginning of blind government.

As an aside the people who should be most scared about this are the provincial governments. They just got made blind and didn’t even ask for it. It is also obvious that the Feds are going to push all sorts of spending on to them (like on prisons) that they didn’t ask for and don’t need. If they were smart, the provinces that have spoken out on the census (all of them except Alberta, BC and Saskatchewan) should announce they will conduct an independent census using the long form. This way they’ll actually have data to push back against the (now blind) federal government with. Better still, the provinces could license the aggregate data to make it free for everyone… except the feds, who when they come asking for the data (which of course they will) can be charge a big fat licensing fee. Perhaps a post worth fleshing out.