Category Archives: public policy

Policy-Making in a Big Data World

Leave a reply

For those interested I appeared on The Agenda with Steve Paikin the other week talking about Big Data and policy making.

There was a good discussion with a cast of character that included (not counting myself):

So much to dive into this space. There are, obviously, the dangers of thinking that data can solve all our problems, but I think the reverse is also true, that there is actually a real shortage of capacity within government (as in the private sector where these skills are highly sought after and compensated) to think critically about and effectively analyze data. Indeed, sadly, one of the few places in government that seems to understand and have the resources to work in this space is the security/intelligence apparatus.
It’s a great example of the growing stresses I think governments and their employees are going to be facing. One I hope we find ways to manage.

What Traffic Lights Say About the Future of Regulation

1 Reply

I have a piece up on TechPresident about some crazy regulations that took place in Florida that put citizens at greater risk all so the state and local governments can make more money.

Here’s a chunk:

In effect, what the state of Florida is saying is that a $20 million increase in revenue is worth an increase in risk of property damage, injury and death as a result of increased accidents. Based onnational statistics, there are likely about 62 deaths and 5,580 injuries caused by red light running in Florida each year. If shorter yellow lights increased that rate by 10 percent (far less than predicted by the USDOT) that could mean an additional 6 deaths and 560 injuries. Essentially the state will raise a measly extra $35,000 for each injury or death its regulations help to cause, and possibly far less.

Some Nice Journalistic Data Visualization – Global’s Crude Awakening

2 Replies

Over at Global, David Skok and his team have created a very nice visualization of the over 28,666 crude oil spills that have happened on Alberta pipelines over the last 37 years (that’s about two a day). Indeed, for good measure they’ve also visualized the additional 31,453 spills of “other” substance carried by Alberta pipeline (saltwater, liquid petroleum, etc..)

They’ve even created a look up feature so you can tackle the data geographically, by name, or by postal code. It is pretty in depth.

Of course, I believe all this data should be open. Sadly, they have to get at it through a complicated Access to Information Request that appears to have consumed a great deal of time and resources and that would probably only be possible by a media organizations with the  dedicated resources (legal and journalistic) and leverage to demand it. Had this data been open there would have still been a great deal of work to parse, understand and visualize it, but it would have helped lower the cost of development.

In fact, if you are curious about how they got the data – and the sad, sad, story it involved – take a look at the fantastic story they wrote about the creation of their oilspill website. This line really stood out for me:

An initial Freedom of Information request – filed June 8, 2012, the day after the Sundre spill – asked Alberta Environment and Sustainable Resource Development for information on all reported spills from the oil and gas industry, from 2006 to 2012.

About a month later, Global News was quoted a fee of over $4,000 for this information. In discussions with the department, it turned out this high fee was because the department was unable to provide the information in an electronic format: Although it maintained a database of spills, the departmental process was to print out individual reports on paper, and to charge the requester for every page.

So the relevant government department has the data in a machine readable form. It just chooses to only give it out in a paper form. Short of simply not releasing the data at all it is hard to imagine a more obstructionist approach to preventing the public from accessing environmental data their tax dollars paid to collect and that is supposed to be in the public interest. You essentially look at thousands of pieces of paper and re-enter tens, if not hundreds of thousands, of data points into spreadsheets. This is a process designed to prevent you from learning anything and frustrating potential users.

Let’s hope that when the time comes for the Global team to update this tool and webpage there will be open data they can download and access to the task is a little easier.

 

Awesome Simple Open Data use case – Welcome Wagon for New Community Businesses

3 Replies

A few weeks ago I was at an event in Victoria, British Columbia at event where people were discussing the possibilities, challenges and risk of open data. During the conversation, one of the participants talked about how they wanted an API for business license applications from the city.

This is a pretty unusual request – people have told me about their desire for business licenses data especially at the provincial/state and national level, but also at the local level. However, they are usually happy with a data dump once a year or quarter since they generally want to analyze the data for urban planning or business planning reasons. But an API – which would mean essentially constant access to the data and the opportunity to see changes to the database in real time (e.g. if a business registered or moved) – was different.

The reason? The individual – who was an entrepreneur and part of the local Business Improvement Area – wanted to be able to offer a “welcome wagon” to other new businesses in his community. If he knew when a business opened up he could reach out and help make them welcome them to the neighborhood. He thought it was always nice when shopkeepers knew one another but didn’t always know what was going on even a few blocks away because, well, he was often manning his own shop. I thought it was a deeply fascinating example of how open data could help foster community and is something I would have never imagined.

Food for thought and wanted to share.

 

The Value of Open Data – Don’t Measure Growth, Measure Destruction

7 Replies

Alexander Howard – who, in my mind, is the best guy covering the Gov 2.0 space – pinged me the other night to ask “What’s the best evidence of open data leading to economic outcomes that you’ve seen?”

I’d like to hack the question because – I suspect – for many people, they will be looking to measure “economic outcomes” in ways that I don’t think will be so narrow as to be helpful. For example, if you are wondering what the big companies are going to be that come out of the open data movement and/or what are the big savings that are going to be found by government via sifting through the data, I think you are probably looking for the wrong indicators.

Why? Part of it is because the number of “big” examples is going to be small.

It’s not that I don’t think there won’t be any. For example several years ago I blogged about how FOIed (or, in Canada ATIPed) data that should have been open helped find $3.2B in evaded tax revenues channeled through illegal charities. It’s just that this is probably not where the wins will initially take place.

This is in part because most data for which there was likely to be an obvious and large economic impact (eg spawning a big company or saving a government millions) will have already been analyzed or sold by governments before the open data movement came along. On the analysis side of the question- if you are very confident a data set could yield tens or hundreds of millions in savings… well… you were probably willing to pay SAS or some other analytics firm 30-100K to analyze it. And you were probably willing to pay SAP a couple of million (a year?) to set up the infrastructure to just gather the data.

Meanwhile, on the “private sector company” side of the equation – if that data had value, there were probably eager buyers. In Canada for example, interest in census data – to help with planning where to locate stores or how to engage in marketing and advertising effectively – was sold because the private sector made it clear they were willing to pay to gain access to it. (Sadly, this was bad news for academics, non-profits and everybody else, for whom it should have been free, as it was in the US).

So my point is, that a great deal of the (again) obvious low hanging fruit has probably been picked long before the open data movement showed up, because governments – or companies – were willing to invest some modest amounts to create the benefits that picking those fruit would yield.

This is not to say I don’t think there are diamonds in the rough out there – data sets that will reveal significant savings – but I doubt they will be obvious or easy finds. Nor do I think that billion dollar companies are going to spring up around open datasets over night since –  by definition – open data has low barriers to entry to any company that adds value to them. One should remember it took Red Hat two decades to become a billion dollar company. Impressive, but it is still a tiny compared to many of its rivals.

And that is my main point.

The real impact of open data will likely not be in the economic wealth it generates, but rather in its destructive power. I think the real impact of open data is going to be in the value it destroys and so in the capital it frees up to do other things. Much like Red Hat is fraction of the size of Microsoft, Open Data is going to enable new players to disrupt established data players.

What do I mean by this?

Take SeeClickFix. Here is a company that, leveraging the Open311 standard, is able to provide many cities with a 311 solution that works pretty much out of the box. 20 years ago, this was a $10 million+ problem for a major city to solve, and wasn’t even something a small city could consider adopting – it was just prohibitively expensive. Today, SeeClickFix takes what was a 7 or 8 digit problem, and makes it a 5 or 6 digit problem. Indeed, I suspect SeeClickFix almost works better in a small to mid-sized government that doesn’t have complex work order software and so can just use SeeClickFix as a general solution. For this part of the market, it has crushed the cost out of implementing a solution.

Another example. And one I’m most excited. Look at CKAN and Socrata. Most people believe these are open data portal solutions. That is a mistake. These are data management companies that happen to have simply made “sharing (or “open”) a core design feature. You know who does data management? SAP. What Socrata and CKAN offer is a way to store, access, share and engage with data previously gathered and held by companies like SAP at a fraction of the cost. A SAP implementation is a 7 or 8 (or god forbid, 9) digit problem. And many city IT managers complain that doing anything with data stored in SAP takes time and it takes money. CKAN and Socrata may have only a fraction of the features, but they are dead simple to use, and make it dead simple to extract and share data. More importantly they make these costly 7 and 8 digital problems potentially become cheap 5 or 6 digit problems.

On the analysis side, again, I do hope there will be big wins – but what I really think open data is going to do is lower the costs of creating lots of small wins – crazy numbers of tiny efficiencies. If SAP and SAS were about solving the 5 problems that could create 10s of millions in operational savings for governments and companies then Socrata, CKAN and the open data movement is about finding the 1000 problems for which you can save between $20,000 and $1M in savings. For example, when you look at the work that Michael Flowers is doing in NYC, his analytics team is going to transform New York City’s budget. They aren’t finding $30 million dollars in operational savings, but they are generating a steady stream of very solid 6 to low 7 digit savings, project after project. (this is to say nothing of the lives they help save with their work on ambulances and fire safety inspections). Cumulatively  over time, these savings are going to add up to a lot. But there probably isn’t going to be a big bang. Rather, we are getting into the long tail of savings. Lots and lots of small stuff… that is going to add up to a very big number, while no one is looking.

So when I look at open data, yes, I think there is economic value. Lots and lots of economic value. Hell, tons of it.

But it isn’t necessarily going to happen in a big bang, and it may take place in the creative destruction it fosters and so the capital it frees up to spend on other things. That may make it potentially harder to measure (I’m hoping some economist much smarter than me is going tell me I’m wrong about that) but that’s what I think the change will look like.

Don’t look for the big bang, and don’t measure the growth in spending or new jobs. Rather let’s try to measure the destruction and cumulative impact of a thousand tiny wins. Cause that is where I think we’ll see it most.

Postscript: Apologies again for any typos – it’s late and I’m just desperate to get this out while it is burning in my brain. And thank you Alex for forcing me to put into words something I’ve been thinking about saying for months.

 

Canada Post and the War on Open Data, Innovation & Common Sense (continued, sadly)

5 Replies

Almost exactly a year ago I wrote a blog post on Canada Post’s War on the 21st Century, Innovation & Productivity. In it I highlighted how Canada Post launched a lawsuit against a company – Geocoder.ca – that recreates the postal code database via crowdsourcing. Canada Posts case was never strong, but then, that was not their goal. As a large, tax payer backed company the point wasn’t to be right, it was to use the law as a way to financial bankrupt a small innovator.

This case matters – especially to small start ups and non-profits. Open North – a non-profit on which I sit on the board of directors – recently explored what it would cost to use Canada Posts postal code data base on represent.opennorth.ca, a website that helps identify elected officials who serve a given address. The cost? $9,000 a year, nothing near what it could afford.

But that’s not it. There are several non-profits that use Represent to help inform donors and other users of their website about which elected officials represent geographies where they advocate for change. The licensing cost if you include all of these non-profits and academic groups? $50,000 a year.

This is not a trivial sum, and it is very significant for non-profits and academics. It is also a window into why Canada Post is trying to sue Geocoder.ca – which offers a version of its database for… free. That a private company can offers a similar service at a fraction of the cost (or for nothing) is, of couse, a threat.

Sadly, I wish I could report good news on the one year anniversary of the case. Indeed, I should be!

This is because what should have been the most important development was how the Federal Court of Appeal made it even more clear that data cannot be copyrighted. This probably made it Canada Post’s lawyers that they were not going to win and made it even more obvious to us in the public that the lawsuit against geocoder.ca – which has not been dropped-  was completely frivolous.

Sadly, Canada Post reaction to this erosion of its position was not to back off, but to double down. Recognizing that they likely won’t win a copyright case over postal code data, they have decided:

a) to assert that they hold trademark on the words ‘postal code’

b) to name Ervin Ruci – the opertator of Geocoder.ca – as a defendent in the case, as opposed to just his company.

The second part shows just how vindictive Canada Post’s lawyers are, and reveals the true nature of this lawsuit. This is not about protecting trademark. This is about sending a message about legal costs and fees. This is a predatory lawsuit, funded by you, the tax payer.

But part a is also sad. Having seen the writing on the wall around its capacity to win the case around data, Canada Post is suddenly decided – 88 years after it first started using “Postal Zones” and 43 years after it started using “Postal Codes” to assert a trade mark on the term? (You can read more on the history of postal codes in canada here).

Moreover the legal implications if Canada Post actually won the case would be fascinating. It is unclear that anyone would be allowed to solicit anybody’s postal code – at least if they mentioned the term “postal code” – on any form or website without Canada Posts express permission. It leads one to ask. Does the federal government have Canada Post’s express permission to solicit postal code information on tax forms? On Passport renewal forms? On any form they have ever published? Because if not, they are, I understand Canada Posts claim correctly, in violation of Canada Post trademark.

Given the current government’s goal to increase the use of government data and spur innovation, will they finally intervene in what is an absurd case that Canada Post cannot win, that is using tax payer dollars to snuff out innovators, increases the costs of academics to do geospatial oriented social research and that creates a great deal of uncertainty about how anyone online be they non-profits, companies, academics, or governments, can use postal codes.

I know of no other country in the world that has to deal with this kind of behaviour from their postal service. The United Kingdom compelled its postal service to make postal code information public years ago.In Canada, we handle the same situation by letting a tax payer subsidized monopoly hire expensive lawyers to launch frivolous lawsuits against innovators who are not breaking the law.

That is pretty telling.

You can read more about this this, and see the legal documents on Ervin Ruci’s blog.  has also done a good job covering this story at canada.com.

How not to sell the Oil Sands

4 Replies

If you haven’t read Tzeporah Berman’s Daily Kos piece – My Government Doesn’t Believe in Climate Change – go check it out. It’s amazing to see how out of sync, and behind the ball, the government has gotten on this issue.

Indeed, the current government really is becoming the best weapon opponents to the pipelines have against their construction and the further development of the oil sands.

First, the Natural Resource Minister called environmentalists who opposed Northern Gateway – the pipeline that would take Oil Sand crude from Alberta to British Columbia’s west coast – a “radical”. In short order it turns out that, well, the vast majority of British Columbians were radicals since opposition to the pipeline has been, and remains, strong.

Worse, by declaring war on environmentalists and those concerned about climate change, the Minister set a table that would become distinctly awkward when it came to trying to get the White House and State Department to approve Keystone XL – the pipeline that would export oil sands oil down to the US. Consider that the US President mentioned his concerns about climate change in his State of the Union address and the US ambassador to Canada has hinted that Canada’s commitment to addressing climate change may factor into their decisions around Keystone. So now, at the very moment the Canadian Government needs help to persuade the Americans they actually are serious about climate change, they just come out of a five year patch where they’ve labeled everyone who disagrees with it as a radical, sicced auditors on them and tried to shut many of them down. So suddenly they are in a difficult position of having few, if any, allies in the environmental movement they can turn to to help it gain credibility around any of the actions they wish to take. It’s a communications and policy disaster.

Of course, it doesn’t help that even in the midst of trying to convince the United States it actually is serious about Climate Change, the Natural Resource Minister has speaking notes that feature strong climate change denier comments. Elizabeth May, Green Party candidate in the house of commons summed up the Minister Oliver’s quotes – made to the editorial board of La Presse – quite nicely:

This is what Oliver told the editorial board of La Presse: “I think that people aren’t as worried as they were before about global warming of two degrees… Scientists have recently told us that our fears (on climate change) are exaggerated.”

Thank goodness the editorial board at La Presse knows how to ask questions. They pressed him to name any scientist who thinks our fears are exaggerated. He couldn’t.

So basically, we have a minister who is willing to go on record to make a claim about the future of the planet without being able to reference a single source. I’m sure the White House and State Department are deeply comforted.

It is all doubly amazing as this government has been masterful at handling communications around virtually every issue – except for the one that matters most to it, where it has suddenly becoming a bumbling idiot.

 

 

Toronto Star Op-Ed: Muzzled Scientists, Open Government and the Limits of Rules

2 Replies

I’ve a piece in today’s Toronto Star “Rules are no substitute for cultivating a culture of open government” about the Information Commissioners decision to investigate the muzzling of Canadian scientists.

Some choice paragraphs:

The actions of the information commissioner are to be applauded; what is less encouraging are the limits of her ability to resolve the problem. The truth is that openness, transparency and accountability cannot be created by the adoption of new codes or rules alone.

This is because even more than programs and regulations, an open government is the result of culture, norms and leadership. And here the message — felt as strongly by government scientists as any other public servants — is clear. Public servants are allowed less and less to have a perspective, to say nothing of the ability to share that perspective.

and on ways I think this has consequences that impact the government’s agenda directly:

This breakdown in culture has consequences — some of which may impact the government’s most important priorities. Take, for example, the United States’ preoccupation with Canada’s environmental record in general and its specific concerns about the oilsands in regards to the proposed Keystone XL pipeline. The government has spent the last month trying to burnish its environmental record in anticipation of the decision. And yet, it is amazing how few in Ottawa recognize the direct link between the openness around which government scientists can speak about their work and the degree of trust that Canadians — as well as our allies — have in our capacity to protect the environment.

I hope you’ll give it a read.

Open Data Day: Lessons for Hacktivists

Leave a reply

This piece is cross-posted on TechPresident where I post articles on the intersection of politics, technology and transparency and serve as an editor.

Three years ago, after a chance encounter with Daniela Silva and Pedro Markun of Sao Paulo and a meeting with Edward Ocampo-Gooding and Mary Beth Baker in Ottawa, with whom I shared a passion about open data, we agreed to simultaneously host events in our three cities on the same day. It would be a hackathon, and because it would take place in at least two countries … we liberally called it “international” inviting others to join us.

In that first year we had about six cities conduct events on every continent save Australia. Now in its third year, Open Data Day events is far bigger than we ever dared imagine. More interesting still is its impact, both expected and unexpected.

Reflecting on it all, I thought it might be worthwhile to share a little bit about the impact I think Open Data has, and some lessons hacktvists may find interesting to draw upon.

Build Community

From the beginning our goal was simple. It wasn’t code, or even opening data per se. It was about community.

We wanted to foster a friendly event where anyone would feel welcome to participate. The team in Ottawa had, in particular, done great work in reaching out an engaging new people in previous hackathons, they’d have a wealth of non-software developers and even full families attend their events. This was as much a hackathon as it was a community event.

So Open Data Day was always meant to serve as a catalyst for community and network creation. Yes, creating, adding to or working on a project was strongly encouraged, but the real output to help open data advocates find and connect with one another, as well as grow the movement by engaging new people. An effective community was always going to be the core ingredient for petitioning a government to make data open, teaching students or policy makers how to use data or have a group of developers launch a project. While we wanted to create cool software, visualizations and analysis, what I wanted even more was to foster local leaders, champions, social glue and community hubs.

And that’s what we got.

One of the strengths of Open Data Day is its incredible decentralized nature. I blog about it to encourage people to organize and help moderate a mailing list, but beyond that everything is done by local volunteers. From a capacity building perspective, it is an incredible event to watch unfold. I’d like to think Open Data Day has played a helpful role in connecting local stakeholders and even knitting them together with regional, national and international peers.

While I believe that a beautiful piece of code can be critical in making policy makers, the public or others see the world in a different light I also think that building community and developing allies who can share that story of that code with leaders and the public at large is, depending on the breadth of your goals, equally important.

What Works for Communities Can Work for Governments

Almost immediately upon launch of Open Data Day, government officials began showing up at the hackathons. Some came unofficially, others officially, and in some places, the hackathons were invited to take place at city hall. Because we set a venue Open Data Day created a predictable publicized space where governments that were curious, reluctant, eager or cautious could come and engage at a speed that worked for them.

When you build a safe place for a broader community – if your event has more of the feel of a public consultation or community meet-up than a gathering of subversive technologists – you can create space for governments. More importantly, the one thing that characterizes most open data hackathons I’ve witnessed or investigated,is that the participants share a desire to make their community better. That is something most government officials can easily wrap their heads around.

Thus, in a way that I don’t think anyone planned, Open Data Day events have become a place where governments want to understand more of what is possible, learn about the community that is interested in data and wrestle with how open data can change the way they work. What began with a few pioneering cities three years ago has evolved where now places like the Victoria Palace in Romania (the seat of government) and the White House in Washington DC are hosting Open Data Day events.

This type of engagement creates new opportunities, and new challenges, but that is exactly the type of progress many movements would like to see.

Plant a Flag

Setting a date and having people around the world step up and embrace had another interesting impact. It created a deadline not just for community organizers who were organizing their local open data day event, but also for governments that want to engage these communities.

Indeed in the past it has been amazing to see how many governments now see Open Data Day as a deadline for launching open data portals or releasing additional data sets. This is a fantastic outcome as it creates subtle pressure on governments to act.

This year was no exception. I heard of governments around the world releasing data sets in anticipation of Open Data Day. In particular there was a flurry of activity from European governments and agencies. Indeed the European Union chose to launch its open data portal in time for Open Data Day. But there was also much activity across the continent. This included that launch of data portals for the Building Performance Institute Europe, the Italian Senate, the city of Venice , the city ofTrento, the region of Puglia in the south of Italy, and the province of Bolzano. But the practice has become widespread. I was particularly happy to see a city a few kilometers from my home town do the same: the City of Victoria, BC launched its Open Data portal for the day.

These announcements show the amplifying effect of getting organized across geographies. The advocacy work of a group in one country helps reinforce a message that benefits advocates in other countries. This is pretty basic stuff — political advocacy 101 if you will — but it is a reminder of why congregating around a single date, and meeting in person still have value for those who are unpersuaded.

Ultimately Open Data Day is important because it serves the needs of local advocates and activists wherever they are. That means keeping the event coherent so that it can continue to be a global event that people understand, yet flexible so that it can satisfy local needs and desires. For example, the event in Manila focused on poverty while in Washington DC it focused on building community and capacity. These types of events are not, in of themselves, going to completely solve a problem, but I do believe that they are part of a broader political effort that activates a community, engages government and creates pressure for change.

How Hackers Will Blow Up The World: China, Cyber-Warfare and the Cuban Missile Crisis

1 Reply

I have a piece on TechPresident I really enjoyed writing about how certain technologies – as they become weaponized – can in turn become highly destabilizing to global stability. The current rash of Cyber-Warfare, or Cyber-Spying or Cyber-crime (depending on the seriousness and intent with which you rate it) could be one such destabilizing technology.

Here’s a long excerpt:

This would certainly not be the first time technology altered a balance of military power and destabilized global political orders everyone thought was robust. One reason the world plunged into global war in 1914 after a relatively minor terrorist attack — the assassination of Arch-Duke Ferdinand — was because the hot new technology of the day, the speedy railway, caused strategists to believe it would confer a decisive advantage on those who mobilized first. The advent of nuclear intercontinental ballistic missiles of the 1950s had a similar effect, with fears that a first strike “decapitation attack” against Moscow from Turkey, or against Washington from Cuba, could preempt a counter attack.

Cyber warfare may be evolving into a similarly destabilizing type of technology. Prior to the 21st century, cyber attacks were relatively localized affairs. People imagined the main threats of a cyber attack being with virtual thefts from banks, identify theft against individuals and even industrial piracy. Serious problems to be sure, but not end-of-the-world stuff. Even when targeted against the state, cyber attacks rarely pose an existential threat to a country. The loss of state secrets, the compromising of some officials could, cumulatively, be corrosive on a state’s ability to defend itself or advance its interests, but it was unlikely even a combination of operations would shake a mature state to its core.

Two things have changed….

You can read the full piece here. Always love feedback.