Monthly Archives: May 2013

Some Nice Journalistic Data Visualization – Global’s Crude Awakening

Over at Global, David Skok and his team have created a very nice visualization of the over 28,666 crude oil spills that have happened on Alberta pipelines over the last 37 years (that’s about two a day). Indeed, for good measure they’ve also visualized the additional 31,453 spills of “other” substance carried by Alberta pipeline (saltwater, liquid petroleum, etc..)

They’ve even created a look up feature so you can tackle the data geographically, by name, or by postal code. It is pretty in depth.

Of course, I believe all this data should be open. Sadly, they have to get at it through a complicated Access to Information Request that appears to have consumed a great deal of time and resources and that would probably only be possible by a media organizations with the dedicated resources (legal and journalistic) and leverage to demand it. Had this data been open there would have still been a great deal of work to parse, understand and visualize it, but it would have helped lower the cost of development.

In fact, if you are curious about how they got the data – and the sad, sad, story it involved – take a look at the fantastic story they wrote about the creation of their oilspill website. This line really stood out for me:

An initial Freedom of Information request – filed June 8, 2012, the day after the Sundre spill – asked Alberta Environment and Sustainable Resource Development for information on all reported spills from the oil and gas industry, from 2006 to 2012.

About a month later, Global News was quoted a fee of over $4,000 for this information. In discussions with the department, it turned out this high fee was because the department was unable to provide the information in an electronic format: Although it maintained a database of spills, the departmental process was to print out individual reports on paper, and to charge the requester for every page.

So the relevant government department has the data in a machine readable form. It just chooses to only give it out in a paper form. Short of simply not releasing the data at all it is hard to imagine a more obstructionist approach to preventing the public from accessing environmental data their tax dollars paid to collect and that is supposed to be in the public interest. You essentially look at thousands of pieces of paper and re-enter tens, if not hundreds of thousands, of data points into spreadsheets. This is a process designed to prevent you from learning anything and frustrating potential users.

Let’s hope that when the time comes for the Global team to update this tool and webpage there will be open data they can download and access to the task is a little easier.

Awesome Simple Open Data use case – Welcome Wagon for New Community Businesses

3 Replies

A few weeks ago I was at an event in Victoria, British Columbia at event where people were discussing the possibilities, challenges and risk of open data. During the conversation, one of the participants talked about how they wanted an API for business license applications from the city.

This is a pretty unusual request – people have told me about their desire for business licenses data especially at the provincial/state and national level, but also at the local level. However, they are usually happy with a data dump once a year or quarter since they generally want to analyze the data for urban planning or business planning reasons. But an API – which would mean essentially constant access to the data and the opportunity to see changes to the database in real time (e.g. if a business registered or moved) – was different.

The reason? The individual – who was an entrepreneur and part of the local Business Improvement Area – wanted to be able to offer a “welcome wagon” to other new businesses in his community. If he knew when a business opened up he could reach out and help make them welcome them to the neighborhood. He thought it was always nice when shopkeepers knew one another but didn’t always know what was going on even a few blocks away because, well, he was often manning his own shop. I thought it was a deeply fascinating example of how open data could help foster community and is something I would have never imagined.

Food for thought and wanted to share.

Duffy, the Government and the problem with “no-notes” meetings

1 Reply

So, for my non-canadian readers, there is a significant scandal brewing up here in Canadaland, regarding a senator, who claimed certain expenses he was not allowed to (to the tune of $90,000) and then had that debt paid for by the Prime Minister’s chief of staff (who has now resigned).

This short paragraph in an article by the star captures the dark nature of the exchange:

…once the repayment was made, Duffy stopped cooperating with independent auditors examining his expense claims. When a Senate committee met behind closed doors in early May to write the final report on the results of the audit, Conservatives used their majority to soften the conclusions about Duffy’s misuse of taxpayers’ money.

So, it was money designed to make a potential minor scandal go away.

Now the opposition party ethics critic Charlie Angus is calling for an RCMP investigation. From the same story:

“Where is the paper trail? Who was involved,” Angus said.

“If there is a signed agreement between Mr. Duffy and Mr. Wright or Mr. Duffy and the Prime Minister’s Office, we need to see that documentation,” he said.

And herein lies an interesting rub. This government is infamous for holding “no-note” meetings. I’ve been told about such meetings on numerous occasions by public servants. Perhaps they are making it up. But I don’t think so. The accusations have happened too many times.

So maybe there is a paper trail between the Prime Minister’s Office and Senator Duffy. There were lawyers involved… so one suspects there was. But who knows. Maybe there isn’t. And the lack of a paper trail won’t give many people confidence. Indeed, with Duffy now no longer in the Conservative Caucus and with the Chief of Staff resigned everyone is now looking at the Prime Minister, people are now starting to focus on the Prime Minister. What did he know?

And therein lies the bigger rub. In an environment where there is no paper trail one cannot audit who knew what or who is responsible. This means everyone is responsible, all the way to the top. So a “no-notes” meeting can be good for keeping the public from knowing certain decisions, and who made them, but the approach fails once the public finds out about one of these decisions and starts to care. If there is no smoking email in which someone else claims responsibility, it is going to be hard for this not to reach the Prime Minister’s desk.

Politicians and political staff will sometimes forget that the rules and processes around meetings – which appear to be designed to simply promote a degree of (annoying to them) transparency and accountability – are as much about creating a record for the public as they are about protecting the people involved.

Thoughts on the White House Executive Order on Open Data

4 Replies

As those steeped in the policy wonk geekery of open data are likely already aware, last Thursday the President of the United States issued an Executive Order Making Open and Machine Readable the New Default for Government Information.

This is, quite frankly, a big deal. Further down in the post I’ve got some links and some further explanations why.

That said, the White House called and asked if I would be willing to provide some context about the significance of the order – which I did. You can read my reaction, along with those of a number of people I respect, here. Carl Malamud is, as always, the most succinct and dramatic.

Here are some further thoughts:

Relevant Links

A link to the press release

White House CTO, Todd Park, and CIO, Steve VanRoekel, explaining Open Government Data and its significance.

A fact sheet on the announcement

A link to the Executive Order

A policy memo about the Executive

And, perhaps most interestingly, a link to Project Open Data, a site with a TON of resources about doing open data within government, including job descriptions, best practices and even tools (e.g. code) you can download to help with an open data deployment in at the city, state or national level. Indeed if you are a public servant reading this (and, I know many of you are), I strongly encourage you to take a look at this site. Tools here could save you tens to hundreds of thousands of dollars in software development costs for a range of projects. I love this example, for one: Kickstart – “A simple WordPress plugin to help agencies kickstart their open data efforts by allowing citizens to browse existing datasets and vote for suggested priorities.”

What the White House did right

Here is the genius of this executive order. At its core it deals with something that is hard to communicate to a lot of people in a meaningful way. Here is the executive order for dummies version: This is essentially a core change to procurement and information publication. From a procurement perspective it basically means from now on, if you work in the US government and you buy a computer or software that is going to store or collect data, it sure as hell better be able to export it in a way that others can re-use it. From a information publication perspective, having the ability to publish the data is not sufficient, you actually have to publish the data.

This change is actually quite wide ranging. So much so that it could be hard for many people to understand its significance. This is why I love the emphasis on what I would refer to as strategic data sets – data sets on healthcare, education, energy and safety. While the order pertains to data that is much, much broader than this, talking about datasets like the 5-Star Safety Ratings System about almost every vehicle in America or data on most appliances’ Energy Star rating brings it down to earth. This is information the average American can wrap their head around and agree should be made more widely available.

The point is that while I’m in favour of making government data more available, I’m particularly interested in using it to drive for policy outcomes that are in the public interest. Finding better ways to get people safety, health, energy or education data in their hands at the moment they are making an important decision is something open data can facilitate. If, when you are making a purchase or about to create a new project, there is some software that can filter your choices by safety rating or prompt you to rethink your criteria in a way that will enhance your safety or reduce your carbon footprint, I find that compelling. So more availability to government data for research or even just access… yes! But access to specific data sets with the goal of improving specific outcomes is also very important, and this is clearly one of the goals of this order.

What this executive order is not

It is important to note what this Executive Order is not. While I think it can help citizens make better choices, improve access to some types of information, offer researchers and policy wonks more data to test theories and propose solutions and improve productivity within and outside government, I do not think it will not change politics in America. Had this order existed, it would not have magically prevented the Iraq War by, for example, making CIA analysis more scrutable. Nor will it directly rein in lobbyists or make money matter less. This is more about changing the way government works, not the effect that politics has on government decisions. Maybe it will have that impact in the long run (or the opposite impact), but it will be through second and third order effects that I’m all too happy to confess that I currently don’t see.

No one is claiming that this release somehow makes the US government “open” – there are still lots of examples about policies and processes in the White House that require greater transparency. Transparency and openness in government move on several axes. Progress along one axis does not automatically mean there is progress along all axes. And even progress will foster new challenges and demand new types of vigilance. For example, I also suspect, over time, the order may impact what data governments elect to collect, if, by default, it is to be made open. The order could, in some cases, make the data more political, something I’ve argued here.

This is to say that there is no panacea and this order does not create some perfectly transparent government. But it is an important step, and one that other governments should be looking at closely. It is an effort to reposition government to better participate in and be relevant in a data driven and networked world, and it does foster a level of access around a class of information, data, that is too often kept hidden from citizens. For that reason, it is worthy of much praise.

LinkedIn is as confused about what I do as everyone else

eaves.ca

if writing is a muscle, this is my gym