Category Archives: cool links

Great Hacks from the Open Data in Vancouver

12 Replies

Last weekend I helped host an Open Data Day in Vancouver. With the generous support of Domain7, who gave us a place to host talks and hack, over 30 Vancouverites braved the sleet and snow to spend the day sharing ideas and working on projects.

We had opening comments from Andy Yan – whose may be the most prolific user of Open Data in Vancouver, possibly Canada. I encourage you to check out his work here. We were also incredibly lucky to have Jeni Tennison – the Technical Director of the Open Data Institute – onsite to talk to participants about the ODI.

After the opening talks, people simply shared what they hoped to work on and people just found projects to contribute to. Minimal organization was involved… and here a taste of the awesome projects that got worked on! Lots of ideas here for other communities.

1. Open Data Licenses Resource: JSON + search + compatibility check = Awesome.

Kent Mewhort, who recently moved to Vancouver from Ottawa (via the Congo) updated his ongoing CLIPol project by adding some of the recently published licenses. If you’ve not seen CLIPol it is… awesome. It allows you to easily understand and compare the restrictions and rights of many open government licenses.

CLIPol Data

Better still CLIPol also lets you to see how compatible a license is (see example here). Possibly the best tool of all is one that allows you to determine what license you can apply to your re-mixed work in a way that is compliant with the original licenses (check out that tool here – screenshot below).

CLIPol compatibility

CLIPol is just such a fantastic tool – can’t recommend it enough and encourage people to add more licenses to it.

2. Vancouver in MineCraft

I have previously written about how Minecraft is being used to help in public consultations and urban planning – I love how the game becomes a simple tool that enables anyone to shape the environment.

So I was crazy excited I heard that Ryan Smith (aka Goldfish) had used the City of Vancouver’s open elevation data to recreate much of the city in Minecraft.

Below is a photo of Ryan presenting at the end of the day. The projection behind him shows Stanley park, near Siwash Rock. The flat feature at the bottom is the sea wall. Indeed Ryan notes that the sea wall makes for one of the clearest features since it creates almost perfectly flat structure along the city’s coast.

Mincraft Data

3. Vancouver’s Capital Budget Visualized in Where Does my Money Go

It is hard to imagine a project going better. I’m going to do a separate blog post on it.

This is a project I’ve always wanted to do – create a bubble tree visualization with Where Does my Money Go. Fortunately two developers – Alexandre Dufournet and Luc Lussier – who had never hacked on open data jumped on the idea. With help from City of Vancouver’s staff who were on site, I found a PDF of the capital budget which we then scraped.

WDMYG Data

The site is not actually live, but for developers who are interested in seeing this work (hint, hint City of Vancouver staff) you can grab their code from github here.

4. Monitoring Vancouver’s Bike Accident Data – Year 3

Eric Promislow has been coming to Open Data Hack-a-thons ever since Luke Closs and I started organizing them in 2009. During the first Open Data Day in 2011 you can read in my wrap up post about a bike accident monitoring website Eric created that day which Eric would eventual name Bent Frame. Well, Bent Frame has been live ever since and getting bigger. (Eric blogs about it here)

Each open data day, Eric updates Bent Frame with new data from ICBC – the province’s insurance monopoly. With over 6 years of data now in Eric is starting to be able to analyze trends – particularly around the decline of bike accidents along many roads with bike lanes, and an increase in accidents where the bike lanes end.


Bike Data

I initially had conversations with ICBC to persuade them to share their data with Eric and they’ve been in touch with him ever since, passing along the data on a regular basis. It is a real example of how an active citizen can change an organization’s policies around sharing important data that can help inform public policy debates.

5. ProactiveDisclosure.ca – Making government information easier to search

Kevin McArthur is the kind of security guy most governments dreads having around but should actually love (example his recent post on e-voting).  He continued to hack on one of his side projects: proactivedisclosure.ca. The site is a sort of front end for open data sets, making it easier to do searches based on people or companies. Thus, want to find all the open data about a specific minister… proactive disclosure organizes it for you.

Proactive Data

Kevin and a small team of players uploaded more data into their site and allowed it to consume unstructured data. Very cool stuff.

6. Better Open Data Search

Herb Lainchbury – another fantastic open data advocate – worked on a project in which he tried to rethink what an open data search engine would look like. This is a topic that I think matters A LOT. There is simply not a lot of good ways to find data that you are interested in.

Herb’s awesome insight was invert the traditional way of thinking about data search. He created a search engine that didn’t search for the data set keywords or titles, but rather searched the meta data exclusively.

One interesting side outcome of this approach is that it made related data sets easier and, made locating identical data sets but from different years a snap. As Herb notes the meta data becomes a sort of “finger print” that makes it easy to see when it has been duplicated. (Quick aside rant: I loath it when governments releases 20 data files of the same data set – say crime data – with each file representing a different year and then claiming that it is 20 unique data sets in their catalogue. No. It is one data set. You just have 20 years of it. Sigh).

7. School Performance Chart

Two local video game programers – Louie Dinh and Raymond Huang – with no experience in open data looked around the BC Government Open Data catalogue and noticed the data on test scores. Since they attended school here in British Columbia they thought it might be interesting to chart the test scores to see how their own schools had preformed over time.

They were able to set up a site which graphed how a number of elementary schools had performed over time by looking at the standardized test scores.

Test SCore Data

This is just a great example of data as a gateway to learning. Here a simple hackathon project become a bridge for two citizens to dive into a area of public policy and learn more about it. No one is claiming that there chart is definitive, rather it is the start of a learning process around what matters and what doesn’t and what can be measured and what can’t in education.

Congratulations to everyone who participated in the day – thank you for making it such an amazing success!

Open Data Day 2014 – Five Fun Events Around the World

Leave a reply

With over 110 Events happening world wide it is impossible to talk about every Open Data Day event. But looking almost every event on the wiki I’ve been deeply moved and inspired by the various efforts, goals and aspirations of the people who have organized these events.

In order to help others understand why Open Data Day matters as well as what can happen on it, here are five open data day events that I’m stumbled across that are doing something particularly fun or interesting.

1. Capetown & Johannesburg, South Africa

Their Description:

Coders, data wranglers and data investigators will pair up to look at one of three openly available datasets we have on hand, and work out the most interesting questions they can ask of it in less than half an hour. After 30 minutes, it’s all change – another desk, another partnership, another exciting data set to turn into a story.

What we’re hoping is that you’ll learn tips and tricks for getting data, querying it, creating quick visualizations and turning it to stories that people want to know about. You’ll learn from different people with a variety of skills, hopefully that you wouldn’t normally work with. And we’re also hoping it will be four hours of fun.

Why I love it: I love the focus on learning. With the participation of hacks and hackers the goal is clearly to help journalists and citizens learn new skills, not so they can do something with the data sets available on open data day, but so they can better play with data sets in the future to pursue stories or help a community. The point of speed data dating is thus not to build a product, the product is the skills and networks developed and, with luck the future stories and analyses that will be told by those who participated.

2. Buenos Aires, Argentina

Their description:

On February 22nd we will go out to the street and play with local data and some street artists to create beautiful visualizations.

Why I love it: Street art open data? What a great way to try to raise awareness of the importance of data literacy and transparency. In addition, how awesome is it to move outside the digital realm and use data to create artifacts that are not necessarily digital. And if there are artists involved? Jer would be so happy to read about this.

3. Greenfield, MA, United States

Their description:

We’re convening a small group to work with the Franklin Regional Council of Governments on a user-friendly way to map private wells in Western Massachusetts…

…Why is it important to map wells?

  • Only about 5% of private wells in Massachusetts are geolocated.
  • Many towns in Western Mass rely 100% on private wells.

Not knowing where our wells are can (and does) lead to water contaminated by nearby septic systems, dumping, and pollutant storage. Aside from the obvious health concerns, there are also financial implications from remediation costs and lowered property values.

Why I love it: Wow, WOW, WOW!!! This is maybe one of the coolest open data day events I’ve ever seen. Here you have a small community focusing on a problem that is real and tangible to them. Moreover, open data could have a direct and meaningful impact on the issue. I love the focus. I love that rallying point. I love the high impact with low resources (their building has minimal heat – so they are advising people to layer up). I wish these crew all the best success and hope to see an update.

4. Nagoya, Japan

Their Description:

Now, highlight of this year is “data of Nagoya Castle!”

The nearly 300 maps and survey drawings of the Nagoya Castle will be made open prior to Open Data Day. The Nagoya Castle office is cooperating with us and has decided that we can use their data for “International Open Data Day.” The references to the image are here.

Why I love it: Well – fill disclosure, my understanding of this event is through the prism of Google translate. But if I understood correctly… there are a few open data events in Japan that have a strong focus on local history which I find totally fascinating. At this event in Nagoya they are bringing in a professor who is an expert in open data as well as expert in the Nagoya castle to talk about the data that is being made open. In addition they are organizing and actual physical tour of the caste. Open Data meets local history buffs!

5. Cairo, Egypt

Their Description:

We will be organizing an online and decentralized event in Cairo, Egypt for the Open Data Day. There are numerous suggested tracks depending on the participants set of expertise:

For translators (المترجمون)

  • Open Data Handbook: The handbook discusses the legal, social and technical aspects of open data. It can be used by anyone but is especially designed for those seeking to open up data. It discusses the why, what and how of open data – why to go open, what open is, and the how to ‘open’ data. Translate it into Arabic here
  • Translate any of the School of Data short tutorials, for example, What is Data?, Telling a Story with Data, Finding Data, or Any other course/modules

For bloggers (المدونون)

Write blog posts about Open Data related topics and case studies, and don’t forget to use the following hashtag, #ODD2014. Possible ideas for blog posts:

  • Write about the concept of Frictionless Data
  • Case studies how you searched for, extracted and used governmental data
  • Listing of local organizations working or promoting Open Data or advocating for more Transparent and Open Governments

For Developers (مطوري البرامج)

  • Scrape data from capmas and put it into Open Format
  • Scrape data (Budget or the The Financial Monthly Bulletin) from the Minister of Finance and upload it to OpenSpending.org,
  • Create a tool to scrap the traffic data from bey2ollak and put it in an open format.

For Data Wranglers (هواة جمع البيانات)

Why I love it: I love that there are calls to action for a variety of people – including those who have no coding skills at all. How genius is it to organize an event to localize/translate the Open Data handbook? This is something a large number of people could do – and better still can help make open data accessible to a still larger pool of people.

And for the other roles the suggestion of projects – particular with a focus on the national budget and government operations data (capmas) suggests there is a strong civil society presence within the open data community. Will be super interested to see what progress they make and if there is broader interest in their work.

Open Data for Development Challenge on Jan 27-28

Leave a reply

This just came across my email via Michael Roberts who has been doing great work in this space.

Mail Attachment

Open Data for Development Challenge
January 27–28, 2014 — Montreal, Canada

Do you want to share your creative ideas and cutting-edge expertise, and make a difference in the world?
Do you want to help Canadians and the world understand how development aid is spent and what its impact is?
Do you want to be challenged and have fun at the same time?

If so, take the Open Data for Development Challenge!

This unique 36-hour ”codathon” organized by Foreign Affairs, Trade and Development Canada will bring together Canadian and international technical experts and policy makers to generate new tools and ideas in the fields of open data and aid transparency and contribute to innovative solutions to the world’s pressing development challenges.

The event will feature keynote speakers Aleem Walji, Director of the World Bank’s Innovation Labs, and Mark Surman, Executive Director of the Mozilla Foundation. It will have two related dimensions:

  • Technical challenges that involve building applications to make existing open aid and development-related data more useful. Proposed topics include building a data viewer compatible with multilingual data, creating a publishing tool suitable for use by mid-sized Canadian non-profit organizations, developing and testing applications for open contracting, and taking a deep dive into the procurement data of the World Bank Group. There is room for challenges proposed by the community. Proposals should be submitted through the event website no later than January 8th. Challenges will be published prior to the event, along with key datasets and other related information, to enable participants to prepare for the event.
  • Policy discussions on how open data and open government can enable development results. This would include the use of big data in development programming, the innovative ways in which data can be mapped and visualized for development, and the impact of open data on developing countries.

The international aid transparency community will be encouraged to take promising tools and ideas from the event forward for further research and development.

An overview of the draft program is attached. The event will be in English and French, with interpretation provided in the plenary sessions and panel discussions.

We invite you to register, at no cost, at this website as soon as possible and no later than January 10. A message confirming your registration and providing additional information about the venue and accommodation will be sent to confirmed participants. Please wait for this confirmation before making any travel arrangements. Participants are asked to make their own accommodation arrangements. A limited number of guest rooms will be available to event participants at a preferential rate.

To find out more about the Open Data for Development Challenge, please go to DFATD’s website.

Open Data Day 2014 is Coming Feb 22 – Time to Join the Fun!

4 Replies

So, with much help from various community members (who reminded me that we need to get this rolling – looking at you Heather Leson), I pleased to say we are starting to gear up for Open Data Day 2014 on February 22nd, 2014.

From its humble beginnings of a conversation between a few friends who were interested in promoting and playing with open data, last year Open Data Day had locally organized events take place in over 100 cities around the world. Check out this video of open data day in Kathmandu last year.

Why makes Open Data Day work? Mostly you. It is a global excuse for people in communities like yours to come together and organize an event that meets their needs. Whether that is a hackathon, a showcase and fair, lectures, workshops for local NGOs and businesses, training on data, or meetings with local politicians – people are free to organize around whatever they think their community needs. You can read more about how Open Data Day works on our website.

Want to join in on the fun? I thought you’d never ask. Listed below are some different ways you can help make Open Data Day 2014 a success in your community!

A) How can I let EVERYONE know about open data day

I love the enthusiasm. Here’s a tweet you can send:

#OpenData Day is community powered in a timezone near you.  http://opendataday.org/ #ODD2014

Yes, our hashtag is #ODD2014. Cause we are odd. And cause we love open data.

B) I’d like to participate!

Great! If you are interested in participating in check out the Open Data Day wiki. We’ve just unlocked the pages so cities haven’t been added yet but feel free to add your city to the list, and put down your name as interested in participating. You can even check to see who organized the event last year to see if they are interested in doing it again.

C) Forget about participating, I want to coordinate an Open Data Day event in my city.

Whoa! Very exciting! Here’s a short checklist of what to do:

  • If you didn’t organize one last year, check to see if anyone in your city did. It would be good to connect with them first.
  • Read the Open Data Day website. Basically, pick up on our vibe: we want Open Data Day to work for everyone, from novices who know little about data to experts like Kaggle participants and uber geeks like Bruce Schneier. These events have always been welcoming and encouraging – it is part of the design challenge.
  • Okay, now add your city to the list, let people know where it will be taking place (or that you are working on securing space), let them know a rough agenda, what to expect, and how they can contribute.
  • Add yourself to the 2014 Open Data Day map. (Hint: Wikipedia lists Lat/Long in the information side bar for each cities wiki page: “Coordinates: 43°42′N 79°24′W”)
  • Join the Open Data Day mailing list. Organizers tend to share best practices and tips here. It’s not serious, really just a help and support group.
  • Check out resources like this and this about how to organize a successful event.
  • Start spreading the news!

D) I want to help more! How can Open Data Day work more smoothly everywhere?

Okay, for the truly hardcore you right, we need help. Open Data day has grown. This means we’ve outgrown a whole bunch of our infrastructure… like our webpage! Everyone involved in this is a volunteer so… we have some extra heavy lifting we need help with. This includes:

a. Website template update: The current Open Data Day template was generously donated by Mark Dunkley (thank you!!!). We’d love to have it scale a little better and refresh the content. You can see the code on github here. Email me if you are interested. Skills required: css, design

b. Translation: Can you help translate the ODD site into your language? You can submit the requests on github or send a document to heather.leson at okfn dot org with the content. She’ll do the github stuff if that’s beyond you.

c. Map: Leaflet and layers helpers wanted! We’d like a map geek to help correct geolocation and keep the 2014 map fresh with accurate geo for all the locations. Github repo is here and the event list is here.

What’s next?

I’m really looking forward to this year… I’ve lots more thoughts I’ll be sharing shortly.

Plus, I can’t wait to hear from you!

Access to Information, Technology and Open Data – Keynote for the Commissioners

Leave a reply

On October 11th I was invited by Elizabeth Denham, the Access to Information and Privacy Commissioner for British Columbia to give a keynote at the Privacy and Access 20/20 Conference in Vancouver to an audience that included the various provincial and federal Information Commissioners.

Below is my keynote, I’ve tried to sync the slides up as well as possible. For those who want to skip to juicier parts:

  • 7:08 – thoughts about the technology dependence of RTI legislation
  • 12:16 –  the problematic approach to RTI implementation that results from these unsaid assumptions
  • 28:25 – the need and opportunity to bring open data and RTI advocates together

Some acronyms used:

StreetMix for testing bike lanes – Burrard St. Bridge Example

Leave a reply

I’m MCing the Code for America Summit at the moment, so short on time to write a post, but I’m just LOVING StreetMix so much I had to give it a shout out. If you are a councillor, urban planner or community activist, StreetMix is a site you HAVE to check out.

What does it do? I basically allows you to create or edit and street you want. It is so simple to use it takes about 1 minute to master. At that point, you can build, copy and redesign any street in the world.

Here, for example I’ve recreated the Burrard St. Bridge in Vancouver as it exists today, with bike lanes and below, as it existed before the addition bike lane.

Burrard Bridge new

Burrard Bridge old

The 311 Open Data Competition is now Live on Kaggle

2 Replies

As I shared the other week, I’ve been working on a data competition with Kaggle and SeeClickFix involving 311 data from four cities: Chicago, New Haven, Oakland and Richmond.

So first things first – the competition is now live. Indeed, there are already 19 teams and 56 submissions that have been made. Fortunately, time is on your side, there are 56 days to go.

As I mentioned in my previous post on the subject, I have real hopes that this competition can help test a hypothesis I have about the possibility of an algorithmic open commons:

There is, however, for me, a potentially bigger goal. To date, as far as I know, predictive algorithms of 311 data have only ever been attempted within a city, not across cities. At a minimum it has not been attempted in a way in which the results are public and become a public asset.

So while the specific problem  this contest addresses is relatively humble, I’d see it as a creating a larger opportunity for academics, researchers, data scientists, and curious participants to figure out if can we develop predictive algorithms that work for multiple cities. Because if we can, then these algorithms could be a shared common asset. Each algorithm would become a tool for not just one housing non-profit, or city program but a tool for all sufficiently similar non-profits or city programs.

Of course I’m also discovering there are other benefits that arise out of these competitions.

This last weekend there was a mini-sub competition/hackathon involving a subset of the data. It was amazing to watch from afar. First, I was floored by how much cooperation there was, even between competitors and especially after the competition closed. Take a look at the forums, they are probably make one of the more compelling cases that open data can help foster more people to want to learn how to manipulate and engage with data. Here are contestants sharing their approaches and ideas with one another – just like you’d want them to. I’d known that Kaggle had a interesting community and that learning played an important role in it, but “riding along” in a mini competition has caused me to look again at the competitions through a purely educational lens. It is amazing how much people both wanted to learn and share.

As in the current competition, the team at the hackathon also ran a competition around visualizing the data. And there were some great visualization of the data that came out of it, as well as another example of where people were trying to learn and share. Here are two of my favourites:

map2

I love this visualization by Christoph Molnar because it reveals the different in request locations in each city. In some they are really dense, whereas in others they are much (more) evenly distributed. Super interesting to me.

Most pressing issues in each city

I also love the simplicity of this image created by miswift. There might have been other things I’d done, like colour coded similar problems to make them easier to compare across cities. But I still love it.

Congratulations to all the winners from this weekends event, and I hope readers will consider participating in the current competition.

Beyond Property Rights: Thinking About Moral Definitions of Openness

2 Replies

“The more you move to the right the more radical you are. Because everywhere on the left you actually have to educate people about the law, which is currently unfair to the user, before you even introduce them to the alternatives. You aren’t even challenging the injustice in the law! On the right you are operating at a level that is liberated from identity and accountability. You are hacking identity.” – Sunil Abraham

I have a new piece up on TechPresident titled: Beyond Property Rights: Thinking About Moral Definitions of Openness.

This piece, as the really fun map I recreated is based on a conversation with Sunil Abraham (@sunil_abraham), the Executive Director of the Centre for Internet and Society in Bangalore.

If you find this map interesting… check the piece out here.

map of open

 

OGP Rules of the Game – Tactical Mistake or Strategic Necessity?

Leave a reply

The other week Martin Tisne, the UK Policy Director at the Omidyar Network, as well as one of the key architects of the Open Government Partnership (OGP), posted a blog post expressing concern that Civil Society participants have misunderstood the OGP. Specifically Tisne is concerned that by focusing on entrance into the OGP rather than on the process which requires them to fulfill commitments towards greater transparency, NGOs are making a tactical mistake.

There is a tremendous amount of good insight in Tisne’s piece and it deserves to be widely read (and has been). There are however, important reasons civil society members spend as much time fretting about entrance into the OGP rather than purely on the process. And contrary to Tisne, I don’t think this is a tactical mistake – it is, in fact, both a tactically and strategically sound choice. Most importantly of all it is a reflection of how power is structured and distributed within the OGP.

For most activists fostering change is about a developing a set of carrots and sticks that can be used to cajole a reluctant actor into making the change you seek. One big carrot is participation in the OGP. This is good. It urges governments to make commitments and sign on to a process. However, it also has a serious impact on civil society’s power in the process. This is because it puts one major carrot – participation – at the beginning of the process while placing the stick – an assessment of how well a government is adhering to its commitments – at the end.

We shouldn’t underestimate the benefit participation confers on many governments. The OGP brand can become a sort of shield that protects a government against all sorts of accusations of opacity. “Of course we are transparent, we participate in the OGP” is an easy line for minister to counter to an uncomfortable question. And that is not the only way participation can diminish civil society’s power. Because a government’s necessarily requires civil society cooperation (they sign off on the commitments), it binds the two together. This means that, in some basic way, civil society has endorsed a – yet to be implemented – government plan. That can provide enormous political cover. In addition, OGP members may cause some citizens (e.g. potential transparency supporters and activists) to adopt a “wait and see” approach to judging their government, or to assume that a reliable process is in place and so they can focus on other issues. Rather than maintain or intensify pressure on a government, the OGP, in the short term, may diminish the power of civil society.

The aforementioned stick in the OGP process is the independent reporting mechanism. And it arrives at the end of the process, a couple of years after the country has joined the OGP. The hope is it provides an objective assessment that civil society members can use to shame and drive for change where the assessment is critical. The challenge, and the reason I suspect many civil society members remain nervous, is that this mechanism remains mostly untested. The OGP carrot and stick model becomes even more challenging if either a) the timeline for fulfilling commitments falls onto the term of the next government or b) a transparency issue arises that runs counter to the OGPs values but falls outside the government’s action plan.  This is what happened in South Africa and so calling for ejection from the OGP became rational (and even necessary) since both the short term carrot (OGP participation) and long term stick – are review of the implementation plan – provided civil society with no leverage or power against a law that distinctly ran counter to the OGPs principles.

Consequently, the threat of striping a government of its OGP membership is not only a rational choice for many civil society members, in some cases it may be one of the few sticks available to them during a period in the process when other forms of influence have been made less effective. Threats of ejection is this not only a rational choice, but possible the only choice.

Indeed, OGP architects should take heart of the fact that civil society members are relatively hawkish about who gets to enter the OGP. As previously mentioned, OGP membership itself denotes a degree credibility – particularly to an unaware public. Civil society members bound to the OGP are potentially more invested in protecting the credibility and brand of the OGP than either the member governments of the OGP secretariat is. This is because, try as the OGP might to not compare countries to one another, civil society members know the company you keep matters.

This is not to say that the OGP should only be a high achievers club. I think the public understands there are differences in capacity, and the entrance of a country like Libya that is making a difficult transition, is broadly seen as positive. However, the participation of an authoritarian government, or even a democracy infamous for jailing journalists, significant corruption and little transparency – damages the the OGP brand for all participants, and particularly for civil society members participating in the process. I can only imagine the Executive Director of a civil society group grimacing as someone asks incredulously: “you are part of a transparency group that includes (insert country with poor record of your choice)?” Civil society actors that are the most invested in protecting the OGP’s brand, if only to ensure that the IRM has credibility when it is finally launched in their country. As such, protesting the potential entry of a country is not a tactical mistake, but a highly strategic decision.

I say this not because Martin is wrong, especially about his four points – civil society participation, OGP stretch goals, relevance check and the IRM – these are indeed critical to the bedrock of the OGP. And I remain exceedingly hopeful about the OGP, although a great deal hinges on the IRM and the degree with which it empowers local civil society actors. Rather I think it bears reminding all involved that we need to continuously have explicit and productive conversations about power, and how it is structured and where it flows, when it comes to the OGP process, as this reveals a lot about why actors act the way they do, and could provide insights in how we can make the OGP more effective.

Some Nice Journalistic Data Visualization – Global’s Crude Awakening

2 Replies

Over at Global, David Skok and his team have created a very nice visualization of the over 28,666 crude oil spills that have happened on Alberta pipelines over the last 37 years (that’s about two a day). Indeed, for good measure they’ve also visualized the additional 31,453 spills of “other” substance carried by Alberta pipeline (saltwater, liquid petroleum, etc..)

They’ve even created a look up feature so you can tackle the data geographically, by name, or by postal code. It is pretty in depth.

Of course, I believe all this data should be open. Sadly, they have to get at it through a complicated Access to Information Request that appears to have consumed a great deal of time and resources and that would probably only be possible by a media organizations with the  dedicated resources (legal and journalistic) and leverage to demand it. Had this data been open there would have still been a great deal of work to parse, understand and visualize it, but it would have helped lower the cost of development.

In fact, if you are curious about how they got the data – and the sad, sad, story it involved – take a look at the fantastic story they wrote about the creation of their oilspill website. This line really stood out for me:

An initial Freedom of Information request – filed June 8, 2012, the day after the Sundre spill – asked Alberta Environment and Sustainable Resource Development for information on all reported spills from the oil and gas industry, from 2006 to 2012.

About a month later, Global News was quoted a fee of over $4,000 for this information. In discussions with the department, it turned out this high fee was because the department was unable to provide the information in an electronic format: Although it maintained a database of spills, the departmental process was to print out individual reports on paper, and to charge the requester for every page.

So the relevant government department has the data in a machine readable form. It just chooses to only give it out in a paper form. Short of simply not releasing the data at all it is hard to imagine a more obstructionist approach to preventing the public from accessing environmental data their tax dollars paid to collect and that is supposed to be in the public interest. You essentially look at thousands of pieces of paper and re-enter tens, if not hundreds of thousands, of data points into spreadsheets. This is a process designed to prevent you from learning anything and frustrating potential users.

Let’s hope that when the time comes for the Global team to update this tool and webpage there will be open data they can download and access to the task is a little easier.