Category Archives: cool links

Brain Candy – Great Quotes from Yesterday

I’m in San Francisco to co-chair the Code for America Summit this week, so lots going on, and some deep blog posts in the works. But first. Fun! Here are some of my favourite quotes I stumbled upon or heard in the last 24 hours.

“The 4-Hour Body” reads as if The New England Journal of Medicine had been hijacked by the editors of the SkyMall catalog.

– Dwight Garner, in the New York Times review of the Four Hour Body by Timothy Ferriss

The entire review is pure genius. Definitely worth reading.

But more quotes await!

“Micro-managing isn’t that third thing that Amazon does better than us, by the way. I mean, yeah, they micro-manage really well, but I wouldn’t list it as a strength or anything. I’m just trying to set the context here, to help you understand what happened. We’re talking about a guy [Jeff Bezos] who in all seriousness has said on many public occasions that people should be paying him to work at Amazon. He hands out little yellow stickies with his name on them, reminding people “who runs the company” when they disagree with him. The guy is a regular… well, Steve Jobs, I guess. Except without the fashion or design sense. Bezos is super smart; don’t get me wrong. He just makes ordinary control freaks look like stoned hippies.”

– Steve Yegge in a now no longer public but still accessible assessment of why Google doesn’t get platforms.

The broader read is fantastic, but this quote – mentioned to me by a friend – I thought was both fun and insightful. There is something to be said for super obsessive bosses. They care about their business. It is worth noting that both Jobs and Bezos founded their companies. A lot of other companies could do with this kind of love and attention – even if, in high doses, it can be totally toxic. It’s a fascinating tension.

So yes, tech and the four hour work week? I must be proximity to the valley… so let’s get away from that.

How about #occupywallst? There is a very interesting analysis of the data behind the We are the 99% tumblr feed over at rortybomb, definitely worth a read. But I was really struck by this quote about the nature of the demands:

The people in the tumblr aren’t demanding to bring democracy into the workplace via large-scale unionization, much less shorter work days and more pay. They aren’t talking the language of mid-twentieth century liberalism, where everyone puts on blindfolds and cuts slices of pie to share. The 99% looks too beaten down to demand anything as grand as “fairness” in their distribution of the economy. There’s no calls for some sort of post-industrial personal fulfillment in their labor – very few even invoke the idea that a job should “mean something.” It’s straight out of antiquity – free us from the bondage of our debts and give us a basic ability to survive.

Ooph. Now that is depressing. But check out his concluding remark.

We have piecemeal, leaky versions of each of these in our current liberal social safety net. Having collated all these responses, I think completing these projects should be the ultimate goal of the 99%

This is what really strikes me. Here you have a welfare state that isn’t even that big by Western standards but is still not trivial in the resources it consumes, and yet it delivers a pretty crappy outcome to a huge number of citizens. It may be that enough funding from the wealthy restores that system and makes it work. But the financial crises in Europe would seem to suggest otherwise. For many, especially in America, the status quo is unacceptable, and the ability to go back may no longer exist. So until we start thinking about what the future looks like, one free of the systems of the past, we’re probably in trouble.

But any effort here is going to run into a pretty serious brick wall when it comes to coalition building. Consider this amazing line from this Change.org petition:

Rhee uses Change.org to post deceptively worded petitions with such titles as: “Join the Fight to Save Great Teachers” and “Pay Effective Teachers What They Deserve.” When you delve into it, she’s working to weaken unions and institute merit pay for teachers. These are insidious, corporate, anti-progressive reforms. Change.org should not be participating in Rhee’s union busting.

Merit pay is an insidious, corporate, anti-progressive reform? You have to be pretty far out on the left to believe that. Indeed, the notion of merit sat at the heart of the progressive revolution. Now I’m no fan of Rhee, but I’m also a believer that good work should be rewarded and, well, bad work should be punished. That isn’t saying your pay should be linked to test scores, but it also doesn’t mean it shouldn’t be linked to nothing other than tenure. If that is union busting, then the progressive movement it totally dead. This is why I think progressive reform is stopped dead in its tracks. The traditional left wants to defend the status quo of government (while happily attacking the equally problematic status quo of wall st.) while I suspect others, many of whom are sympathetic to the #occupywallst message, are actually equally uncomfortable with the status quo in both the public and private sphere.

This is bad news for those of us who don’t want to return to the Gilded Age. There may not be a coalition that can counter the conservatives on the left and the right. Maybe there needs to be a collapse of this complex system before there can be a rebuilding. It’s a pretty depressing and sobering thought.

Okay. so, you got suckered in by a few fun quotes only to find yourself in the serious world of protest politics. Sorry about that, but that’s the kind of technology fueled, politically driven 24 hours its been.

Hope to see you tomorrow…

International Open Data Hackathon 2011: Better Tools, More Data, Bigger Fun

12 Replies

Last year, with only a month of notice, a small group passionate people announced we’d like to do an international open data hackathon and invited the world to participate.

We were thinking small but fun. Maybe 5 or 6 cities.

We got it wrong.

In the end people from over 75 cities around the world offered to host an event. Better still we definitively heard from people in over 40. It was an exciting day.

Last week, after locating a few of the city organizers email addresses, I asked them if we should do it again. Every one of them came back and said: yes.

So it is official. This time we have 2 months notice. December 3rd will be Open Data Day.

I want to be clear, our goal isn’t to be bigger this year. That might be nice if it happens. But maybe we’ll only have 6-7 cities. I don’t know. What I do want is for people to have fun, to learn, and to engage those who are still wrestling with the opportunities around open data. There is a world of possibilities out there. Can we seize on some of them?

Why.

Great question.

First off. We’ve got more data. Thanks to more and more enlightened governments in more and more places, there’s a greater amount of data to play with. Whether it is Switzerland, Kenya, or Chicago there’s never been more data available to use.

Second, we’ve got better tools. With a number of governments using Socrata there are more API’s out there for us to leverage. Scrapperwiki has gotten better and new tools like Buzzdata, TheDataHub and Google’s Fusion Tables are emerging every day.

And finally, there is growing interest in making “openess” a core part of how we measure governments. Open data has a role to play in driving this debate. Done right, we could make the first Saturday in December “Open Data Day.” A chance to explain, demo and invite to play, the policy makers, citizens, businesses and non-profits who don’t yet understand the potential. Let’s raise the world’s data literacy and have some fun. I can’t think of a better way than with another global open data hackathon – an maker’s fair like opportunity for people to celebrate open data by creating visualizations, writing up analyses, building apps or doing what ever they want with data.

Of course, like last time, hopefully we can make the world a little better as well. (more on that coming soon)

How.

The basic premises for the event would be simple, relying on 5 basic principles.

1. Together. It can be as big or as small, as long or as short, as you’d like it, but we’ll be doing it together on Saturday, December 3rd, 2011.

2. It should be open. Around the world I’ve seen hackathons filled with different types of people, exchanging ideas, trying out new technologies and starting new projects. Let’s be open to new ideas and new people. Chris Thorpe in the UK has done amazing work getting young and diverse group hacking. I love Nat Torkington’s words on the subject. Our movement is stronger when it is broader.

3. Anyone can organize a local event. If you are keen help organize one in your city and/or just participate add your name to the relevant city on this wiki page. Where ever possible, try to keep it to one per city, let’s build some community and get new people together. Which city or cities you share with is up to you as it how you do it. But let’s share.

4. You can work on anything that involves open data. That could be a local or global app, a visualization, proposing a standard for common data sets, scraping data from a government website to make it available for others in buzzdata.

It would be great to have a few projects people can work on around the world – building stuff that is core infrastructure to future projects. That’s why I’m hoping someone in each country will create a local version of MySociety’s Mapit web service for their country. It will give us one common project, and raise the profile of a great organization and a great project.

We also hope to be working with Random Hacks of Kindness, who’ve always been so supportive, ideally supplying data that they will need to run their applications.

5. Let’s share ideas across cities on the day. Each city’s hackathon should do at least one demo, brainstorm, proposal, or anything that it shares in an interactive way with at members of a hackathon in at least one other city. This could be via video stream, skype, by chat… anything but let’s get to know one another and share the cool projects or ideas we are hacking on. There are some significant challenges to making this work: timezones, languages, culture, technology… but who cares, we are problem solvers, let’s figure out a way to make it work.

Like last year, let’s not try to boil the ocean. Let’s have a bunch of events, where people care enough to organize them, and try to link them together with a simple short connection/presentation.Above all let’s raise some awareness, build something and have some fun.

What next?

1. If you are interested, sign up on the wiki. We’ll move to something more substantive once we have the numbers.

2. Reach out and connect with others in your city on the wiki. Start thinking about the logistics. And be inclusive. Someone new shows up, let them help too.

3. Share with me your thoughts. What’s got you excited about it? If you love this idea, let me know, and blog/tweet/status update about it. Conversely, tell me what’s wrong with any or all of the above. What’s got you worried? I want to feel positive about this, but I also want to know how we can make it better.

4. Localization. If there is bandwidth locally, I’d love for people to translate this blog post and repost it locally. (let me know as I’ll try cross posting it here, or at least link to it). It is important that this not be an english language only event.

5. If people want a place to chat with other about this, feel free to post comments below. Also the Open Knowledge Foundation’s Open Data Day mailing list will be the place where people can share news and help one another out.

Once again, I hope this will sound like fun to a few committed people. Let me know what you think.

Ada Lovelace Day – On Dr. Connie Eaves

2 Replies

For those who don’t know: Today – October 7th – is Ada Lovelace Day. It’s a day where you “share your story about a woman — whether an engineer, a scientist, a technologist or mathematician — who has inspired you to become who you are today.”

It would be remiss for me not to blog about Dr. Connie Eaves. For anyone who thinks I travel a lot, work long hours, or have a passion for evidence and data, I am really just a pale shadow when compared to this inspiring and globally recognized cancer researcher. For those not familiar with her – which is probably anyone outside the field of cancer research and not an avid reader of the journal Blood – you can catch her bio on Wikipedia here.

She is, of course, also my mom.

Obviously, if you are a woman (or a man) interested in getting into science – particularly human biology and stem cell research – I would point you to my mother (and father) as people to get to know, but for me her inspiration is much simpler. At a basic level, there are two invaluable gifts my mother has given me, which I feel are particularly salient to her scientific achievements.

The first, and most important, was the building blocks of critical thinking: To break down an argument and understand it from every angle, as well as dissect the evidence embedded within it. These lessons were hard ones. I learned a lot of it just through observation, and sometimes – more painfully – from trying to engage her in debate. I’ve seen graduate students tremble in fear about engaging my mother in debate. While my victories have been few, I’ve been doing it since probably the age of five or earlier, and it has helped shape my brain in powerful ways in which, I suspect, many masters or doctoral students would happily travel around the world to be exposed to. I am exceedingly lucky.

The second gift my mom bestowed me is her work ethic and drive. I have grown up believing that working for 12 hours, 7 days a week may actually be normal behaviour. There is good and bad in taking on such norms. Neither one of us probably thinks it is healthy when we skip eating all day because we zone out into our work. But that intensity has its upsides, and I’m grateful to have been exposed to it. Indeed, I’d like to think I work hard, but standing next to her, I still often just feel lazy.

I mention these two traits not just because they have had such a great impact on me, but also because I think they’re a reflection of what extraordinary skills were required by my mother to be a successful woman scientist embarking on a career in the 1960s. The simple fact is that in that era, as much as we’d like to think it was not true, I suspect that to be a women scientist – to get on tenure track – you had to be smarter and work harder than almost anyone around you. It is one reason why I think the women scientists of that generation are generally so remarkable. The sad truth is: They had to be.

The happy upside is that for me, purely selfishly, is I got the benefit of being raised by someone who survived and thrived in what I imagine was at times a hostile environment to women. Paradoxically, the benefits I enjoyed are those I would wish on any child in a heartbeat, while the asymmetric expectation are those I would wish on no one.

Happy Ada Lovelace mom.

Upcoming talks, events and other activities

Okay, last in a summer series: the Fish Taco from Tacofino, delicious

1 Reply

okay, look at this photo. Tell me that does not look good:

On the left is the classic fish taco from Tacofino, on the right, their halibut special. Both are as good to eat as they look in this photo. Both are pretty minimalist, with the traditional fish taco serving up some lightly fried white fish, some light coleslaw style greens, tomato and a delicious mayo based sauce. The halibut taco doing a more of a light coleslaw greens with the blackberrys and a lighter sauce. All goodness. Not pictured, but also good are the freshies you can order with them. These are frozen drinks/natural slushie, they do a lemon ginger that is quite good, as well as a lime version. Compliments the food real well.

I wish the tacofino link I was sending you too had more than the funky but minimalist webpage they’ve got going on (maybe a menu or something) but you are just going to have to trust me that despite the weak web presence, the food is very much worth checking out.

In Tofino, you find these bad boys made in the beaches shopping complex in their pimped out Tacofino Van…

…which comes pretty close to the type of vehicle you want a taco like that to come out of. The good news is that you no longer have to go to the edges of the earth (literally, Tofino is at the very edge of north america) to get this tasty treat. Although I’m getting conflicting reports on their location, they are definitely on the street of Vancouver s well. I’m hearing English Bay (Denman and Davie) in downtown Vancouver, although the good peeps over at Foodology.ca are saying Howe and Robson (although now I understand that this location is serving a Vietnamese-inspired menu).

I’m hoping to do a recon mission into English Bay at some point in the near future. Test out the wares and see if they are as good as the original (above pictured) location.

Anyways, this is default lunch material when I’m chilling (or engaged in a work vacation) in Tofino, my four square presence is significant.

Just wanted to share some of the lighter facts of the Dave Eaves life as we seem to finally have hit summer in the news cycle and life’s got me busy with so many great projects – can’t wait to share more on them later.

How Dirty is Your Data? Greenpeace Wants the Cloud to be Greener

4 Replies

My friends over at Greenpeace recently published an interesting report entitled “How dirty is your data?
A Look at the Energy Choices That Power Cloud Computing.”

For those who think that cloud computing is an environmentally friendly business, let’s just say… it’s not without its problems.

What’s most interesting is the huge opportunity the cloud presents for changing the energy sector – especially in developing economies. Consider the follow factoids from the report:

Data centres to house the explosion of virtual information currently consume 1.5-2% of all global electricity; this is growing at a rate of 12% a year.
The IT industry points to cloud computing as the new, green model for our IT infrastructure needs, but few companies provide data that would allow us to objectively evaluate these claims.
The technologies of the 21st century are still largely powered by the dirty coal power of the past, with over half of the companies rated herein relying on coal for between 50% and 80% of their energy needs.

The 12% growth rate is astounding. It essentially makes it the fastest growing segment in the energy business – so the choices these companies make around how they power their server farms will dictate what the energy industry invests in. If they are content with coal – we’ll burn more coal. If they demand renewables, we’ll end up investing in renewables and that’s what will end up powering not just server farms, but lots of things. It’s a powerful position big data and the cloud hold in the energy marketplace.

And of course, the report notes that many companies say many of the right things:

“Our main goal at Facebook is to help make the world more open and transparent. We believe that if we want to lead the world in this direction, then we must set an example by running our service in this way.”

– Mark Zuckerberg

But then Facebook is patently not transparent about where its energy comes from, so it is not easy to assess how good or bad they are, or how they are trending.

Indeed it is worth looking at Greenpeace’s Clean Cloud report card to see – just how dirty is your data?

I’d love to see a session at the upcoming (or next year) Strata Big Data Conference on say “How to use Big Data to make Big Data more Green.” Maybe even a competition to that effect if there was some data that could be shared? Or maybe just a session where Greenpeace could present their research and engage the community.

Just a thought. Big data has got some big responsibilities on its shoulders when it comes to the environment. It would be great to see them engage on it.

Mind. Prepare to be blown away. Big Data, Wikipedia and Government.

3 Replies

Okay, super psyched about this. Back at the Strata Conference in Feb (in San Diego) I introduced my long time uber-quant friend and now Wikimedia Foundation data scientist Diederik Van Liere to fellow Gov2.0 thinker Nicholas Gruen (Chairman) and Anthony Goldbloom (Founder and CEO) of an awesome new company called Kaggle.

As usually happens when awesome people get together… awesomeness ensued. Mind. Be prepared to be blown.

So first, what is Kaggle? They’re a company that helps companies and organizations post their data and run competitions with the goal of having it scrutinized by the world’s best data scientists towards some specific goal. Perhaps the most powerful example of a Kaggle competition to date was their HIV prediction competition, in which they asked contestants to use a data set to find markers in the HIV sequence which predict a change in the severity of the infection (as measured by viral load and CD4 counts).

Until Kaggle showed up the best science to date had a prediction rate of 70% – a feat that had taken years to achieve. In 90 days contributors to the contest were able to achieve a prediction rate of 77%. A 10% improvement. I’m told that achieving an similar increment had previously taken something close to a decade. (Data geeks can read how the winner did it here and here.)

Diederik and Anthony have created a similar competition, but this time using Wikipedia participation data. As the competition page outlines:

This competition challenges data-mining experts to build a predictive model that predicts the number of edits an editor will make in the five months after the end date of the training dataset. The dataset is randomly sampled from the English Wikipedia dataset from the period January 2001 – August 2010.

The objective of this competition is to quantitively understand what factors determine editing behavior. We hope to be able to answer questions, using these predictive models, why people stop editing or increase their pace of editing.

This is of course, a subject matter that is dear to me as I’m hoping that we can do similar analysis in open source communities – something Diederik and I have tried to theorize with Wikipedia and actually do Bugzilla data.

There is a grand prize of $5000 (along with a few others) and, amazingly, already 15 participants and 7 submissions.

Finally, I hope public policy geeks, government officials and politicians are paying attention. There is power in data and an opportunity to use it to find efficiencies and opportunities. Most governments probably don’t even know how to approach an organization like Kaggle or to run a competition like this, despite (or because?) it is so fast, efficient and effective.

It shouldn’t be this way.

If you are in government (or any org), check out Kaggle. Watch. Learn. There is huge opportunity here.

12:10pm PST – UPDATE: More Michael Bay sized awesomeness. Within 36 hours of the wikipedia challenge being launched the leading submission has improved on internal Wikimedia Foundation models by 32.4%

Links on Social Media & Politics: Notes from "We Want Your Thoughts #4"

1 Reply

Last night I had a great time taking the stage with Alexandra Samuel in Vancouver for “We Want Your Thoughts” at the Khafka coffee house on Main St. The night’s discussion was focused on Social Media – from chit chat to election winner – what next?” (with a little on the social media driven response to the riots thrown in for good measure).

Both Alex and I promised to post some links from our blogs for attendees so what follows is a list of some thoughts on the subject I hope everyone can find engaging.

On Social Media generally, probably the most popular post on this blog is this piece: Twitter is my Newspaper: explaining twitter to newbies. More broadly thinking about the internet and media, this essay I wrote with Taylor Owen is now a chapter in this university textbook on journalism. Along with this post as a sidebar note (different textbook), which has been one of my most read.

On the riots, I encourage you to read Alexandra Samuel’s post on the subject (After a Loss in Vancouver, Troubling Signals of Citizen Surveillance) and my counter thoughts (Social Media and Rioters) – a blogging debate! You can also hear me talk about the issue on an interview on CBC’s Cross Country Checkup on the issue (around hour 1).

On social media and politics, maybe some of the most notable pieces include a back forth between myself and Michael Valpy who felt that social media was ending our social cohesion and destroying democracy (obviously, this was pre-Middle East Riots and the proroguing Parliament debate). I responded with a post on why his arguments were flawed and that actually the reverse was true. He responded to that post in The Mark. And I posted response to that as well. It all makes for a good read.

Rob Cottingham’s Visual Notes of the first 15 minutes

Then there were some pieces on Social Media and the Proroguing of Parliament. I had this piece in the Globe and then this post talking a little more about the media’s confused relationship with social media and politics.

Finally, one of the points I referred to several times yesterday was the problem of assuming social values won’t change when talking about technology adoption and its impact, probably the most explicit post I’ve written on the subject is this one: Why the Internet Will Shape Social Values (and not the other way around)

Finally, some books/articles I mentioned or on topic:

Everything Bad is Good for You by Steven Johnson

What Technology Wants by Kevin Kelly

Here Comes Everybody by Clay Shirky

The Net Delusion: How Not to Liberate the World by Evgeny Morozov

The Inside Story of How Facebook Responded to Tunisian Hacks an article in the Atlantic by Alexis Madrigal

I hope this is interesting.

The next Open Data battle: Advancing Policy & Innovation through Standards

21 Replies

With the possible exception of weather data, the most successful open data set out there at the moment is transit data. It remains the data with which developers have experimented and innovated the most. Why is this? Because it’s been standardized. Ever since Google and the City of Portland creating the General Transit Feed Specification (GTFS) any developer that creates an application using GTFS transit data can port their application to over 100+ cities around the world with 10s and even 100s of millions of potential users. Now that’s scale!

All in all the benefits of a standard data structure are clear. A public good is more effectively used, citizens receive enjoy better service and companies (both Google and the numerous smaller companies that sell transit related applications) generate revenue, pay salaries, etc…

This is why, with a number of jurisdictions now committed to open data, I believe it is time for advocates to start focusing on the next big issue. How do we get different jurisdictions to align around standard structures so as to increase the number of people to whom an application or analysis will be relevant? Having cities publish open data sets is a great start and has led to real innovation, next generation open data and the next leaps in innovation will require some more standards.

The key, I think, is to find areas that meet three criteria:

Government Data: Is there relevant government data about the service or issue that is available?
Demand: Is this a service for which there is regular demand? (this is why transit is so good, millions of people touch the service on a daily basis)
Business Model: Is there a business that believes it can use this data to generate revenue (either directly, or indirectly)

Two comments on this.

First, I think we should look at this model because we want to find places where the incentives are right for all the key stakeholders. The wrong way to create a data structure is to get a bunch of governments together to talk about it. That process will take 5 years… if we are lucky. Remember the GTFS emerged because Google and Portland got together, after that, everybody else bandwagoned because the value proposition was so high. This remains, in my mind, not the perfect, but the fastest and more efficient model to get more common data structures. I also respect it won’t work for everything, but it can give us more successes to point to.

Which leads me to point two. Yes, at the moment, I think that target in the middle of this model is relatively small. But I think we can make it bigger. The GTFS shows cities, citizens and companies that there is value in open data. What we need are more examples so that a) more business models emerge and b) more government data is shared in a structured way across multiple jurisdictions. The bottom and and right hand circles in this diagram can, and if we are successful will, move. In short, I think we can create this dynamic:

So, what does this look like in practice?

I’ve been trying to think of services that fall in various parts of the diagram. A while back I wrote a post about using open restaurant inspection data to drive down health costs. Specifically around finding a government to work with a Yelp!, Bing or Google Maps, Urban Spoon or other company to integrate the inspection data into the application. That for me is an example of something that I think fits in the middle. Government’s have the data, its a service citizens could touch on a regular base if the data appeared in their workflow (e.g. Yelp! or Bing Maps) and for those businesses it either helps drive search revenue or gives their product a competitive advantage. The Open311 standard (sadly missing from my diagram), and the emergence of SeeClickFix strike me as another excellent example that is right on the inside edge of the sweet spot).

Here’s a list of what else I’ve come up with at the moment:

You can also now see why I’ve been working on Recollect.net – our garbage pick up reminder service – and helping develop a standard around garbage scheduling data – the Trash & Recycling Object Notation. I think it is a service around which we can help explain the value of common standards to cities.

You’ll notice that I’ve put “democracy data” (e.g. agendas, minutes, legislation, hansards, budgets, etc…) in the area where I don’t think there is a business plan. I’m not fully convinced of this – I could see a business model in the media space for this – but I’m trying to be conservative in my estimate. In either case, that is the type of data the good people at the Sunlight Foundation are trying to get liberated, so there is at least, non-profit efforts concentrated there in America.

I also put real estate in a category where I don’t think there is real consumer demand. What I mean by this isn’t that people don’t want it, they do, but they are only really interested in it maybe 2-4 times in their life. It doesn’t have the high touch point of transit or garbage schedules, or of traffic and parking. I understand that there are businesses to be built around this data, I love Viewpoint.ca – a site that takes mashes opendata up with real estate data to create a compelling real estate website – but I don’t think it is a service people will get attached to because they will only use it infrequently.

Ultimately I’d love to hear from people on ideas they on why might fit in this sweet spot. (if you are comfortable sharing the idea, of course). Part of this is because I’d love to test the model more. The other reason is because I’m engaged with some governments interested in getting more strategic about their open data use and so these types of opportunities could become reality.

Finally, I just hope you find this model compelling and helpful.

Lost Open Data Opportunities

6 Replies

Even sometimes my home town of Vancouver gets it wrong.

Reading Chad Skelton’s blog (which I frequently regularly and recommend to my fellow Vancouverites) I was reminded of the great work he did creating an interactive visualization of the city’s parking tickets as part of a series around parking in Vancouver. Indeed, it is worth noting that the entire series was powered by data supplied by the city. Sadly, it just wasn’t (and still isn’t) open data. Quite the opposite, it was data that was wrestled, with enormous difficulty, via an FOI (ATIP) request.

In the same blog post Chad recounts how he struggled to get the parking data from the city:

Indeed, the last major FOI request I made to the city was for its parking-ticket data. I had to fight the city tooth and nail to get them to cough up the information in the format I wanted it in (for months their FOI coordinator claimed, falsely, that she couldn’t provide the records in spreadsheet format). Then, when the parking ticket series finally ran, I got an email from the head of parking enforcement. He was wondering how he could get reprints of the series — he thought it was so good he wanted to hand it out to new parking enforcement officers during their training.

What is really frustrating about this paragraph is the last sentence. Obviously the people who find the most value in this analysis and tool are the city staff who manage parking infractions. So here is someone who, for free(!), provides an analysis and some stories that they now use to train new officers and he had to fight to get the data. The city would have been poorer without Chad’s story and analysis. And yet it fought him. Worse, an important player in the civic space (and an open data ally) feels frustrated by the city.

There are of course, other uses I could imagine for this data. I could imagine the data embedded into an application (ideally one like Washington DC’s Park IT DC – which let’s you find parking meters on a map, identify if they are available or not, and see local car crime rates for the area) so that you can access the risk of getting a ticket if you choose not to pay. This feels like the worse case scenario for the city, and frankly, it doesn’t feel that bad and would probably not affect people’s behaviour that much. But there may be other important uses of this data – it may correlate in some interestingly and unpredictably against other events – connections that if made and shared, might actually allow the city to leverage its enforcement officers more efficiently and effectively.

Of course, we won’t know what those could be, since the data isn’t shared, but it is the kind of thing Vancouver should be doing, given the existence of its open data portal. But all government’s should take note. There is a cost to not sharing data. Lost opportunities, lost insights and value, lost allies and networks of people interested in contributing to your success. It’s all our loss.

eaves.ca

if writing is a muscle, this is my gym