Yearly Archives: 2011

GoDaddy, Mashable and Kernel – mistaking protests for mobs

This week, while enjoying a little down time, I’ve been peeking online from time to time to see what has been going on with the Go Daddy boycott. For those who haven’t been following the story, Go Daddy, a internet domain registration company, came out in support of SOPA – a proposed US law that, in my and the minds of many internet users puts intellectual property rights ahead of civil liberties and creates an “all-you-can-sue-buffet” that essential eliminates due process for many creators who choose to publish on the internet.

As a result of the boycott thousands of internet users – including a number of famous sites such as Wikipedia – have been transfering the domain registration from Go Daddy to alternative companies.

It has been great to see so many internet citizens choose to stand up for their rights and vote with their pocket books. What has been less inspiring is seeing some social media experts completely miss the boat.

For example, I was stunned to see of Mashable write a piece entitled It’s Time to Cut Go Daddy a Break and Milo Yiannopoulos of Kernel write GoDaddy, a hapless victim of the mob both of which profoundly misunderstand what is going on because they look at the issue solely through the lens of social media and not through that of political action.

In the Mashable piece Wasserman seems to believe that once Go Daddy reversed its position – the protests should end:

Now that Go Daddy has unequivocally opposed SOPA, haters are still up in arms because the company seems to have only done it because its business was at risk. Wasn’t this the point?

As it stands now, people seem to be angry at Go Daddy for not succumbing to groupthink. It’s as if just thinking differently than the majority is some sort of crime.

This, unfortunately, is a common phenomenon of the social media age.

Errr… I’m not sure that this is an issue of group think. I agree this can be a problem, but I’m not sure thousands of people engaging in a product boycott out of concerns for their civil liberties is the example I would jump to. This thoughtful piece on the Penny Arcade cyberbulling and counter cyberbullying of Ocean Media would make for a MUCH better example.

I’m actually deeply comfortable with thousands of people crying foul about a company that sought to bring technology sector legitimacy to a deeply, deeply problematic piece of legislation and demanding that it demonstrate some deep introspection, rather the changing its position a couple of times and then grudgingly arriving at an answer that is somewhat okay. Groupthink suggests an unthinking hoard. The boycott is composed of very rational, relatively informed (compared to the general public on this issue) group of people with a political agenda.

This is why Wasserman’s question is naive, it seeks to answer a social media question, not engage the politics of the problem that motivates people – getting Go Daddy to change its position was only part of the point. The real goal here isn’t just to change Go Daddy, it’s to change a whole range of actors position. This is a political action as much as a conusmer action. More importantly, it appears to be working since EA, Sony and Nintendo have quietly dropped their support for SOPA.

But more to the point, the manner by which Go Daddy changed its position has done little to give protestors (yes that is what they are, not a mob) confidence that they should stop. Again this is something that could only be understood by looking at this issue through a political and not purely social media lens. Consider Go Daddy’s statement on their blog:

Go Daddy opposes SOPA because the legislation has not fulfilled its basic requirement to build a consensus among stake-holders in the technology and Internet communities. Our company regrets the loss of any of our customers, who remain our highest priority, and we hope to repair those relationships and win back their business over time.

There is no recognition about the concerns of their protestors/fleeing customers and no discussion about how Go Daddy would contemplate decisions like this in the future. This is maybe the most grudging change of heart ever made. When Johnson & Johnson pulled Tylenol, everyone had confidence that decision making at the company was aligned with their values and their concerns. Go Daddy has gone out of its way to do the opposite.

Does Go Daddy have the right to do this? Absolutely. But do we have the right to boycott them. Definitely. But to label those seeking to defend their civil liberties who – for pretty good reasons – think the target of their boycott is less than sincere as a mob merely engaged in groupthink is a gross oversimplification. Or maybe its just link-bait, I’m sure it drove a lot of traffic. But I suspect it isn’t going to earn any speaking engagements with Greenpeace, Amnesty or pretty much anyone trying to change the world.

Of course Yiannopoulos piece in Kernel is even worse. Here Go Daddy is a “hapless victim.” Indeed, the most priceless line in the piece is this one:

Holding authority to account is an art best practised by professionals in the media.

Yes, I’m sure a San Francisco Chronicle editorial would have had a much bigger impact.

Apparently, to Yiannopoulos, consumer boycotts are a bad thing, or at least, too important to be left to actually citizens and consumers. This is even more the case when they are organized online and can actually achieve scale and impact. Oh, and better still, those participating in the protest “had no choice but to comply” as they were “swept up in the flood.” The pure contempt for people is truly staggering. None of you have free will or critical thinking skills! Leave all this important stuff to important people who can think!

And this from a internet culture expert. Yeow.

There are real risks of mob mentality online but this is about the poorest example you could use at the moment. Here you have thousands of people engaged in effective political and consumer protest, one that is causing larger actors to rethink their positions and potentially achieve political goals. And somehow this is all a mon engaged in groupthink? My sense is a lot of people protesting Go Daddy actually have thought about the issue – they are at least informed enough to care. More importantly thank god they have a sufficiently developed sense of civic duty to act.

The Future of Academic Research

Yesterday, Nature – one of the worlds premier scientific journals recognized University of British Columbia scientist Rosie Redfield as one of the top 10 science newsmakers of 2011.

The reason?

After posting a scathing attack on her blog about a paper that appeared in the journal Science, Redfield decided to attempt to recreate the experiment and has been blogging about her effort over the past year. As Nature describes it:

…that month, Redfield took matters into her own hands: she began attempting to replicate the work in her lab at the University of British Columbia in Vancouver, and documenting her progress on her blog (http://rrresearch.fieldofscience.com).

The result has been a fascinating story of open science unfolding over the year. Redfield’s blog has become a virtual lab meeting, in which scientists from around the world help to troubleshoot her attempts to grow and study the GFAJ-1 bacteria — the strain isolated by Felisa Wolfe-Simon, lead author of the Science paper and a microbiologist who worked in the lab of Ronald Oremland at the US Geological Survey in Menlo Park, California.

While I’m excited about Redfields blog (more on that below) we should pause and note the above paragraph is a very, very sad reminder of the state of affairs in science. I find the term “open science” to be an oxymoron. The scientific process only works when it is, by definition, open. There is, quite arguably, no such thing as “closed science.” And yet it is a reflection of how 18th century the entire science apparatus remains that Redfields awesome experiment is just that – an experiment. We should celebrate her work, and ask ourselves, why is this not the norm?

So first, to celebrate her work… when I look at Redfields blog, I see exactly what I hope the future of scientific, and indeed all academic research, will look like. Here is someone who is constantly updating their results and sharing what they are doing with their peers, as well as getting input and feedback from colleagues and others around the world. Moreover, she plays to the mediums strengths. While rigorous, she remains inviting and, from my reading, creates a more honest and human view into the world of science. I suspect that this might be much more attractive (and inspiring) to potential scientists. Consider, these two lines from one of her recent posts:

So I’m pretty sure I screwed something up.  But what?  I used the same DNA stock tube I’ve used many times before, and I definitely remember putting 3 µl of DNA into each assay tube.  I made fresh sBHI + novobiocin plates using pre-made BHI agar,, and I definitely remember adding the hemin (4 ml), NAD (80 µl) and novobiocin (40 µl) to the melted agar before I poured the plates.

and

UPDATE:  My novobiocin plates had no NovR colonies because I had forgotten to add the required hemin supplement to the agar!  How embarrassing – I haven’t made that mistake in years.

and then this blog post title:

Some control results! (Don’t get excited, it’s just a control…)

Here is someone literally walking through their thought processes in a thorough, readable way. Can you imagine anything more helpful for a student or young scientist? And the posts! Wonderfully detailed walk throughs of what has been tried, progress made and set backs uncovered. And what about the candor! The admission of error and the attempts to figure out what went wrong. It’s the type of thinking I see from great hackers as well. It’s also the type of dialogue and discussion you won’t see in a formal academic paper but is exactly what I believe every field (from science, to non-profit, to business) needs more of.

Reading it all, and I’m once again left wondering. Why is this the experiment? Why isn’t this the norm? Particularly at publicly funded universities?

Of course, the answer lies in another question, one I first ran into over a year ago reading this great blog post by Michael Clarke on Why Hasn’t Scientific Publishing Been Disrupted Already? As he so rightly points out:

When Tim Berners-Lee created the Web in 1991, it was with the aim of better facilitating scientific communication and the dissemination of scientific research. Put another way, the Web was designed to disrupt scientific publishing. It was not designed to disrupt bookstores, telecommunications, matchmaking services, newspapers, pornography, stock trading, music distribution, or a great many other industries…

…The one thing that one could have reasonably predicted in 1991, however, was that scientific communication—and the publishing industry that supports the dissemination of scientific research—would radically change over the next couple decades.

And yet it has not.

(Go read the whole article, it is great). Mathew Ingram also has a great piece on this published half a year later called So when does academic publishing get disrupted?

Clarke has a great breakdown on all of this, but my own opinion is that scientific journals survive not because they are an efficient means of transmitting knowledge (they are not – Redfield’s blog shows there are much, much faster ways to spread knowledge). Rather journals survive in their current form because they are the only rating system scientists (and more importantly) universities have to deduce effectiveness, and thus who should get hired, fired, promoted and, most importantly, funded. Indeed, I suspect journals actually impede (and definitely slow) scientific progress. In order to get published scientists regularly hold back sharing and disclosing discoveries and, more often still, data, until they can shape it in such a way that a leading journal will accept it. Indeed, try to get any scientists to publish their data in machine readable formats – even after they have published with it -it’s almost impossible… (notice there are no data catalogs on any major scientific journals websites…) The dirty secret is that this is because they don’t want others using it in case it contains some juicy insight they have so far missed.

Don’t believe me? Just consider this New York Times article on the break throughs in Alzheimer’s. The whole article is about a big break through in scientific research process. What was it? That the scientists agreed they would share their data:

The key to the Alzheimer’s project was an agreement as ambitious as its goal: not just to raise money, not just to do research on a vast scale, but also to share all the data, making every single finding public immediately, available to anyone with a computer anywhere in the world.

This is unprecedented? This is the state of science today? In an era where we could share everything, we opt to share as little as possible. This is the destructive side of the scientific publishing process that is linked to performance.

It is also the sad reason why it is a veteran, established researcher closer to the end of her career that is blogging this way and not a young, up and coming researcher trying to establish herself and get tenure. This type of blog is too risky to ones career. Today “open” science, is not a path forward. It actually hurts you in a system that prefers more inefficient methods at spreading insights, research and data, but is good at creating readily understood rankings.

I’m thrilled that Rosie Redfield has been recognized by Nature (which clearly enjoys the swipe at Science – its competitor). I’m just sad that the today’s culture of science and universities means there aren’t more like her.

 

Bonus material: If you want to read an opposite view, here is a seriously self-interested defensive of the scientific publishing industry that was totally stunning to read. It’s fascinating that this man and Michael Clarke share the same server. If you look in the comments of that post, there is a link to this excellent post by a researcher at a University in Cardiff that I think is a great counter point.

 

Why is Finding a Post Box so Hard?

Sometimes it is the small things that show how government just gets it all so wrong.

Last Thursday The Daily Show’s Wyatt Cenac has a little bit on the US Post Office and its declining fortunes as people move away from mail. There is no doubt that the post offices days are numbered, but that doesn’t mean the decline has to be as steep as it is. Besides there are things they could be doing to make life a little easier to use them (and god knows they should be doing anything they can, to be more appealing).

Take, for example, the humble post office box. They can be frustratingly hard to locate. Consider Broadway and Cambie – one of the busiest intersections in Vancouver – and yet there is no post box at the intersection. (I eventually found it one block east on broadway) but I carried around a letter for 3 weeks before I eventually found one.

In short why is there not digital map (or for techies, and API) for post box locations? I could imagine all sorts of people that might make use of it. Would it be nice to just find out – where is the closest post box to where I’m standing? More importantly, it might actually help the post office attract a few extra customers. It certainly wouldn’t hurt customer service. I’ve wondered for a couple of years why it doesn’t publish this data set.

Turns out I’m not the only with this frustration. My friend Steven Tannock has channeled his frustration into a simple app called Wherepost.ca. It’s a simple website – optimized for mobile phone use – that allows users to add post boxes as well as find the one nearest to them. In short, Steven’s trying to create a public data set of post box locations by crowd sourcing the problem. If Canada Post won’t be helpful… we’ll help one another.

Launched on Thursday with 20 post office box locations, there are now over 400 boxes mapped (mostly in the Vancouver area) with several dozen users contributing. In addition, Steven tells me users in at least 2 other countries have asked for new icons so they can add post boxes where they live. It seems Canadians aren’t the only ones frustrated about not knowing where the nearest post box is.

The ideal, of course, would be for Canada Post to publish an API of all post box locations. I suspect however, that they either don’t actually know where they all are in a digital form (at which point they should really help Steven as he is doing them a huge service) or revealing their location will be seeing as sacrificing some important IP that people should pay for. Remember, this is an organization that refuses to make Postal Code data open, a critical data set for companies, non-profits and governments.

This isn’t the worlds fanciest app but its simplicity is what makes it so great, and so useful. Check it out at WherePost.ca and… of course, add a post box if you see one.

 

 

Not Brain Candy: A Review of The Information Diet by Clay Johnson

My body no longer kills me when I come back from the gym. However, I had a moment of total humiliation today: theoretically my ideal body weight is 172 pounds and I weigh 153 Ibs. The woman at the gym calibrated my fat/water/meat/bone ratios, made an inward gasp and I asked her what was wrong. She said (after a tentative, you-have-cancer pause), “You’re what’s technically known as a ‘thin fat person.’ “

- Douglas Copeland, Microserfs

We know that healthy eating – having a good, balanced diet – is the most important thing we can do for our physical health. What if the same is true of our brains?  This is the simple but powerful premise that lies at the heart of Clay Johnson’s excellent book The Information Diet.

It’s also a timely thesis.

Everyone seems worried about how we consume information, about what it is doing to our brains and how it impacts society. Pessimists believe Google and social media are creating a generation of distracted idiots unable or unwilling to steep themselves in any deep knowledge. From the snide ramblings of Andrew Keen in The Cult of the Amateur to alarmed New York Times executive editor Bill Keller – who equates letting his daughter join Facebook to passing her a crystal meth pipe – the internet and the type of information it creates are apparently destroying our minds, our society and, of course, our children.

While I disagree with the likes of Keen and Keller, your humble author admits he’s an information addict. I love reading the newspaper or my favourite columnists/bloggers; I’m regularly distracted by both interesting and meaningless articles via Twitter and Facebook; and I constantly struggle to stay on top of my email inbox. I’m a knowledge worker in an information society. If anyone should be good at managing information, it should be me. Reading The Information Diet forces me to engage with my ability in a way I’ve not done before.

What makes The Information Diet compelling is that Johnson embraces the concerns we have about the world of information overload – from those raised by New York Magazine authors and celebrated pundits to the challenges we all feel on a day to day basis – and offers the best analysis to date of its causes, and what we can do about it. Indeed, rather than being a single book, The Information Diet is really three. It’s an analysis of what is happening to the media world; it’s a self-help book for information-age workers, consumers and citizens; and it’s a discussion about the implications of the media environment on our politics.

InfoDietIt is in its first section that the book shines the brightest. Johnson is utterly persuasive in arguing that the forces at play in the food industry are a powerful mirror for our media environment. Today the main threat to Americans (and most others living in the developed world) is not starvation; it’s obesity. Our factory farms are so completely effective at pumping out produce that it isn’t a lack of food the kills us, it’s an overabundance of it. And more specifically, it’s the over-consumption of food that we choose to eat, but that isn’t good for us in anything greater than small quantities.

With information, our problem isn’t that we consume too much – Johnson correctly points out that physically, this isn’t possible. What’s dangerous is consuming an overabundance of junk information – information that is bad for us. Today, one can choose to live strictly on a diet of ramen noodles and Mars bars. Similarly, it’s never been easier to restrict one’s information consumption to that which confirms our biases. In an effort to better serve us, everywhere we go, we can chomp on a steady diet of information that affirms and comforts rather than challenges – information devoid of knowledge or even accuracy; cheaply developed stories by “big info” content farms like Demand Media or cheaply created opinion hawked by affirmation factories like MSNBC or FOX News; even emails and tweets that provide dopamine bursts but little value. In small quantities, these information sources can be good and even enjoyable. In large quantities, they deplete our efficiency, stress us out, and can put us in reality bubbles.

And this is why I found The Information Diet simultaneously challenging, helpful and worrying.

Challenging, because reading The Information Diet caused me to think of my own diet. I like to believe I’m a healthy consumer, but reflecting on what I read, where I get my information and who I engage with, in parts of my life, I may be that dreaded thin-fat person. I look okay, but probe a little deeper and frankly, there are a few too many confirmation biases, too many common sources, leaving my brain insufficiently challenged and becoming a shade flabby. I certainly spend too much time on email, which frankly is a type of information fix that really does sap my productivity.

Helpful, because in part The Information Diet is a 21st-century guide to developing and honing critical thinking and reasoning skills. At its most basic, it’s a self-help book that provides some solid frameworks and tools for keeping these skills sharp in a world where the opportunities for distraction and confirmation bias remain real and the noise-to-signal ratio can be hard to navigate.  To be clear, none of this advice is overly refined, but Johnson doesn’t pretend it is. You can’t download critical thinking skills – no matter what Fox News’s slogan implies. In this regard, the book is more than helpful – it’s empowering. Johnson, correctly I believe, argues that much like the fast food industry – which seeks to exploit our body’s love of salty, fatty food – many media companies are simply indulging our desire for affirming news and opinion. It’s not large companies that are to blame. It’s the “secret compact” (as Johnson calls it) that we make with them that makes them possible. We are what we consume. In this regard, for someone that those on the right might consider (wrongly) to be a big government liberal, The Information Diet has an strong emphasis on personal responsibility.

There is, of course, a depressing flip side to this point: one that has me thinking about the broader implications of his metaphor. In a world of abundant food, we have to develop better discipline around dieting and consumption.

But the sad fact is, many of us haven’t. Indeed, almost a majority has not.

As someone who believes in democratic discourse, I’ve always accepted that as messy as our democratic systems may be, over time good ideas – those backed by evidence and effective track records – will rise to the top. I don’t think Johnson is suggesting this is no longer true. But he is implying that in a world of abundant information, the basic ante of effective participation is going up. The skills are evolving and the discipline required is increasing. If true, where does that leave us? Are we up for the challenge? Even many of those who look informed may simply be thin fat people. Perhaps those young enough to grow up in the new media environment will automatically develop the skills Clay says we need to explicitly foster. But does this mean there is a vulnerable generation? One unable to engage critically and so particularly susceptible to the siren song of their biases?

Indeed, I wish this topic were tackled more, and initially it felt like it would be. The book starts off as a powerful polemic on how we engage in information; it is then a self-help book, and towards the end, an analysis of American politics. It all makes for fascinating reading. Clay has plenty of humour, southern charm and self-deprecating stories that the pages flow smoothly past one another. Moreover, his experience serves him well. This is man who worked at Ask Jeeves in its early days, helped create the online phenomenon of the Howard Dean campaign, and co-founded Blue State Digital – which then went on to create the software that powered Obama’s online campaign.

But while his background and personality make for compelling reading, the last section sometimes feels more disconnected from the overall thesis. There is much that is interesting and I think Clay’s concerns about the limits of transparency are sound (it is a prerequisite to success, but not a solution). Much like most people know Oreos are bad for them, they know congressmen accept huge bundles of money. Food labels haven’t made America thinner, and getting better stats on this isn’t going to magically alter Washington. Labels and transparency are important tools for those seeking to diet. Here the conversation is valuable. However, some of the arguments, such as around scalability problems of representation, feel less about information and more about why politics doesn’t work. And the chapter closes with more individual advice. This is interesting, but his first three chapters create a sense of crisis around America’s information diet. I loved his suggestions for individuals, but I’d love to hear some more structural solutions, or if he thinks the crisis is going to get worse, and how it might affect our future.

None of this detracts from the book. Quite the opposite – it left me hungry for more.

And I suspect it will do the same for anyone interested in participating as a citizen or worker in the knowledge economy. Making The Information Diet part of your information diet won’t just help you rethink how you consume information, live and work. It will make you think. As a guy who knows he should eat more broccoli but doesn’t really like the taste, it’s nice to know that broccoli for your brain can be both good for you and tasty to read. I wish I had more of it in my daily diet.

For those interested you can find The Information Diet Blog here – this has replaced his older well known blog – InfoVegan.com.

Full disclosure: I should also share that I know Clay Johnson. I’ve been involved in Code for America and he sits on the Advisory Board. With that in mind, I’ve done my best to look at his book with a critical eye, but you the reader, should be aware.

Open Government Consultation, Twitter Townhalls & Doing Advocacy Wrong

Earlier this week the Canadian Federal Government launched its consultation process on Open Government. This is an opportunity for citizens to comment and make suggestions around what data the federal government should make open and what information it should share, and provide feedback on how it can consult more effectively with Canadians. The survey (which, handily, can be saved midway through completion) contains a few straightforward multiple choice questions and about eight open ended questions which I’ve appended to the end of this post so that readers can reflect upon them before starting to fill out the form.

In addition to the online consultations, Tony Clement – the Minister responsible for the Open Government file – will host a Twitter townhall on Open Government this Thursday (December 15). Note! The townhall will be hosted by the treasury board twitter account @TBS_Canada (English) and @SCT_Canada (French) not Minister Clement’s personal (and better known) twitter account. The townhall will first take place in French from 4-4:45pm EST using the hashtags #parlonsgouvert and then in English from 5-5:45 EST using the hashtag #opengovchat.

Some of you may have also noticed that Democracy Watch issued a strongly worded press release last week with the (somewhat long) headline “Federal Conservatives break all of their international Open Government Partnership commitments by failing to consult with Canadians about their draft action plan before meeting in Brazil this week.” This seems to have prompted the CBC to write this article.

Now, to be clear, I’m a strong advocate for Open Government, and there are plenty of things for which one could be critical about this government for not being open about. However, to be credible – especially around issues of transparency and disclosure – one must be factual. And Democracy Watch did more than just stretch the truth. The simple fact is, that while I too wish the government’s consultations had happened sooner, this does not mean it has broken all of its Open Government Partnership commitments. Indeed, it hasn’t broken any of its commitments. A careful read of the Open Government Partnership requirements would reveal that the recent December meeting was to share drafts plans (including the plans by which to consult). The deadline that Democracy Watch is screaming about does not occur until March of 2012.

It would have been fair to say the government has been slow in fulfilling its commitments, but to say it has broken any of them is flatly not true. Indeed the charge feels particularly odd given that in the past two weeks the government signed on greater aid transparency via IATI and released an additional 4000 data sets, including virtually all of StatsCan’s data, giving Canadian citizens, non profits, other levels of governments and companies access to important data sets relevant for social, economic and academic purposes.

Again, there are plenty of things one could talk about when it comes to transparency and the government.  Yes, the consultation could have gotten off the ground faster. And yes, there is much more to done. But this screaming headline is somewhat off base. Publishing it damages both the credibility of the organization making the charge, and risk hurting the credible of open government advocates in general.

 

List of Open Ended Questions in the Open Government Consultation.

1. What could be done to make it easier for you to find and use government data provided online?

2. What types of open data sets would be of interest to you? Please pick up to three categories below and specify what data would be of interest to you.

3. How would you use or manipulate this data?

4. What could be done to make it easier for you to find government information online?

7. Do you have suggestions on how the Government of Canada could improve how it consults with Canadians?

8. Are there approaches used by other governments that you believe the Government of Canada could/should model?

9. Are there any other comments or suggestions you would like to make pertaining to the Government of Canada’s Open Government initiative?

 

Open Data Day 2011 – Recaps from Around the World

This last Saturday was International Open Data Day with hackathons taking place in cities around the world.

How many you ask? We can’t know for certain, but organizers around the world posted events to the wiki in over 50 cities around the world. Given the number of tweets with the #odhd hashtag, and the locations they were coming from, I don’t think we were far off that mark. If you assume 20 people at each event (some had many more – for instance there were over 100 in Ottawa, Vancouver had close to 50, 120+ in New York) it’s safe to say more than 1000 people were hacking on open data projects around the world.

It’s critical to understand that Open data Day is a highly decentralized event. All the work that makes it a success (and I think it was a big success) is in the hands of local organizers who find space, rally participants, push them to create stuff and, of course, try to make the day as fun as possible. Beyond their hard work and dedication there isn’t much, if any, organization. No boss. No central authority. No patron or sponsor to say thank you. So if you know any of the fine people who attended, or even more importantly, helped organize an event, please shake their hand or shoot them a thank you. I know I’m intensely grateful to see there are so many others out there that care about this issue, that want to connect, learn, meet new people, have fun and, of course, make something interesting. Given the humble beginnings of this event, we’ve had two very successful years.

So what about the day? What was accomplished? What Happened?

Government Motivator

I think one of the biggest accomplishments of Open Data Day has been how it has become a motivator – a sort of deadline – for governments keen to share more open data. Think about this. A group of volunteers around the world is moving governments to share more data – to make public assets more open to reuse. For example, in Ireland Fingal County Council released data around trees, parking, playing pitches & mobile libraries for the day. In Ontario, Canada the staff for the Region of Waterloo worked extra hard to get their open data portal up in time for the event. And it wasn’t just local governments. The Government of BC launched new high value data sets in anticipation of the event and the Federal Government of Canada launched 4000 new data sets with International Open Data Day in mind. Meanwhile, the open data evangelist of Data.gov was prepared to open up data sets for anyone who had a specific request.

While governments should always be working to make more data available I think we can all appreciate the benefits of having a deadline, and Open Data Day has helped become just that for more and more governments.

In other places, Open Data Day turns into a place where governments can converse with developers and citizens about why open data matters, and do research into what data the public is interested in. This is exactly what happened in Enschede in the Netherlands where local city staff worked with participants around prioritizing data sets to make open.

Local Events & Cool Hacks

A lot of people have been blogging about, or sharing videos of, Open Data Day events around the world. I’ve seen blog posts and news articles on events in places such as Madrid, Victoria BC, Oakland, Mexico City, Vancouver, and New York City. If there are more, please email them to me or post them on the wiki.

I haven’t been able to keep track of all the projects that got worked on, but here are a sampling of some that I’ve seen via twitter, the wiki and other forums:

Hongbo: The Emergency Location Locator

In Cotonou, Benin the open data day participants developed a web application called Hongbo the Goun word for “Gate.” Hongbo enables users to locate the nearest hospital, drugstore and police stations. As they noted on the open data day wiki, the data sets for this application were public but not easily accessible. They hope Benin citizen can use it quickly identify who to call or where to go in emergencies.

Tweet My Council

In Sydney, Australia participants created Tweetmycouncil. A fantastic simply application that allows a user to know which jurisdiction they are standing in. Simply send a tweet to the hashtag #tmyc and the app will work where you, what council’s jurisdiction you are in and send you a tweet with the response.

Mexican Access to Information Tracker

In Mexico City one team created an application to compare Free of Information requests between different government departments. This could be a powerful tool for citizens and journalists. (Github repo)

Making it Easier for the Next Guy

Another project out of Mexico City, a team from Oaxaca created an API that creates a json file for any public data set. Would be great for this team to connect with Max Ogden and talk about Gut.

Making it Even Easier for the Next Guy

Speaking of, Max Ogden in Oakland shared more on Gut, which is less of a classic app then a process that enables users to convert data between different formats. It had a number of people excited including open data developers at other locations such as Luke Closs and Mike West.

Mapping Census Data in Barcelona

A team of hackers in Barcelona mapped census tracts so they could be visualized, showing things, like say, the number of parks per census tract. You can find the data sets they used in Google Fusion Tabels here.

Foreign Aid Visualizations

In London UK and in Seattle (and possibly other places) developers were also very keen on the growing amount of aid data being made available and in a common structure thanks to IATI. In Seattle developers created this very cool visualization of US Aid over the last 50 years. I know the London UK team has visualizations of their own they’d like to share shortly.

Food Hacking!

One interesting thing about Open Data Day is how it bridges some very different communities. One of the most active are the food hackers which came out in force in both New York and Vancouver.

In New York a whole series of food related tools, apps and visualization got developed, most of which are described here and here. The sheer quantity of participants (120+) and projects developed is astounding, but also fantastic is how inclusive their event is, with lots of people not just working on apps, but analyzing data and creating visualizations to help others understand an issue they share a common passion for: the Food Bill. Please do click on those links to see some of the fun visuals created.

The Ultimate Food API

In Vancouver, the team at FoodTree – who hosted the hackathon there – focused on shipping an API for developers interested in large food datasets.  You can find their preliminary API and datasets in github. You can also track the work they’ve done on their Open Food Wiki.

Homelessness

In Victoria, BC a team created a map of local walk-in community services that you can check out at http://ourservices.ca/.

BC Emergency Tweeting System

Another team in Victoria, BC focused on creating twitter hashtags for each official place in the province with the hopes that the province’s Provincial Emergency Program.

Mapping Shell’s Oils Spills in Nigeria

The good people at the Open Knowledge Foundation worked on getting a ton more data into the Datahub, but they also had people learning how to visualize data, one of whom created this visualization of oil spills in Nigeria. Always great to see people experimenting and learning!

Mapping Vancouver’s Most Dangerous Intersections for Bikes

Open Data hacking and biking accident data have a long history together and this hackathon I uploaded 5 years worth of bike accident I managed to get from ICBC to Buzzdata. As a result – even though I couldn’t be present in Vancouver – two different developers took it and mapped it. You can see @ngriffithshere and @ericp’s will be up soon. It was interesting to learn that Broadway and Cambie is the most dangerous intersection in the city for cyclists?

Looking Forward

Last year open data day attracted individual citizens: those with a passion for an issue (like food) or who want to make their government more effective or citizens lives a little easier. However, this year we already started to see the community grow – the team at Socrata hosted a hackathon at their offices in Seattle. Buzzdata had people online trying to help people share their data. In addition to these private companies some of the more established non-profits were out in force. The Open Knowledge Foundation had a team working on making openspending.org more accessible while MySociety helped a team in Canada set up a local version of MapIt.

For those who think that open data can change the world or, even build medium sized economic ecosystems, over night, we need to reset their expectations. But it is growing. No longer are participants just citizens and hacktavists – there are real organizations and companies participating. Few, but they are there. My hope is that this trend will continues. That open data day will continue to have meaning for individuals and hackers but will also be something that larger more established organizations, non-profits and companies will use as a rallying point as well. Something to shoot for next year.

Feedback

As I mentioned at the beginning, Open Data Day is a very decentralized event. We are, of course, not wedded to that approach and I’d love to hear feedback from people, good or bad, about worked or didn’t work. Please do feel free to email me, post it to the mailing list or simply comment below.

 

 

Postscript

Finally, some of you may have noticed I became conspicuously absent on the day. I want to apologize to everyone. My partner went into labour on Friday night and so by early morning Saturday it was obvious that my open data day was going to be spent with her. Our baby was 11 days over due so we really thought that we’d be in the clear by Dec 3rd… but our baby had other plans. The good news is that despite 35 hours of labour, baby and boy are doing well!

StatsCan's free data costs $2M – a rant

So the other day a reader sent me an email pointing me to a story in iPolitics titled “StatsCan anticipates $2M loss from move to open data” and asked me what I thought.

Frustrated, was my response.

$2M is not a lot of money. Not in a federal budget of almost $200B.  And, the number may have been less. The StatsCan person quoted in the article called this expected loss of revenue a “maximum net loss.” This may mean that the loss from making the data free does not take into account the fact the StatsCan’s expenditures may also go down. For instance, if StatsCan no longer has to handle as many financial transactions or chase down invoices and so forth, the reduction if staff over other overhead (unrelated to its core mission by the way) and so result in lower operating costs not reflected in the $2M cited above.

Moreover it is still unclear to me where the $2M figure comes from. As I noted in a blog post earlier this year, in StatsCan’s own reports it outlined that its online database (the one just made free) generated $559,000 in revenue (not profit) in 2007-08 and was estimated to generate $525,000 in revenue in 2010-11. Where does the extra $1.5M come from? I’m open to the fact that I’m reading these reports incorrectly… but it is hard to see how.

But all this is really an aside.

What really, really, really, frustrates me is that the hard number of $2M. It is a pittance.

This is the unbearable cost that’s been holding up open StatsCan data for years? This may be the tiniest golden goose ever killed. Maybe more like a lame duck. Can anyone believe the loss of $2M (or 500K) was going to break the organization?

Give me a break.

What a colossal lack of imagination and sense of economic and social prosperity on the part of every government since Mulroney (who made StatsCan engage in cost recovery). In the United States open statistical data has helped businesses, the social sector, local and state governments, as well as researchers and academics. Heck, even Canadian teachers tell me that they’ve been forced to train students on US data because they couldn’t afford to train their students on Canadian data. All this lost innovation, efficiency, jobs and social benefits for a measly $2M dollars (if that). Oh lack of vision, at all levels! Both at the top of the political order, and within StatsCan, which has been reluctant to go down this route for years.

Now that we see the “cost” this battle seems more pathetic than ever.

Sigh. Rant over.

Using Open Data to Map Vancouver’s Trees

This week, in preparation for the International Open Data Hackathon on Saturday, the Vancouver Parks Board shared one neighborhood of its tree inventory database (that I’ve uploaded to Buzzdata) so that we could at least see how it might be leveraged by citizens.

What’s interesting is how valuable this data is already (and why it should be open). As it stands this data could be used by urban landscape students and architects, environmentalists, and of course academics and scientists. I could imagine this data would even be useful for analyzing something as obtuse as the impact of the tree’s Albedo effect on the city’s climate. Of course, locked away in the city’s data warehouse, none of those uses are possible.

However, as I outlined in this blog post, having lat/long data would open up some really fun possibilities that could promote civic engagement. People could adopt trees, care for them, water them, be able to report problems about a specific tree to city hall. But to do all this we need to take the city’s data and make it better – specifically, identify the latitude and longitude of each tree. In addition to helping citizens it might make the inventory more use to the city (if they chose to use it) as well as help out the other stakeholders I outlined above.

So here’s what I’ve scoped out would be ideal to do.

Goal

Create an app that would allow citizens to identify the latitude and longitude of trees that are in the inventory.

Data Background

A few things about the city’s tree inventory data. While they don’t have an actual long/lat for each individual tree, they do register trees by city address. (Again, you can look at the data yourself here.) But this means that we can narrow the number of trees down based on proximity to the user.

Process

So here is what I think we need to be able to do.

  1. Convert the addresses in the inventory into a format that can be located within Google Maps
  2. Just show the trees attached to addresses that are either near the user (on a mobile app), or near addresses that are currently visible within Google Maps (on a desktop app).
  3. Enable the user to add a lat/long to a specific tree’s identification number.

Awesome local superstar coder/punk rock star Duane Nickull whipped together a web app that would allow one to map lat/longs. So based on that, I could imagine at desktop app that allows you to map trees remotely. This obviously would not work for many trees, but it would work for a large number.

Tree-MApper-Screen-shot-11

You’ll notice in the right-hand corner, I’ve created an illustrative list of trees to choose from. Obviously, given the cross-section of the city we are looking at, it would be much longer, but if you were zoomed in all the way I could imagine it was no longer than 5-20.

I’ve also taken the city’s data and parsed it in a way that I think makes it easier for users to understand.

tree-language-parsed

This isn’t mind-blowing stuff, but helpful. I mean who knew that dbh (diameter at breast height) was an actual technical term when measuring tree diameters! I’ve also thrown in some hyperlinks (it would be nice to have images people can reference) so users can learn about the species and ideally, even see a photo to compare against.

Tree-Mapper-Screenshot-2

So, in short, you can choose a tree, locate it in Google Maps and assign a lat/long to it. In Google Maps where you can zoom even closer than ESRI, you could really pick out individual trees.

In addition to a desktop web app, I could imagine something similar for the iPhone where it locates you using the GPS, identifies what trees are likely around you, and gives you a list such as the one on the right hand side of the screenshot above, the user then picks a tree from the list that they think they’ve identified, stands next to the tree and then presses a button in the app that assigns the lat/long of where they are standing to that tree.

Canada’s Foreign Aid Agency signs on to IATI: Aid Data get more transparent

Last night, while speaking at the High Level Forum on Aid Effectiveness in Busan Korea, Minister of International Cooperation Bev Oda announced that Canada would be signing on to the International Aid Transparency Initiative (IATI).

So what is IATI and why does this matter?

IATI has developed a common, open and international standard for sharing foreign aid data. By signing on to IATI Canada is agreeing to publish all the data about its projects and who it funds in a form and structure that makes it easy to compare with others who use the IATI standard. This should make it easier to understand where Canadian aid money ends up, in turn allowing analysts to spot efficiencies as well as compare funding and efforts across donor and recipient countries as well as other stakeholders. In short, aid data should become easier to understand, to compare, and to use.

In the medium term it should also make the data available on CIDA’s open data portal (already helpful to non-profits, development groups and students) even more useful.

This is an enormous win for the good people at Engineers Without Borders, as well as the team at Publish What You Fund. Both groups have been working hard for over a year talking Canadian politicians and public servants through the ins and outs – as well as the benefits – of signing onto IATI. I’ve been working with both groups as well, pushing IATI when meeting with Federal Ministers (I recommended we make it part of our Open Government Partnership goals) as well as writing supportive op-eds in newspapers, so needless to say I’m excited about this development.

This really is good news. As governments become increasingly aware of the power data can have in facilitating cooperation and coordination as well as in improving effectiveness and efficiency, it will be critical to push standards around structuring and sharing data so that such coordination can happen easily across and between jurisdictions. IATI is a great example of such an effort and I hope there are more of these, with Canada taking an early lead, in the months and years ahead.

 

 

International Open Data Hackathon, Dec 3rd. It's coming together.

So a number of things have started to really come together for this Saturday Dec 3rd. I’ve noticed a number of new cities being tweeted about (hello Kuala Lumpur & Oakland!) and others adding themselves to the wiki. Indeed, we seem to be above 40 cities. It is hard to know how many people will be showing up in each given city, but in Vancouver I know that we already over 20 registered, while in Ottawa they are well above 40. If other cities have similar numbers it’s a great testament to the size of the community out there interested in playing with open government data.

A few thoughts to share with people as we get ready for the big day.

1. Leverage existing projects.

I’ve mentioned a few times that there are some great existing projects out there that can be easily leveraged.

In that vein I’ve noticed the good people at the Open Knowledge Foundation, who are behind OpenSpending (the project that powers WherDoesMyMoneyGo.org) have not only made their software easier to use but have put up some helpful instructions for creating your own instance up on the wiki. One hope I have for Saturday is that a number of different places might be able to visualize local budgets in much easier to understand ways. OpenSpending has the potential of being an enormously helpful tool for communities trying to understand their budget – hopefully we can provide some great examples and feedback for its creators.

In addition, the folks at MySociety have provided some helpful advice on the wiki for those interested in spinning up a version of MapIt for their country.

2. Get Data Now, Not on Saturday!

Here in Vancouver, my friend Luke C asked if we could get bicycle accident data for the city or province as he wanted to play around with it and maybe visualize it on December 3rd. It just so happened I had a contact at the Insurance Company of British Columbia (ICBC) which insures every vehicle in the province. I reached out and, after going through their request process, now have the data set to share with Luke.

The key piece here: now is the time to check and see if data you are interested in is available, see investigate what is out there, and request it from various stakeholders if it is not.

3. Share Your Code, Share your Data

Indeed, one advantage of having the BC bicycle accident data early is that I can start sharing it with people immediately. I’ve already uploaded the data set (all 6400 lines) onto BuzzData’s site here so others can download it, clone it, and share their own work on it. That way, even if Luke and I get separated, he’s still got something to hack on!

So please do let people know where they can find data you are hacking on, as well as project you’re hacking on. The Open Data Day Projects 2011 wiki page currently sits empty (as should be expected). But take a swing by the page 2010 project page, notice how it is quite full… I’d love to see us replicate this success. I’m hoping people link to not just their projects, but also Github repos, scraperwiki creations, BuzzData accounts and other places.

If you have a project and you think people in open data day hackathons in other cities might be interested, put it in the project page and tweet about it using the #odhd hashtag. You may discover there are people out there who feel as passionately about your project as you do!

4. Let’s Get Connected

Speaking of sharing, my friend Edward O-G, who is organizing the hackathon in Ottawa, did a great job last year setting up some infrastructure so people from different hackathons could video conference with one another. This year I think we’ll try using Google hangouts on google+. However, there is a non-trivial risk that this will not scale super well.

So…

Edward also suggested (brilliantly) that people create YouTube videos of whatever they create during the hackathon or in the days and weeks that follow. Please post those links to the Open Data Day Projects 2011 wiki page as well. There were a few projects last year that had youtube videos and they were very helpful, particularly when a project isn’t quite ready for prime time. It gives us a taste of what will be available. It also becomes something we can point people to.

5. Have Fun, Do What Is Interesting

Remember, Open Data Day is about meeting people, learning about open data, and working on something that you feel passionate about. This is all very decentralized and informal – no one is going to come and save your hackathon… it is up to you! So make sure you find something you think is worth caring about and work on it. Share your idea, and your passion, with others, that’s what makes this fun.

Can’t wait to hear what people are up to. Please feel free to email or tweet at me what you’re working on. I’d love to hear about it and blog about them.

 

Here in Vancouver, the open data hackathon will be happening at the offices of FoodTree, which has some of its own developers working on Open Food data.(If you haven’t signed up yet, definitely do so here).