Category Archives: open data

Developing Community Management Metrics and Tools for Mozilla

Background – how we got here

Over the past few years I’ve spent a great deal of time thinking about how we can improve both the efficiency of open source communities and contributors experience. Indeed, this was the focus, in part, of my talk at the Mozilla Summit last summer. For some years Diederik Van Liere – now with the Wikimedia foundation’s metrics team – and I have played with Bugzilla data a great deal to see if we could extract useful information from it. This led us to engaging closely with some members of the Mozilla Thunderbird team – in particular Dan Mosedale who immediately saw its potential and became a collaborator. Then, in November, we connected with Daniel Einspanjer of Mozilla Metrics and began to imagine ways to share data that could create opportunities to improve the participation experience.

Yesterday, thank’s to some amazing work on the part of the Mozilla Metrics team (listed at bottom of the post), we started sharing some of work at the Mozilla all hands. Specifically, Daniel demoed the first of a group of dashboards that describe what is going on in the Mozilla community, and that we hope, can help enable better community management. While these dashboards deal with the Mozilla community in particular I nonetheless hope they will be of interest to a number of open source communities more generally. (presently the link is only available to Mozilla staffers until the dashboard goes through security review – see more below, along with screen shots – you can see a screencast here).

Why – the contributor experience is a key driver for success of open source projects

My own feeling is that within the Mozilla community the products, like Firefox, evolve quickly, but the process by which people work together tends to evolve more slowly. This is a problem. If Mozilla cannot evolve and adopt new approaches with sufficient speed then potential and current contributors may go where the experience is better and, over time, the innovation and release cycle could itself cease to be competitive.

This task is made all the more complicated since Mozilla’s ability to fulfill its mission and compete against larger, better funded competitors depends on its capacity to tap into a large pool of social capital – a corps of paid and unpaid coders whose creativity can foster new features and ideas. Competing at this level requires Mozilla to provide processes and tools that can effectively harness and coordinate that energy at minimal cost to both contributors and the organization.

As I discussed in my Mozilla Summit talk on Community Management, processes that limit the size or potential of our community limit Mozilla. Conversely, making it easier for people to cooperate, collaborate, experiment and play enhances the community’s capacity. Consequently, open source projects should – in my opinion – constantly be looking to reduce or eliminate transactions costs and barriers to cooperation. A good example of this is how Github showed that forking can be a positive social contribution. Yes it made managing the code base easier, but what it really did was empower people. It took something everyone thought would kill open source projects – forking – and made it a powerful tool of experimentation and play.

How – Using data to enable better contributor experience

Unfortunately, it is often hard to quantitatively asses how effectively an open source community manages itself. Our goal is to change that. The hope is that these dashboards – and the data that underlies them – will provide contributors with an enhanced situational awareness of the community so they could improve not just the code base, but the community and its processes. If we can help instigate a faster pace of innovation of change in the processes of Mozilla, then I think this will both make it easier to improve the contributor experience and increase the pace of innovation and change in the software. That’s the hope.

That said, this first effort is a relatively conservative one. We wanted to create a dashboard that would allow us to identify some broader trends in the Mozilla Community, as well as provide tangible, useful data to Module Owners – particularly around identifying contributors who may be participating less frequently.

This dashboard is primarily designed to serve two purposes. First is to showcase what dashboards could be with the hope of inspiring the Mozilla community members to use it and, more importantly, to inspire them to build their own. The second reason was to provide module owners with a reliable tool with which to more effective manage their part of the community. So what are some of the ways I hope this dashboard might be helpful? One important feature is the ability to sort contributors by staff or volunteer. An open source communities volunteer contributors should be a treasured resource. One nice things about this dashboard is that you can not only see just volunteers, but you can get a quick sense of those who haven’t submitted a patch in a while.

In the picture below I de-selected all Mozilla employees so that we are only looking at volunteer contributors. Using this view we can see who are volunteers who are starting to participate less – note the red circle marked “everything okay?” A good community manager might send these people an email asking if everything is okay. Maybe they are moving on, or maybe they just had a baby (and so are busy with a totally different type of patch – diapers), but maybe they had a bad experience and are frustrated, or a bunch of code is stuck in review. These are things we would want to know, and know quickly, as losing these contributors would be bad. In addition, we can also see who are the emerging power contributors – they might be people we want to mentor, or connect with mentors in order to solidify their positive association with our community and speed up their development. In my view, this should be core responsibilities of community managers and this dashboard makes it much easier to execute on these opportunities.

Below you can see how zooming in more closely allows you to see trends for contributors over time. Again, sometimes large changes or shifts are for reasons we know of (they were working on features for a big release and its shipped) but where we don’t know the reasons maybe we should pick up the phone or email this person to check to see if everything is okay.

Again, if this contributor had a negative experience and was drifting away from the community – wouldn’t we want to know before they silently disappeared and moved on? This is in part the goal.

Some of you may also like the fact that you can dive a little deeper by clicking on a user to see what specific patches that user has worked on (see below).

Again, these are early days. My hope is that other dashboards will provide still more windows into the community and its processes so as to show us where there are bottlenecks and high transaction costs.

Some of the features we’d like to add to this or other dashboards include:

a code-review dashboard that would show how long contributors have been waiting for code-review, and how long before their patches get pushed. This could be a powerful way to identify how to streamline processes and make the experience of participating in open source communities better for users.
a semantic analysis of bugzilla discussion threads. This could allow us to flag threads that have become unwieldy or where people are behaving inappropriately so that module owners can better moderate or problem solve them
a dashboard that, based on your past bugs and some basic attributes (e.g. skillsets) informs newbies and experienced contributors which outstanding bugs could most use their expertise
Ultimately I’d like to see at least 3 core dashboards – one for contributors, one for module owners and one for overall projects – emerge and, as well as user generated dashboards developed using Mozilla metrics data.
Access to all the data in Bugzilla so the contributors, module owners, researchers and others can build their own dashboards – they know what they need better than we do

What’s Next – How Do I Get To Access it and how can I contribute

Great questions.

At the moment the dashboard is going through security review which it must complete before being accessible. Our hope is that this will be complete by the end of Q2 (June).

More importantly, we’d love to hear from contributors, developers and other interested users. We have a standing weekly call every other Friday at 9am PST where we discuss development issues with this and the forthcoming code-review dashboard, contributors needs and wanted features, as well as use cases. If you are interested in participating on these calls please either let me know, or join the Mozilla Community Metrics Google group.

Again, a huge shout out is deserved by Daniel Einspanjer and the Mozilla Metrics team. Here is a list of contributors both so people know who they are but also in case anyone has question about specific aspects of the dashboard:
Pedro Alves – Team Lead
Paula Clemente – Dashboard implementor
Nuno Moreira – UX designer
Maria Roldan – Data Extraction
Nelson Sousa – Dashboard implementor

City of Vancouver Wins Top Innovator Award from BC Business

Calgary Launches Business Plan and Budget App

1 Reply

So this is interesting. The City of Calgary has launched a Business Plan & Budget app for free from iTunes.

It’s a smart move as it creates an easy, “one button” option for citizens to participate in and learn about the city’s financial planning process. You can read (a tiny bit) more at the City of Calgary’s blog.

Looking more closely at the app, it doesn’t offer a huge amount but don’t dismiss it too quickly. Consolidating all the information into a single place and making it available to people on the go is a great starting point. Secondly, it is worth remembering that this is just a starting point – there is obviously lots to be learned about how to engage citizens online – especially using mobile technology. If this is done right, Calgary will be learning these lessons first, which means their 2nd and 3rd generation versions of the app and the process will be more sophisticated while others are left catching up (think of Apple and the iPad).

So while the app is fairly light on features today… I can imagine a future where it becomes significantly more engaging and comprehensive, using open data on the data and city services to show maps of where and how money is spent, as well as post reminders for in person meet ups, tours of facilities, and dial in townhall meetings. The best way to get to these more advanced features is to experiment with getting the lighter features right today. The challenge for Calgary on this front is that it seems to have no plans for sharing much data with the public (that I’ve heard of), it’s open data portal has few offerings and its design is sorely lacking. Ultimately, if you want to consult citizens on planning and the budget it might be nice to go beyond surveys and share more raw data and information with them, it’s a piece of the puzzle I think will be essential. This is something no city seems to be tackling with any gusto and, along with crime data, is emerging as a serious litmus test of a city’s intention to be transparent.

The possibilities that Calgary’s consultation app presents are exciting – and again it is early days – so it will be interesting if developers in Calgary and elsewhere can begin to figuring out how to easily extend and enhance this type of approach. Moreover, it’s nice to see a city venturing out and experimenting with this technology, I hope other cities will not just watch, but start experiments of their own, it’s the best way to learn.

Access to Information is Fatally Broken… You Just Don’t Know it Yet

13 Replies

I’ve been doing a lot of thinking about access to information, and am working on a longer analysis, but in the short term I wanted to share two graphs – graphs that outline why Access to Information (Freedom of Information in the United States) is unsustainable and will, eventually, need to be radically rethought.

First, this analysis is made possible by the enormous generosity of the Canadian Federal Information Commissioners Office which several weeks ago sent me a tremendous amount of useful data regarding access to information requests over the past 15 years at the Treasury Board Secretariat (TBS).

The first figure I created shows both the absolute number of Access to Information Requests (ATIP) since 1996 as well as the running year on year percentage increase. The dotted line represents the average percentage increase over this time. As you can see the number of ATIP requests has almost tripled in this time period. This is very significant growth – the kind you’d want to see in a well run company. Alas, for those processing ATIP requests, I suspect it represents a significant headache.

That’s because, of course, such growth is likely unmanageable. It might be manageable if say, the costs of handling each requests was dropping rapidly. If such efficiencies were being wrestled out of the system of routing and sorting requests then we could simply ignore the chart above. Sadly, as the next chart I created demonstrates this is not the case.

In fact the costs of managing these transactions has not tripled. It has more than quadrupled. This means that not only are the number of transactions increasing at about 8% a year, the cost of fulfilling each of those transactions is itself rising at a rate above inflation.

Now remember, I’m not event talking about the effectiveness of ATIP. I’m not talking about how quickly requests are turned around (as the Information Commissioner has discussed, it is broadly getting worse) nor am I discussing less information is being restricted (it’s not, things are getting worse). These are important – and difficult to assess – metrics.

I am, instead, merely looking at the economics of ATIP and the situation looks grim. Basically two interrelated problems threaten the current system.

1) As the number of ATIP requests increase, the manpower required to answer them also appears to be increasing. At some point the hours required to fulfill all requests sent to a ministry will equal the total hours of manpower at that ministry’s disposal. Yes that day may be far off, but they day where it hits some meaningful percentage – say 1%, 3% or 5% of total hours worked at Treasury Board, may not be that far off. That’s a significant drag on efficiency. I recall talking to a foreign service officer who mentioned that during the Afghan prisoner scandal an entire department of foreign service officers – some 60 people in all – were working full time on assessing access to information requests. That’s an enormous amount of time, energy and money.

2) Even more problematic than the number of work hours is the cost. According to the data I received, Access to Information requests costs The Treasury Board $47,196,030 last year. Yes, that’s 47 with a “million” behind it. And remember, this is just one ministry. Multiply that by 25 (let’s pretend that’s the number of ministries, there are actually many more, but I’m trying to be really conservative with my assumptions) and it means last year the government may have spent over $1.175 Billion fulfilling ATIP requests. That is a staggering number. And its growing.

Transparency, apparently, is very, very expensive. At some point, it risks becoming too expensive.

Indeed, ATIP reminds me of healthcare. It’s completely unsustainable, and absolutely necessary.

To be clear, I’m not saying we should get rid of ATIP. That, I believe, to be folly. It is and remains a powerful tool for holding government accountable. Nor do I believe that requesters should pay for ATIP requests as a way to offset costs (like BC Ferries does) – this creates a barrier that punishes the most marginalized and threatened, while enabling only the wealthy or well financed to hold government accountable.

I do think it suggests that governments need to radical rethink how manage ATIP. More importantly I think it suggests that government needs to rethink how it manages information. Open data, digital documents are all part of a strategy that, I hope, can lighten the load. I’ve also felt that if/as government’s move their work onto online platforms like GCPEDIA, we should simply make non-classified pages open to the public on something like a 5 year timeline. This could also help reduce requests.

I’ve more ideas, but at its core we need a system rethink. ATIP is broken. You may not know it yet, but it is. The question is, what are we going to do before it peels off the cliff? Can we invent something new and better in time?

Election Mashup!

1 Reply

Since we are, apparently, heading into an election up here in Canada, I thought it would be great time to share this fantastic website my friend Diederik Van Liere recently pointed out to me.

The site, created by Montreal developer Cedric Sam, is a mashup of 2008 federal election and polling data, federal open data from the Geogratis website and Google Earth. It allows users to see how support for candidates was distributed within ridings. Something any political junkie could enjoy.

You can read more about the project on Cedric’s blog. Here’s his description:

I used cartographic data from the Geogratis.gc.ca website. I imported the Shapefiles to a PostgreSQL database with Postgis. Then, I processed results by polling divisions from the 2008 election, data available on the Elections Canada website. It was put in a separate table on the same database. A custom program in Python using the very handy libkml (a code library developed and supported by Google) took the data and outputted pretty KML code. It was packed as a KMZ and uploaded to my webspace. [E-mail me, if you want to exchange ideas on the code].

I of course love the fact that Cedric, like many great hackers I meet, is always keen to work with others to make the code better or explore how it might be enhanced.

As I now live in Vancouver Centre I obviously couldn’t resist pulling it up. Here’s a screen shot… but I encourage you to check out the site yourself!

Canada launches data.gc.ca – what works and what is broken

55 Replies

Those on twitter will already know that this morning I had the privilege of conducting a press conference with Minister Day about the launch of data.gc.ca – the Federal Government’s Open Data portal. For those wanting to learn more about open data and why it matters, I suggest this and this blog post, and this article – they outline some of the reasons why open data matters.

In this post I want to review what works, and doesn’t work, about data.gc.ca.

What works

Probably the most important thing about data.gc.ca is that it exists. It means that public servants across the Government of Canada who have data they would like to share can now point to a website that is part of government policy. It is an enormous signal of permission from a central agency that will give a number of people who want to share data permission, a process and a vehicle, by which to do this. That, in of itself, is significant.

Indeed, I was informed that already a number of ministries and individuals are starting to approach those operating the portal asking to share their data. This is exactly the type of outcome we as citizens should want.

Moreover, I’ve been told that the government wants to double the number of data sets, and the number of ministries, involved in the site. So the other part that “works” on this site is the commitment to make it bigger. This is also important, as there have been some open data portals that have launched with great fanfare, only to have the site languish as neither new data sets are added and the data sets on the site are not updated and so fall out of date.

What’s a work in progress

The number of “high value” datasets is, relatively speaking, fairly limited. I’m always cautious about this as, I feel, what constitutes high value varies from user to user. That said, there are clearly data sets that will have greater impact on Canadians: budget data, line item spend data by department (as the UK does), food inspection data, product recall data, pretty much everything on the statscan website, Service Canada locations, postal code data and, mailbox location data, business license data, Canada Revenue data on charities and publicly traded companies are all a few that quickly come to mind, clearly I can imagine many, many more…

I think the transparency, tech, innovation, mobile and online services communities will be watching data.gc.ca closely to see what data sets get added. What is great is that the government is asking people what data sets they’d like to see added. I strongly encourage people to let the government know what they’d like to see, especially when it involves data the government is already sharing, but in unhelpful formats.

What doesn’t work

In a word: the license.

The license on data.gc.ca is deeply, deeply flawed. Some might go so far as to say that the license does not make it data open at all – a critique that I think is fair. I would say this: presently the open data license on data.gc.ca effectively kills any possible business innovation, and severally limits the use in non-profit realms.

The first, and most problematic is this line:

“You shall not use the data made available through the GC Open Data Portal in any way which, in the opinion of Canada, may bring disrepute to or prejudice the reputation of Canada.”

What does this mean? Does it mean that any journalist who writes a story, using data from the portal, that is critical of the government, is in violation of the terms of use? It would appear to be the case. From an accountability and transparency perspective, this is a fatal problem.

But it is also problematic from a business perspective. If one wanted to use a data set to help guide citizens around where they might be well, and poorly, served by their government, would you be in violation? The problem here is that the clause is both sufficiently stifling and sufficiently negative that many businesses will see the risk of using this data simply too great.

UPDATE: Thursday March 17th, 3:30pm, the minister called me to inform me that they would be striking this clause from the contract. This is excellent news and Treasury Board deserves credit for moving quickly. It’s also great recognition that this is a pilot (e.g. beta) project and so hopefully, the other problems mentioned here and in the comments below will also be addressed.

It is worth noting that no other open data portal in the world has this clause.

The second challenging line is:

“you shall not disassemble, decompile except for the specific purpose of recompiling for software compatibility, or in any way attempt to reverse engineer the data made available through the GC Open Data Portal or any part thereof, and you shall not merge or link the data made available through the GC Open Data Portal with any product or database for the purpose of identifying an individual, family or household or in such a fashion that gives the appearance that you may have received or had access to, information held by Canada about any identifiable individual, family or household or about an organization or business.”

While I understand the intent of this line, it is deeply problematic for several reasons. First, many business models rely on identifying individuals, indeed, frequently individuals ask businesses to do this. Google, for example, knows who I am and offers custom services to me based on the data they have about me. It would appear that terms of use would prevent Google from using Government of Canada data to improve its service even if I have given them permission. Moreover, the future of the digital economy is around providing customized services. While this data has been digitized, it effectively cannot be used as part of the digital economy.

More disconcerting is that these terms apply not only to individuals, but also to organizations and businesses. This means that you cannot use the data to “identify” a business. Well, over at Emitter.ca we use data from Environment Canada to show citizens facilities that pollute near them. Since we identify both the facilities and the companies that use them (not to mention the politicians whose ridings these facilities sit in), are we not in violation of the terms of use? In a similar vein, I’ve talked about how government data could have prevented $3B of tax fraud. Sadly, data from this portal would not have changed that since, in order to have found the fraud, you’d have to have identified the charitable organizations involved. Consequently, this requirement manifestly destroys any accountability the data might create.

It is again worth noting that no other open data portal in the world has this clause.

And finally:

4.1 You shall include and maintain on all reproductions of the data made available through the GC Open Data Portal, produced pursuant to section 3 above, the following notice:

Reproduced and distributed with the permission of the Government of Canada.

4.2 Where any of the data made available through the GC Open Data Portal is contained within a Value-Added Product, you shall include in a prominent location on such Value-Added Product the following notice:

This product has been produced by or for (your name – or corporate name, if applicable) and includes data provided by the Government of Canada.

The incorporation of data sourced from the Government of Canada within this product shall not be construed as constituting an endorsement by the Government of Canada of our product.

or any other notice approved in writing by Canada.

The problem here is that this creates what we call the “Nascar effect.” As you use more and more government data, these “prominent” displays of attribution begin to pile up. If I’m using data from 3 different governments, each that requires attribution, pretty soon all your going to see are the attribution statements, and not the map or other information that you are looking for! I outlined this problem in more detail here. The UK Government has handled this issue much, much more gracefully.

Indeed, speaking of the UK Open Government License, I really wish our government had just copied it wholesale. We have a similar government system and legal systems so I see no reason why it would not easily translate to Canada. It is radically better than what is offered on data.gc.ca and, by adopting it, we might begin to move towards a single government license within Commonwealth countries, which would be a real win. Of course, I’d love it if we adopted the PDDL, but the UK Open Government License would be okay to.

In Summary

The launch of data.gc.ca is an important first step. It gives those of us interested in open data and open government a vehicle by which to get more data open and improve the accountability, transparency as well as business and social innovation. That said, there is much work to be done still: getting more data up and, more importantly, addressing the significant concerns around the license. I have spoken to Treasury Board President Stockwell Day about these concerns and he is very interested and engaged by them. My hope is that with more Canadians expressing their concerns, and with better understanding by ministerial and political staff, we can land on the right license and help find ways to improve the website and program. That’s why we to beta launches in the tech world, hopefully it is something the government will be able to do here too.

Apologies for any typos, trying to get this out quickly, please let me know if you find any.

MP Jim Abbott: The Face of the Sad State of Open Data in Canada

4 Replies

“I guess my attack to this has always been from the perspective of are we working in a bubble. In other words, when this was… under this initiative by the President, how quick was the takeup by the population at large? Not by the people that we affectionately call geeks, or people who don’t have a life, or don’t come up out of the dark, or whatever. The average person walking through Times Square I guess is what I’m trying to say. How quick was their take up, and in fact has there been a takeup?”

– Jim Abbott, ETHI Meeting No. 47, Open Government Study, March 2, 2011

Yes, the above quote comes from Jim Abbott, Member of Parliament (Conservative) for Kootenay—Columbia during the testimony of Beth Noveck, President Obama’s former Deputy Chief Technology Officer for Open Government (her statement can be found here). You can see the remarks in the online video here, at around the 1:17:50 mark.

First, I want to be clear. This is disappointing, not on a political level, but on an individual level. During my testimony for the ETHI committee (which I intend to blog about) I found members of all parties – NDP, Liberal, Bloc Quebecois and Conservative – deeply interested in the subject matter, asking thoughtful questions and expressing legitimate concerns. Indeed, I was struck by Pierre Poilievre, the Conservative MP for Napean-Carleton, who asked a number of engaging questions, particularly around licenses. That’s a level of sophistication around the issue that many people don’t care to ask about. Moreover, many of the committee members grasped the economic and social opportunity around open data.

Jim Abbott, in contrast, may believe that describing technologists and geeks as people who “don’t have a life” or “don’t come up out of the dark” is affectionate, but I’m not so sure these stereotypes are so endearing, especially given how they aren’t true. Moreover, his comments are particularly unfortunate as it’s the people he (affectionately) demeans who created RIM, OpenText, Cognos, and thousands of other successful technology companies that pump billions into the Canadian economy, employ hundreds of thousands, and do actually impact the “person on the street.” But a few simple demeaning words can make one forget these contributions or worse, make them sound insignificant.

Of course, it will be the work of these people that creates the open data applications that, in the US at least, already impact the average person walking through Times Square (consider this lifesaving app that was created by a hacker using opendata). Indeed, there are a growing number of businesses consuming and using open data, some even valued in the billions of dollars and used by millions of americans every day.

The sad part is they will only be available to the people in Times Square, or Trafalgar Square or on the Champs-Élysées since the Americans, British and French all have national open data portals (among numerous other countries). There will be no uptake for people on Wellington St., Queen St., Robson St. or wherever, since without a national open data portal in Canada, there can be no uptake. (It’s not easy to be behind the French government on an issue related to the digital economy, but we’ve somehow managed).

But forget the economic opportunity. There is also the question of government transparency and accountability. What makes the above statement so disappointing is that it exposes how an MP who for so long railed for greater transparency in government, has suddenly decided that transparency is no longer important unless “there is sufficient uptake.”

One wonders what Jim Abbott of 2000 would say of Jim Abbott of 2011? Because back in a pre-2001 era Jim Abbott had fantastic quotes like this:

I suggest in the strongest way possible to the minister that even if we can get him to clear up the history of the Canada Information Office, which I do not have a lot of hope for but I am asking for, from this point forward there must be proper transparency of the Canada Information Office. The country needs openness and transparency because democracy cannot be true democracy without openness and transparency.

– Jim Abbott, June 8th, 2000 / 11:10 a.m.

and this

Second, the difficulty the government has created with the Canada Information Office is that many of the contracts and much of the ongoing activity have been conducted in a way that does not befit what we are in Canada, which is a democracy. In a democracy the people depend on the people in the Chamber to hold the government accountable for the affairs of the government and to be as transparent as possible.

– Jim Abbott, June 8th, 2000 / 11:10 a.m.

and this

It will never have the transparency that it must have in a democracy. It is just absolutely unacceptable.

– Jim Abbott, June 16th, 1995 / 3:25 p.m.

I could go on…

(If you are wondering how I was able to dig up these quotes, please check out OpenParliament.ca – it really is extraordinary tool and again, shows the power of open (parliamentary) data).

But more importantly, and on point, it seems to me that Jim Abbott from the year 2000 would see open data as a important way to ensure greater transparency. Wouldn’t it have been nice if the Canada Information Office had had its budget and expenditures available as open data? Wouldn’t that have brought about some of the accountability the 2000 Jim Abbott would have sought? Sadly, and strangely, Jim Abbott of 2011 no longer seems to feel that way.

Yes, if only he could meet Jim Abbott of 2000, I think they’d have a great debate.

Of course, Jim Abbott of 2000 can’t meet Jim Abbott of 2011, and so it is up to us to (re)educate him. And on that front, I have, so far, clearly failed the tech community, the open data community and the government accountability community. Hopefully with time and more effort, that will change. Maybe next time I’m in Ottawa, Jim Abbott and I can grab coffee and I can try again.

Launching an Open Data Business: Recollect.net (Vantrash 2.0)

1 Reply

Have you ever forgotten to take the garbage or recycling out? Wouldn’t it be nice if someone sent you a reminder the night before, or the morning of? Maybe an email, or an SMS, or even a phone call?

Now you can set it up so somebody does. Us.

Introducing Recollect: the garbage and recycling collection reminder service.

For People

We’ve got the garbage schedules for a number of Canadian cities big and small (with American ones coming soon) – test our site out to see if we support yours.

You can set up a reminder for the night before – or the day of – your garbage pickup, and we’ll email, text or call you letting you know your garbage day is imminent and what will be picked up (say, recycling, yard waste or garbage). Our email and Twitter reminders are free, and text message and phone calls cost $1.50 a month.

If you think you, your sibling, friends, or your parents might like a service like this, please come check out our website.

It’s simple and we hope you’ll give it a whirl.

For Cities

We don’t think that Recollect is going to change the world, but we do think we can help better manage citizens’ expectations around customer service. For cities (and companies) interested in connecting with their citizens and customers, we have have a number of partnering options we have already started to explore with some cities.

More importantly, if you’d like to see Recollect come to your city, have your garbage schedule and zones available for download – like Edmonton and Vancouver.

On either of these fronts, if you are a politician, city employee or a business owner who needs a reminder service of some kind, please contact us.

Background – an open data municipal business

In June of 2009, as Vancouver was preparing to launch its open data portal I wrote a blog post called How Open Data even makes Garbage collection sexier, easier and cheaper in which I talked about how, using city data, a developer could create a garbage pickup reminder service for Vancouverites. Tim Bray called it his Hello World moment for Open Data. More importantly, Luke Closs and Kevin Jones, two Vancouver programers (and now good friends) took the idea and made it real. The program was called Vantrash, and in two quiet, low-maintenance years – with no advertising or marketing – it garnered over 3000 users.

Last week we retired Vantrash. Today, we launched Recollect.

Yes, Recollect is more beautiful than its predecessor, but more importantly it is going to start serving your community. At a high level, we want to see if we can scale an open data business to a continental level. Can we use open data to serve a range of cities across North America?

At a practical level, the goal of Recollect is more basic: To help make citizens’ lives just a little bit easier by providing them customized reminders for services they use, to the device of their choice, at the time of their choice.

Let’s face it: We are all too busy being parents, holding down jobs or enjoying the limited free time we have to remember things like garbage day or little league schedules. Our job is to make your life easier by finding ways to free our minds of wasting time remembering these small details. If you aren’t trying to remember to take out the garbage, hopefully it means you can spend a little more time thinking about your family, your work or whatever your passion may be.

In short, we believe that city services should be built around your life – and we are trying to take a small step to bring that a little closer to reality.

Again, we don’t expect Recollect to change the world. But we do hope that it will serve as a building block for rethinking the government-user experience that will lay the foundations so that others will be able to change the world.

The Curious Case of Media Opposing Government Transparency

10 Replies

My gosh there is a lot going on. Republicans – REPUBLICANS(!) who were in charge of America’s prison system are warning Canada not to follow the Conservatives plan on prisons, the Prime Minister has renamed the government, after himself and my friends at Samara had in Toronto the Guardian’s Emily Bell to talk wikileaks and data journalism (wish I could have been there).

It’s all very interesting… and there is a media story here in British Columbia that’s been brewing where a number of journalists have become upset about a government that has become “too” transparent.

It’s an important case as it highlights some of the tensions that will be emerging in different places as governments rethink how they share information.

The case involves BC Ferries, a crown corporation that runs ferries along critical routes around the province. For many years the company was not subject to the province’s Freedom of Information legislation. However, a few months ago the government stated the crown corporation would need to comply with the act. This has not pleased the corporation’s president.

To comply with the act BC Ferries has created an FOI tracker website on which it posts the text of FOI requests received. Once the records are processed they are posted online and some relevant listservs. As a result they can be read by an audience (that cares).

Broadly, journalists, are up in arms for two reasons. One bad, the other even worse.

The terrible reasons was raised by Chad Skelton (who’s a great reporter for whom I have a lot of respect and whose column should be read regularly).

Skelton argues that BC Ferries deserves part of the blame for stories with errors as the process lead news agencies to rush (carelessly) in order to beat each other in releasing the story. This is a disappointing position. It’s the news media’s job to get the facts right. (It’s also worth noting here that Skelton’s own media organizations did not make the mistakes in question). Claiming that BC Ferries is even partly responsible seems beyond problematic since they are in no way involved in the fact and error checking processes. We trust the media (and assess it) to get facts right in fast moving situations… why should this be different?

More interesting is the critique that this model of transparency undermines the ability of journalists to get a scoup and thus undermines the business model of traditional media.

What makes this so interesting is that is neither true nor, more importantly, relevant.

First, it’s not the job of government to support the business model of the media. The goal of government should be to be as transparent as possible about its operations. This can, and should, include its FOI requests. Indeed, one thing I like about this process is that an FOI request that is made but isn’t addressed starts to linger on the site – and that the organization can be held to account, publicly, for the delay. More importantly, however, I’m confident that the media will find new ways to exploit the process and that, while painful, new business models will emerge.

Second, the media is not the only user of FOI. It strikes me as problematic to expect that the FOI system should somehow be tailored to meet needs alone. Individuals, non-profits, businesses, opposition politicians and others all use the FOI process. Indeed, the policy strengthens many of these use cases since, as mentioned above, delays in processing will be visible and open the organization up to greater pressure and scrutiny. Why are all the use cases of these other institutions somehow secondary to those of journalists and the media? Indeed, the most important use case – that of the citizen – is better served. Isn’t that the most important outcome?

Third, this form of transparency could make for better media. One of my favourite quotes (which I got via Tim O’Reilly) comes from Clayton Christensen in a 2005 Harvard Business Review article:

“When attractive profits disappear at one stage in the value chain because a product becomes modular and commoditized, the opportunity to earn attractive profits with proprietary products will usually emerge at an adjacent stage.”

So BC Ferries has effectively commoditized FOI requests. That simply means that value will shift elsewhere. One place it could shift to is analysis. And wouldn’t that be a good thing to have the media compete on? Rather than simply who got the fact fastest (a somewhat silly model in the age of the internet) readers instead started to reward the organization with the best insights? Indeed, it makes me think that on superficial issues, like say, the salary of an employee, it may be hard for one individual or organization to scoop another. But most often the value of these stories is also pretty low. On a more significant story, one that requires research and digging and a knowledge of the issue, it’s unclear that transparency around FOI requests will allow others to compete. More interestingly, some media organizations, now that they have access to all FOI requests, might start analyzing them for deeper more significant patterns or trends that might reveal more significant problems that the current scattered approach to FOI might never reveal.

What’s also been interesting is the reaction stories by journalists complaining about this issue have been received. It fits nicely in with the piece I wrote a while ago (and now published as part of a journalism textbook) about Journalism in an Open Era. The fact is, the public trust of opaque institutions is in decline – and the media is itself a pretty opaque institution. Consider these three separate comments people wrote after the stories I’ve linked to above:

“I wonder over the years how many nuggets of information reporters got through FOI but the public never heard about because they didn’t deem it “newsworthy”. Or worse, that it was newsworthy but didn’t follow their storyline.” (found here)

“And the media whining about losing scoops — well, tough beans. If they post it all online and give it to everyone, they are serving the public –the media isn’t the public, and never has been.” (found here)

“The media’s track record, in general, for owning up to its blunders continues to be abysmal. Front page screw-ups are fixed several days (or weeks) later with a little “setting it straight” box buried at the bottom of P. 2 — and you think that’s good enough. If the media were more open and honest about fixing its mistakes, I might cut you a little slack over the BC Ferries’ policy of making your life difficult. But whining about it is going to be counterproductive, as you can see from most of the comments so far.” (found here)

While some comments were supportive of the articles, the majority have not been. Suggesting that at the minimum that the public does not share the media’s view that this new policy is a “controversial.”

This is not, of course, to say that BC Ferries implemented its policy because it sought to do the right thing. I’m sure it’s president would love for their to be fewer requests and impede the efforts of journalists. I just happen to think he will fail. Dismally. More concerning is the fact that FOI requests are not archived on the site and are removed after a few months. This is what should get the media, the public and yes, the Information and Privacy Commissioner, up in arms.

Today in the Toronto Star: End the silence on aid

2 Replies

Sorry for the cross post – I have this piece today on the opinion page of the Toronto Star. They’ve actually done a nice graphic for it so do encourage you to check it out.

End the silence on aid

For the past two weeks, Canadians have slowly watched the minister of international development, Bev Oda, implode. Caught in a slowly escalating scandal, it’s become clear that the minister misled Parliament — and the public — about how the government chooses whom it funds to do international development work.

The scandal around Oda, however, is a metaphor for a much larger problem in Canada’s foreign aid. The world is dividing itself into donors who hold forth an open model of evidence, accountability and, above all, transparency, and those who cling to a model of patronage, ideology and opacity.

So the question is: Where will Canada land on this debate? So far, the answer is not promising.

Internationally, the Kairos decision suggests Canada is on the wrong side of the divide. Indeed, the gap between CIDA and the world’s leading institutions is growing. Consider a recent report by the U.K.-based international advocacy group Publish What You Fund. Of the 30 institutions assessed in its 2010 report on aid transparency, the Canadian International Development Agency ranked 23rd. Among countries, Canada ranked 15th out of 22 (the Netherlands, U.K. and Ireland held the top three spots).

We are, by any metric, near the bottom of the pack. For a country and a government that prides itself on accountability and transparency, it’s a damning assessment.

What’s all the more frustrating is that transparency isn’t just about accountability. It’s about effectiveness and saving taxpayers’ money — something our major allies have already figured out.

So while Canada’s international development minister fights allegations of making the decision-making process more opaque, a coalition of leading countries is moving forward — without Canada — to do the opposite.

Take, for example, the newly founded International Aid Transparency Initiative (IATI). A coalition of donor governments, developing countries and NGOs, the IATI has a single goal: to improve aid effectiveness by making information about aid spending easier to access, use and understand.

It’s a deeply pragmatic exercise, one far removed from the partisan politics around aid seen in Canada. In one of its first reports, it outlines how setting up systems to make aid data available would involve a one-time cost of between $50,000 and $500,000, but would save taxpayers in countries like Canada several times that amount every year.

Part of these savings would come just from reducing bureaucracy. Making data publicly available would eliminate the need for civil servants to respond to duplicate information requests from international organizations, other governments and Canadian organizations. Instead, the relevant information could just be downloaded. It’s the kind of efficiency we expect from our government.

It’s also the kind of transparency Canadians are starting to see elsewhere. The World Bank — at one time loathed for its opacity — has made transparency a core value of its operations. It recently launched an open data portal where it shares enormous quantities of information on the global economy and aid projects. It has also promised much more and is slowly rolling out a “mapping for results” website where every project the bank funds and how much money it receives can be viewed on a downloadable map.

Canada sits on the sidelines while others move forward implementing proposals that could — ironically — fund several Kairoses every year.

The costs aren’t borne just by taxpayers, but also by Canadian NGOs. They have to provide the same information, but in different forms, to every government and organization that funds them. This means aid workers spend precious time and money filling out CIDA’s unique forms. Repeat this cost over the hundreds of projects that CIDA funds and the collective waste is enormous.

Perhaps more importantly, making our aid more transparent and accessible would close another gap — our inability to measure our effectiveness. One of the reasons countries like the U.K., Denmark and Sweden have signed up to the International Aid Transparency Initiative is so they can more easily compare the projects they fund with one another. These are countries that are serious about getting bang for their buck — they want to compare the evidence, see which projects work, and which ones fail.

It’s a lesson leading Canadian organizations are taking to heart. Engineers Without Borders, for example, regularly publishes a “failure report” in which it outlines which of its projects didn’t work and why. This honest, open and evidence-based approach to development is exactly what we need to demand of our government. Anything less constitutes a waste of our tax dollars.

And yet, the current debate in Parliament suggests we may be mapping a different route — one of opaque, ideologically driven development that is blind to both effectiveness and accountability. This serves neither Canadians nor donor recipients well.

Regardless of whether Oda resigns, Canadians should not lose sight of the larger issue and opportunity. We are in the midst of a global movement for international development aid transparency.

The benefits are clear, our allies are present, and even five of our focus recipient countries have signed up. And yet, Canada is nowhere to be found.

eaves.ca

if writing is a muscle, this is my gym

Category Archives: open data

Developing Community Management Metrics and Tools for Mozilla

City of Vancouver Wins Top Innovator Award from BC Business

Calgary Launches Business Plan and Budget App

Access to Information is Fatally Broken… You Just Don’t Know it Yet

Election Mashup!

Canada launches data.gc.ca – what works and what is broken

MP Jim Abbott: The Face of the Sad State of Open Data in Canada

Launching an Open Data Business: Recollect.net (Vantrash 2.0)

The Curious Case of Media Opposing Government Transparency

Today in the Toronto Star: End the silence on aid

End the silence on aid