Category Archives: open data

Sharing Critical Information with the public: Lessons for Governments

Increasingly governments are looking for new and more impactful ways to communicate with citizens. There is a slow but growing awareness that traditional sources of outreach, such as TV stories and newspaper advertisements are either not reaching a significant portion of the population and/or have little impact on raising awareness of a given issue.

The exciting thing about this is that there is some real innovation taking place in governments as they grapple with this challenge. This blog post will look at one example from Canada and talk about why the innovation pioneered to date – while a worthy effort – falls far short of its potential. Specifically, I’m going to talk about how when governments share data, even when they use new technologies, they remain stuck in a government-centric approach that limits effectiveness. The real impact of new technology won’t come until governments begin to think more radically in terms of citizen-centric approaches.

The dilemma around reaching citizens is probably felt most acutely in areas where there is a greater sense of urgency around the information – like, say, in issues relating to health and safety. Consequently, in Canada, it is perhaps not surprising to see that some of the more innovative outreach work has thus been pioneered by the national agency responsible for many of these issues, Health Canada.

The most cutting edge stuff I’ve seen is an effort by Health Canada to share advisories from Health Canada, Transport Canada and the Canadian Food Inspection Agency via three vehicles: an RSS feed, a mobile app available for Blackberry, iPhone (pictured far right) and Android, and finally as a widget (pictured near right) that anyone can install into their blog.

I think all of these are interesting ideas and have much to commend them. It is great to see information of a similar type, from three different agencies, being shared through a single vehicle – this is definitely a step forward from a user’s perspective. It’s also nice to see the government experiment with different vehicles for delivery (mobile and other parties’ websites).

But from a citizen-centric perspective, all these innovations share a common problem: They don’t fundamentally change the citizen’s experience with this information. In other words, they are simply efforts to find new ways to “broadcast” the information. As a result, I predict that these intiatives will have a minimal impact as currently structured. There are two reasons why:

The problem isn’t about access: These tools are predicated on the idea that the problem to conveying this information is about access to the information. It isn’t. The truth is, people don’t care. We can debate about whether they should care but the fact of the matter is, they don’t. Most people won’t pay attention to a product recall until someone dies. In this regard these tools are simply the modern day version of newspaper ads, which, historically, very few people actually paid attention to. We just couldn’t measure it, so we pretended like people read them.

The content misses the mark: Scrape a little deeper on these tools and you’ll notice something. They are all, in essence, press releases. All of these tools, the RSS feed, blog widget and mobile apps, are simply designed to deliver a marginally repackaged press release. Given that people tuned out of newspaper ads, pushing these ads onto them in another device will likely have a limited impact.

As a result, I suspect that those likely to pay attention to these innovations were probably those who were already paying attention. This is okay and even laudable. There is a small segment of people for whom these applications reduce the transactions costs of access. However, with regard to expanding the numbers of Canadians impacted my this information or changing behaviour in a broader sense, these tools have limited impact. To be blunt, no one is checking a mobile application before they buy a product, nor are they reading these types of widgets in a blog, nor is anyone subscribing to an RSS feed of recalls and safety warnings. Those who are, are either being paid to do so (it is a requirement of their job) or are fairly obsessive.

In short, this is a government-centric solution – it seeks to share information the government has, in a context that makes sense to government – it is not citizen-centric, sharing the information in a form that matters to citizens or relevant parties, in a context that makes sense to them.

Again, I want to state while I draw this conclusion I still applaud the people at Health Canada. At least they are trying to do something innovative and creative with their data and information.

So what would a citizen-centric approach look like? Interestingly, it would involve trying to reach out to citizens directly.

People are wrestling with a tsunami of information. We can’t simply broadcast them with information, nor can we expect them to consult a resource every time they are going to make a purchase.

What would make this data far more useful would be to structure it so that others could incorporate it into software and applications that could shape people’s behaviors and/or deliver the information in the right context.

Take this warning, for example: “CERTAIN FOOD HOUSE BRAND TAHINI OF SESAME MAY CONTAIN SALMONELLA BACTERIA” posted on Monday by the Canadian Food Inspection Agency. There is a ton of useful information in this press release including things like:

The geography impacted: Quebec

The product name, size and better still the UPC and LOT codes.

Product	Size	UPC	Lot codes
Tahini of Sesame	400gr	6 210431 486128	Pro : 02/11/2010 and Exp : 01/11/2012
Tahini of Sesame	1000gr	6 210431 486302	Pro: 02/11/2010 and Exp: 01/11/2012
Premium Halawa	400gr	6 210431 466120	Pro: 02/11/2010 and Exp: 01/11/2012
Premium Halawa	1000gr	6 210431 466304	Pro: 02/11/2010 and Exp: 01/11/2012

However, all this information is buried in the text so is hard to parse and reuse.

If the data was structured and easily machine-readable (maybe available as an API, but even as a structured spreadsheet) here’s what I could imagine happening:

Retailers could connect the bar code scanners they use on their shop floors to this data stream. If any cashier swipes this product at a check out counter they would be immediately notified and would prevent the product from being purchased. This we could do today and would be, in my mind, of high value – reducing the time and costs it takes to notify retailers as well as potentially saving lives.
Mobile applications like RedLaser, which people use to scan bar codes and compare product prices could use this data to notify the user that the product they are looking at has been recalled. Apps like RedLaser still have a small user base, but they are growing. Probably not a game changer, but at least context sensitive.
I could install a widget in my browser that, every time I’m on a website that displays that UPC and/or Lot code would notify me that I should not buy that product and that it’s been recalled. Here the potential is significant, especially as people buy more and more goods over the web.
As we move towards having “smart” refrigerators that scan the RFID chips on products to determine what is in the fridge, they could simply notify me via a text message that I need to throw out my jar of Tahini of Sesame. This is a next generation use, but the government would be pushing private sector innovation in the space by providing the necessary and useful data. Every retailer is going to want to sell a “smart” fridge that doubles as a “safe” fridge, telling you when you’ve got a recalled item in it.

These are all far more citizen-centric, since they don’t require citizens to think, act or pay attention. In short, they aren’t broadcast-oriented, they feel customized, filtering information and delivering it where citizens need it, when they need it, sometimes without them even needing to know. (This is the same argument I made in my How Yelp Could Help Save Millions in Healthcare Costs). The most exciting thing about this is that Health Canada already has all the data to do this, it’s just a question of restructuring it so it is of greater use to various consumers of the data – from retailers, to app developers, to appliance manufactuers. This should not cost that much. (Health Canada, I know a guy…)

Another advantage of this approach is that it also gets the Government out of the business of trying to find ways to determine the best and most helpful way to share information. This appears to be a problem the UK government is also interested in solving. Richard A. sent me this excellent link in which a UK government agency appeals to the country’s developers to help imagine how it can better share information not unlike that being broadcast by Health Canada.

However, at the end of the day even this British example falls into the same problem – believing that the information is most helpfully shared through an app. The real benefit of this type of information (and open data in general) won’t be when you can create a single application with it, but when you can embed the information into systems and processes so that it can notify the right person at the right time.

That’s the challenge: abandoning a broadcast mentality and making things available for multiple contexts and easily embeddable. It’s a big culture shift, but for any government interested in truly exploring citizen-centric approach, it’s the key to success.

The State of Open Data in Canada: The Year of the License

13 Replies

Open Data now an established fact in a growing list of Canadian cities. Vancouver, Toronto, Edmonton, Ottawa have established portals, Montreal, Calgary, Hamilton and some other cities are looking into launching their own and a few provinces are rumored to be exploring open data portals as well.

This is great news and a significant accomplishment. While at the national level Canadian is falling further behind leaders such as England, the United States, Australia and New Zealand, at the local and potentially provincial/state level, Canada could position itself as an international leader.

There is however, one main obstacle: our licenses.

The current challenge:

So far most Open Data portals adopt what has been termed the Vancouver License (it was created by Vancouver for its open data portal and has subsequently been adopted, with occasional minor changes, by virtually every other jurisdiction).

The Vancouver license, however, suffers from a number of significant defects. As someone who was involved in its creation these “bugs” were a necessary tradeoff. If we were looking for a perfect license that satisfied all stakeholders, I suspect we’d still be arguing about it and there’d be no open data or data portal with the Vancouver license. Today, thanks in part to the existence of these portals our politicians, policy makers and government lawyers understanding of this issue has expanded. This fact, in combination with a growing number of complaints about the licenses from non-profits and businesses interested in using open data, has fostered growing interest in adjusting it.

This is encouraging. And we must capitalize on the moment. I wish to be clear: until Canadian governments get the licensing issue right, Open Data cannot advance in this country. Open Data released by governments will not enjoy significant reuse undermining one of the main reasons for doing Open Data.

There are a few things everyone agrees a new license needs to cover. It must establish there is no warranty to the data and that the government cannot be held liable for any reuse. So let’s focus on the parts that governments most often get wrong.

Here, there are 3 things a new license needs to get right.

1. No Attribution

Nascar Jeff Gordon #24 by Dan Raustadt licensed CC-NC-ND

We need a license that does not require attribution. First, attribution gets messy fast – all those cities logos crammed in on a map, on a mobile phone? It’s fine when you are using data from one or two cities, but what happens when you start using data from 10 different governments, or 50? Pretty soon you’ll have NASCAR apps, that will look ugly and be unusable.

More importantly, the goal of open data isn’t to create free advertising for governments, its to support innovation and reuse. These are different goals and I think we agree on which one is more important.

Finally, what government is actually going to police this part of the license? Don’t demand what you aren’t going to enforce – and no government should waste precious resources by paying someone to scour the internet to find websites and apps that don’t attribute.

2. No Share alike

One area the Vancouver license falls down is on the share is in this clause:

If you distribute or provide access to these datasets to any other person, whether in original or modified form, you agree to include a copy of, or this Uniform Resource Locator (URL) for, these Terms of Use and to ensure they agree to and are bound by them but without introducing any further restrictions of any kind.

The last phrase is particularly problematic as it makes the Vancouver license “viral.” Any new data created through a mash up that involves data with the Vancouver license must also use the Vancouver license. This will pretty much eliminate any private sector use of the data since any new data set a company creates they will want to be able to license in manner that is appropriate to their business model. It also has a chilling effect on those who would like to use the data but would need to keep the resulting work private, or restricted to a limited group of people. Richard Weait has an unfortunately named blog post that provides an excellent example of this problem.

Any new license should not be viral so as to encourage a variety or reuses of any data.

3. Standardized

The whole point of Open Data is to encourage the reuse of a public asset. So anything a government does that impedes this reuse will hamper innovation and undermine the very purpose of the initiative. Indeed, the open data movement has, in large part, come to life because one traditional impediment to using data has disappeared: data can now usually be downloaded and available in open formats that anyone can use. The barriers to use have declined so more and more people are interested.

But the other barrier to re-use is legal. If licenses are not easily understood then individuals and businesses will not reuse data, even when it is easily downloadable from a government’s website. Building a businesses or a new non-profit activity on a public asset to which your rights are unclear is simply not viable for many organizations. This is why you want every government should want its license to be easily understood – lowering the barriers to access means making data downloadable and reducing the legal barriers.

Most importantly, it is also why it is ideal if there is a single license in the whole country, as this would significantly reduce transaction and legal costs for all players. This is why I’ve been championing Canada’s leading cities to adopt a single common license.

So, there are two ways of doing this.

The easiest is for Canadian governments to align themselves with several of the international standardized open data licenses that already exist. There are a variety out there. My preference is the Open Commons’ Public Domain Dedication and License (PDDL), although they also publish the Open Database License (ODC-ODbL) and the Attribution License (ODC-By). There is also Creative Commons CC-0 license which Creative Commons suggests to use for open data (I actually recommend against all of these except the PDDL for governments, but more on that later).

These licenses has several advantages.

First, standardized licenses are generally well understood. This means people don’t have to educate themselves on the specifics of dozens of different licenses.

Second, they are stable. Because these licenses are managed by independent authorities and many people use them, they evolve cautiously, and balance the interest of consumers and sharers of data or information.

Third, these licenses balance interests responsible. The creators of these licenses are thought through all the issues that pertain to open data and so give both consumers of data and distributors of data comfort in knowing that they have a licenses that will work.

A second option is for governments in Canada to align around a self-generated common license. Indeed, this is one area where the Federal Government could show (some presently lacking) leadership.(although GeoGratis does have a very good license). This, for example appears to be happening in the UK, where the national government has created an Open Government Licence.

My hope is that, before the year is out, jurisdictions in Canada began to move towards a common licenses, or begin adopting some standard licenses.

Specifically, it would be great to see various Canadian jurisdictions either:

a) Adopt the PDDL (like the City of Surrey, BC). There are some reference to European Data Rights in the PDDL but these have no meaning in Canada and should not be an obstacle – and may even reassure foreign consumers of Canadian data. The PDDL is the most open and forward looking license.

b) Adopt the UK government’s Open Government Licence. This license is the best created by any government to date (with the exemption of simple making the data public domain, which, of course, is far more ideal.

c) Use a modified version of the Geogratis license that adjusts the “3.0 PROTECTION AND ACKNOWLEDGEMENT OF SOURCE” clause to prevent the NASCAR effect from taking place.

What I hope does not happen is that:

a) More and more jurisdictions continue to use the Vancouver License. There are better options and it is an opportunity to launch an open data policy and leapfrog the current leaders in the space.

b) Jurisdictions adopt a Creative Commons license. Creative Commons was created to help license copyrighted material. Since data cannot be copyrighted, the use of creative commons risks confusing the public about the inherent rights they have to data. This is, in part, a philosophical argument, but it matters, especially for governments. We – and our governments especially – cannot allow people to begin to believe that data can be copyrighted.

c) There is no change to the current licenses being used, or a new license, like Open Database License (ODC-ODbL) which goes against the attributes described above, is adopted.

Let’s hope we make progress on this front in 2011.

Open Knowledge Foundation Open Data Advocate

5 Replies

My colleagues over at the Open Knowledge Foundation have been thinking about recruiting an Open Data Advocate, someone who can coordinate a number of the activities they are up to in the open data space. I offered to think about what the role should entail and how that person could be effective. Consequently, in the interests of transparency, fleshing out my thinking and seeing if there might be feed back (feel free to comment openly, or email me personally if you wish to keep it private) I’m laying out my thinking below.

Context

These are exciting times for open government data advocates. Over the past few years a number of countries, cities and international organizations have launched open data portals and implemented open data policies. Many, many more are contemplating joining the fray. What makes this exciting is that some established players (e.g. United States, UK, World Bank) are continue to push forward and will, I suspect, be refining and augmenting their services in the coming months. At the same time there are still a number of laggards (e.g. Canada federally, Southern Europe, Asia) in which mobilizing local communities, engaging with public servants and providing policy support is still the order of the day.

This makes the role of an Open Data Advocate complex. Obviously, helping pull the laggards along is an important task. Alternatively (or in addition) they may need to also be thinking longer term. Where is open data going, what will second and third generation open data portals need to look like (and what policy infrastructure will be needed to support them).

These are two different goals and so either choosing, or balancing, between them will not be easy.

Key Challenges

Some of the key challenges spring quite obviously from that context. But there are also other challenges, I believe to be looming as well. So what do I suspect are the key challenges around open data over the next 1-5 years?

Getting the laggards up and running
Getting governments to use standardized licenses that are truly open (be it the PDDL, CC-0 or one of the other available licenses out there
Cultivating/fostering an eco-system of external data users
Cultivating/fostering an eco-system of internal government user (and vendors) for open data (this is what will really make open data sustainable)
Pushing jurisdictions and vendors towards adopting standard structures for similar types of data (e.g. wouldn’t it be nice if restaurant inspection data from different jurisdictions were structured similarly?)
Raising awareness about abuses of, and the politicization of, data. (e.g. this story about crime data out of New York which has not received nearly enough press)

The Tasks/Leverage Points

There are some basic things that the role will require including:

Overseeing the Working Group on Open Government Data
Managing opengovernmentdata.org
Helping organize the Open Government Data Camp 2011, 2012 and beyond

But what the role will really have to do is figure out the key leverage points that can begin to shift the key challenges listed above in the right direction. The above mentioned tasks may be helpful in doing that… but they may not be. Success is going to be determined but figuring how to shift systems (government, vendor, non-profit, etc…) to advance the cause of open data. This will be no small task.

My sense is that some of these leverage points might include:

Organizing open data hackathons – ideally ones that begin to involve key vendors (both to encourage API development, but also to get them using open data)
Leveraging assets like Civic Commons to get open data policies up on online so that jurisdictions entertaining the issue can copy them
Building open data communities in key countries around the world – particularly in key countries in such as Brasil and India where a combination of solid democratic institutions and a sizable developer community could help trigger changes that will have ramifications beyond their borders (I suspect there are also some key smaller countries – need to think more on that)
I’m sure this list could be enhanced…

Metrics/Deliverables

Obviously resolving the above defined challenges in 1-5 years is probably not realistic. Indeed, resolving many of those issues is probably impossible – it will be a case of ensuring each time we peel back one layer of the onion we are well positioned to tackle the next layer.

Given this, some key metrics by which the Open Knowledge Foundation should evaluate the person in this role might be:

At a high level, possible some metrics might include:

Number of open data portals world wide? (number using CKAN?)
Number of groups, individuals, cities participating in Opendata hackathons
Number of applications/uses of open data
Awareness of CKAN and its mission in the public, developer space, government officials, media?
Number of government vendors offering open data as part of their solution

More additional deliverables, could include:

Running two Global OpenData Hackathons a year?
Developing an OKFN consulting arm specializing in open data services/implementation
Create an open data implementation policy “in a box” support materials for implementing an open data strategy in government
Develop a global network of OKFN chapters to push their local and national governments, share best practices
Run opendata bootcamps for public servants and/or activists
Create a local open data hackathon in a box kit (to enable local events)
Create a local “how to be an open data activist” site
Conduct some research on the benefits of open data to advance the policy debate
Create a stronger feedback loop on CKAN’s benefits and weaknesses
Create a vehicle to connect VC’s and/or money with open data drive companies and app developers (or at least assess what barriers remain to use open data in business processes).

Okay, I’ll stop there, but if you have thoughts please send them or comment below. Hope this stimulates some thinking among fellow open data geeks.

How Yelp Could Help Save Millions in Health Care Costs

21 Replies

Okay, before I dive in, a few things.

1) Sorry for the lack of posts last week. Life’s been hectic. Between Code for America, a number of projects and a few articles I’m trying to get through, the blogging slipped. Sorry.

2) I’m presenting on Open Data and Open Government to the Canadian Parliament Access to Information, Privacy and Ethics Committee today – more on that later this week

3) I’m excited about this post

When it comes to opening up government data many of us focus on Governments: we cajole, we pressure, we try to persuade them to open up their data. It’s approach we will continue to have to take for a great deal of the data our tax dollars pay to collect and that government’s continue to not share. There is however another model.

Consider transit data. This data is sought after, intensely useful, and probably the category of data most experimented with by developers. Why is this? Because it has been standardized. Why has it been standardized. Because local government’s (responding to citizen demand) have been desperate to get their transit data integrated with Google Maps (See image).

It turns out, to get your transit data into Google Maps, Google insists that you submit to them the transit data in a single structured format. Something that has come to be known as the General Transit Feed Specification (GTFS). The great thing about the GTFS is that it isn’t just google that can use it. Anyone can play with data converted into the GTFS. Better still, because the data structure us standardized an application someone develops, or analysis they conduct, can be ported to other cities that share their transit data in a GTFS format (like, say, my home town of Vancouver).

In short, what we have here is a powerful model both for creating open data and standardizing this data across thousands of jurisdictions.

So what does this have to do with Yelp! and Health Care Costs?

For those not in the know Yelp! is a mobile phone location based rating service. I’m particularly a fan of its restaurant locator: it will show you what is nearby and how it has been rated by other users. Handy stuff.

But think bigger.

Most cities in North America inspect restaurants for health violations. This is important stuff. Restaurants with more violations are more likely to transmit diseases and food born illnesses, give people food poisoning and god knows what else. Sadly, in most cases the results of these tests are posted in the most useless place imaginable. The local authorities website.

I’m willing to wager almost anything that the only time anyone visits a food inspection website is after they have been food poisoned. Why? Because they want to know if the jerks have already been cited.

No one checks these agencies websites before choosing a restaurant. Consequently, one of the biggest benefits of the inspection data – shifting market demand to more sanitary options – is lost. And of course, there is real evidence that shows restaurants will improve their sanitation, and people will discriminate against restaurants that get poor ratings from inspectors, when the data is conveniently available. Indeed, in the book Full Disclosure: The Perils and Promise of Transparency Fung, Graham and Weil noted that after Los Angeles required restaurants to post food inspection results, that “Researchers found significant effects in the form of revenue increases for restaurants with high grades and revenue decreases for C-graded (poorly rated) restaurants.” More importantly, the study Fung, Graham and Weil reference also suggested that making the rating system public positively impacted healthcare costs. Again, after inspection results in Los Angeles were posted on restaurant doors (not on some never visited website), the county experienced a reduction in emergency room visits, the most expensive point of contact in the system. As the study notes these were:

an 18.6 percent decline in 1998 (the first year of program operation), a 4.8 percent decline in 1999, and a 5.4 per- cent decline in 2000. This pattern was not observed in the rest of the state.

This is a stunning result.

So, now imagine that rather than just giving contributor generated reviews of restaurants Yelp! actually shared real food inspection data! Think of the impact this would have on the restaurant industry. Suddenly, everyone with a mobile phone and Yelp! (it’s free) could make an informed decision not just about the quality of a restaurant’s food, but also based on its sanitation. Think of the millions (100s of millions?) that could be saved in the United States alone.

All that needs to happen is for a simple first step, Yelp! needs approach one major city – say a New York, or a San Francisco – and work with them to develop a sensible way to share food inspection data. This is what happened with Google Maps and the GTSF, it all started with one city. Once Yelp! develops the feed, call it something generic, like the General Restaurant Inspection Data Feed (GRIDF) and tell the world you are looking for other cities to share the data in that format. If they do, you promise to include it in your platform. I’m willing to bet anything that once one major city has it, other cities will start to clamber to get their food inspection data shared in the GRIDF format. What makes it better still is that it wouldn’t just be Yelp! that could use the data. Any restaurant review website or phone app could use the data – be it Urban Spoon or the New York Times.

The opportunity here is huge. It’s also a win for everyone: Consumers, Health Insurers, Hospitals, Yelp!, Restaurant Inspection Agencies, even responsible Restaurant Owners. It would also be a huge win for Government as platform and open data. Hey Yelp. Call me if you are interested.

What Canada’s Realtors could learn from Canada’s Lawyers

5 Replies

Lawyers aren’t generally known to be the most technologically forwarding looking group – but here in Canada they have done one thing really, really well. Making radically efficient the transaction costs around sharing critical information regarding their industry.

CanLII – the non-profit managed by the Federation of Law Societies of Canada has the goal “to make Canadian law accessible for free on the Internet.” In essence CanLII copies all of the materials produced by the courts, organizes it and makes it searchable and re-usable by anyone. For realtors wondering about their future, looking over this service might be a good place to start.

Consider MLS.ca (now rebranded as realtor.ca) the website run by the Canadian Real Estate Association (CREA) that shares information on what homes are for sale where. A few of you may also know that the Competition Bureau and CREA have recently been tangling over access to MLS. While the it is now easier for people to list properties on MLS, the data within MLS is very restricted. Much of the data only realtors can see and re-use of the data appears strictly verboten. These restrictions cause Canadians to suffer from what I like to call the Hulu Syndrome – they can see what a more open system would look like by surfing the various property websites in the United States – but they are stuck using MLS when trying to browse for a home to buy.

Canadian realtors wanting to know what the future looks like for a professional service in a world where data and information is widely available, CanLII offers both a window and a model. Unlike MLS, the great thing about CanLII is that it serves everyone, not just lawyers. It isn’t hard to imagine a world where lawyers insisted that only they can access the cataloging system they pay for (lawyers pay a small annual fee to support CanLII) much like only realtors can access the full database of MLS. In such world if you wanted to read a judgement, or view court documents on a specific case, only a lawyer could access it for you, and then they would interpret it for you, and, to carry the analogy to its logically conclusion, you would rarely or likely never see the original documents.

Thankfully for both the legal system, the market place for legal services and for our democracy, CanLII doesn’t work this way. As mentioned anyone can search, find and download all the information. Indeed, look at CanLII’s Terms of Use:

Subject to the following paragraph and the below conditions pertaining to prohibited use, legal materials published on the CanLII website, such as legislation, regulations and decisions, including editorial enhancements inserted into the documents by CanLII, such as hyperlinks and information in headers and footers, can be copied, printed and distributed by Users free of charge and without any other authorization from CanLII, provided that CanLII is identified as the source of the document.

Compare this to MLS’s terms of use:

This database and all materials on this site are protected by copyright laws and are owned by The Canadian Real Estate Association (CREA) or by the member who has supplied the data. Property listings and other data available on this site are intended for the private, non-commercial use by individuals. Any commercial use of the listings or data in whole or in part, directly or indirectly, is specifically forbidden except with the prior written authority of the owner of the copyright.

(Side note, I’m pretty sure you can’t copyright data – so not sure what the legal rights being exercised here are).

Of course, even though CanLI makes legal documents are freely available, many people still want to use lawyers because they don’t have time or, just as often, realize they need expert advice in this complicated field.

The same would be true of MLS. Many, many buyers will still want to use a realtor, although the buyers and sellers in the market place would be smarter and more informed – but this would probably lead to a better marketplace and happier customers. There are of course, a number of buyers and sellers who will simply freeload off MLS’s data to sell or buy their home on their own (much like some people probably “freeload” off CanLII to represent themselves or do research). But these are probably clients who would prefer to be doing it this way anyway – giving them full access to the database may cause them to a) realize they do need professional help or b) remove customers who don’t really want to use a realtor in the first place and are thus… terrible customers.

This isn’t to say that sharing MLS data won’t be disruptive, I suspect that some people will automate the buying/selling process which a percentage of the market place will prefer to a handheld process – but I suspect that, at some point, this will happen anyway (someone will figure out a model to make it work) at which point CREA and the realtors will have been firmly entrenched in the minds of Canadians as the obstacle to a better, more efficient marketplace, not the leaders who helped foster it.

Lawyers aren’t often known for clarity and simplicity, but clearly when they get it right, they get it right. I hope other professional services will look at what they are up to.

The Next International Open Data Hack Day – initial thoughts

4 Replies

Yesterday I got to meet up with Edward Ocampo-Gooding and Mary Beth Baker in Ottawa and we started talking about what the next international open data hackathon: when might be a good time to do it, what might it look like, etc…

One idea is to set a theme that might help inspire people and serve as something to weave the events together in stronger way. Edward proposed the theme of Mom’s and, since, in many, many, many countries, Mother’s day is in May, it seemed like a nice suggestion.

It also has two nice benefits:

it gets us away from an exclusive focus on government and might get people in the headspace of creating applications with tangible uses – something almost everyone can relate to
many people have mom’s! so getting into the shoes of a mom and imagining what might be interesting, engaging and/or helpful shouldn’t be impossible
it might engage new people in the open data movement and in the local events

In addition, another suggestion that was raised is the idea of focusing on a few projects that have already been speced out in advanced – much like Random Hacks of Kindness does with their hackathons. Think this could be fruitful to explore.

Finally, regarding timelines, I’m thinking May. It works thematically (if that theme gets used). More importantly, however, it’s far enough out to plan, near enough to be tangible and sets a nice pace of two global hackathons a year which feels sufficiently ambitious for a group of volunteers, doesn’t crowd out/compete with other hackathons or local events, and seems like a good check in timeline for volunteer driven projects… Also, it might give people a chance to use scrapperwiki in the interim to get data together for projects they want to work on.

Thoughts on all this? Please blog, post a comment below, or if you are feeling shy, drop me a note (david at eaves.ca or @daeaves with hastag #odhd on twitter). I’ve also created a page on the Open Data Day wiki to discuss this if people are more comfortable with that.

What I’m doing at Code for America

1 Reply

For the last two weeks – and for much of January – I’m in San Francisco helping out with Code for America. What’s Code for America? Think Teach for America, but rather than deploying people into classrooms to help provide positive experiences for students and teachers while attempting to shift the culture of school districts, Code for America has fellows work with cities to help develop reusable code to save cities money, make local government as accessible as your favorite website, and help shift the government’s culture around technology.

The whole affair is powered by a group of 20 amazing fellows and an equally awesome staff that has been working for months to make it all come together. My role – in comparison – is relatively minor, I head up the Code for America Institute – a month long educational program the fellows go through when they first arrive. I wanted to write about what I’ve been trying to do both because of the openness ideals of Code for America and to share any lessons for others who might attempt a similar effort.

First, to understand what I’m doing, you have to understand the goal. On the surface, to an outsider, the Code for America change process might look something like this:

Get together some crazy talented computer programers (hackers, if you want to make the government folks nervous)
Unleash them on a partner city with a specific need
Take resulting output and share across cities

Which of course, would mistakenly frame the problem as technical. However, Code for America is not about technology. It’s about culture change. The goal is about rethinking and reimagining government as better, faster, cheaper and adaptive. It’s about helping think of the ways its culture can embrace government as a platform, as open and as highly responsive.

I’m helping (I think) because I’ve enjoyed some success in getting government’s to think differently. I’m not a computer developer and at their core, these successes were never technology problems. The challenge is understanding how the system works, identify the leverage points for making change, develop partners and collaborate to engage those leverage points, and do whatever it takes to ensure it all comes together.

So this is the message and the concept the speakers are trying to impart on the fellows. Or, in other words, my job is to help unleash the already vibrant change agents within the 20 awesome fellows and make them effective in the government context.

So what have we done so far?

We’ve focused on three areas:

1) Understand Government: Some of the fellows are new to government, so we’ve had presentations from local government experts like Jay Nath, Ed Reiskin and Peter Koht as well as the Mayor of Tuscon’s chief of staff (to give a political perspective). And of course, Tim O’Reilly has spoken about how he thinks government must evolve in the 21st century. The goal: understand the system as well as, understand and respect the actors within that system.

2) Initiate & Influence: Whether it is launching you own business (Eric Ries on startups), starting a project (Luke Closs on Vantrash) or understanding what happens when two cultures come together (Caterina Fake on Yahoo buying Flickr) or myself on negotiating, influence and collaboration, our main challenges will not be technical, they will be systems based and social. If we are to build projects and systems that are successful and sustainable we need to ask the right questions and engage with these systems respectfully as we try to shift them.

3) Plan & Focus: Finally, we’ve had experts in planning and organizing. People like Allen Gunn (Gunner) and the folks from Cooper Design, who’ve helped the fellows think about what they want, where they are going, and what they want to achieve. Know thyself, be prepared, have a plan.

The last two weeks will continue to pick up these themes but also give the fellows more time to (a) prepare for the work they will be doing with their partner cities; and (b) give them more opportunities to learn from one another. We’re half way through the institute at this point and I’m hoping the experience has been a rich – if sometimes overwhelming – one. Hopefully I’ll have an update again at the end of the month.

Honourable Mention! The Mozilla Visualization Challenge Update

2 Replies

Really pleased to share that Diederik and I earned an honourable mention for our submission to the Mozilla Open Data Competition.

For those who missed it – and who find opendata, open source and visualization interesting – you can read a description of and see images from our submission to the competition in this blog post I wrote a month ago.

Canada's Secret Open Data Strategy?

10 Replies

Be prepared for the most boring sentence to an intriguing blog post.

The other night, I was, as one is wont to do, reading through a random Organization for Economic Coordination and Development report entitled Towards Recovery and Partnership with Citizens: The Call for Innovative and Open Government. The report was, in fact, a summary of its recent Ministerial Meeting of the OECD’s Public Governance Committee.

Naturally, I flipped to the section authored by Canada and, imagine the interest with which I read the following:

The Government of Canada currently makes a significant amount of open data available through various departmental websites. Fall 2010 will see the launch of a new portal to provide one-stop access to federal data sets by providing a “single-window” to government data. In addition to providing a common “front door” to government data, a searchable catalogue of available data, and one-touch data downloading, it will also encourage users to develop applications that re-use and combine government data to make it useful in new and unanticipated ways, creating new value for Canadians. Canada is also exploring the development of open data policies to regularise the publication of open data across government. The Government of Canada is also working on a strategy, with engagement and input from across the public service, developing short and longer-term strategies to fully incorporate Web 2.0 across the government.

In addition, Canada’s proactive disclosure initiatives represent an ongoing contribution to open and transparent government. These initiatives include the posting of travel and hospitality expenses, government contracts, and grants and contribution funding exceeding pre-set thresholds. Subsequent phases will involve the alignment of proactive disclosure activities with those of the Access to Information Act, which gives citizens the right to access information in federal government records.

Lots of interesting things packed into these two paragraphs, something I’m sure readers concerned with open data, open government and proactive, would agree with. So let’s look at the good, the bad and the ugly, of all of this, in that order.

The Good

So naturally the first sentence is debatable. I don’t think Canada makes a significant amount of its data available at all. Indeed, across every government website there is probably no more than 400 data sets available in machine readable format. That’s less that the city of Washington DC. It’s about (less than) 1% of what Britain or the United States disclose. But, okay,let’s put that unfortunate fact aside.

The good and really interesting thing here is that the Government is stating that it was going to launch an open data portal. This means the government is thinking seriously about open data. This means – in all likelihood – policies are being written, people are being consulted (internally), processes are being thought through. This is good news.

It is equally good news that the government is developing a strategy for deploying Web 2.0 technologies across the government. I hope this will be happening quickly as I’m hearing that in many departments this is still not embraced and, quite often, is banned outright. Of course, using social media tools to talk with the public is actually the wrong focus (Since the communications groups will own it all and likely not get it right for quite a while), the real hope is being allowed to use the tools internally.

The Bad

On the open data front, the bad is that the portal has not launched. We are now definitely passed the fall of 2010 and, as for whatever reason, there is no Canadian federal open data portal. This may mean that the policy (despite being announced publicly in the above document) is in peril or that it is simply delayed. Innumerable things can delay a project like this (especially on the open data front). Hopefully whatever the problem is, it can be overcome. More importantly, let us hope the government does something sensible around licensing and uses the PDDL and not some other license.

The Ugly

Possibly the heart stopping moment in this brief comes in the last paragraph where the government talks about posting travel and hospitality expenses. While these are often posted (such as here) they are almost never published in machine readable format and so have to be scrapped in order to be organized, mashed up or compared to other departments. Worse still, these files are scattered across literally hundreds of government websites and so are virtually impossible to track down. This guy has done just that, but of course now he has the data, it is more easily navigable but no more open then before. In addition, it takes him weeks (if not months) to do it, something the government could fix rather simply.

The government should be lauded for trying to make this information public. But if this is their notion of proactive disclosure and open data, then we are in for a bumpy, ugly ride.

Canada ranks last in freedom of information

3 Replies

For those who missed it over the weekend it turns out Canada ranks last in freedom of information study that looked at the world’s western Parliamentary democracies. What makes it all the more astounding is that a decade ago Canada was considered a leader.

Consider two from the Information Commissioner, Suzanne Legault quotes pulled from the piece:

Only about 16 per cent of the 35,000 requests filed last year resulted in the full disclosure of information, compared with 40 per cent a decade ago, she noted.

And delays in the release of records continue to grow, with just 56 per cent of requests completed in the legislated 30-day period last year, compared with almost 70 per cent at the start of the decade.

These are appalling numbers.

The sad thing is… don’t expect things to get better. Why?

Firstly, the current government seems completely uninterested in access to information, transparency and proactive disclosure, despite these being core planks of its election platform and core values of the reform movement that re-launched Canadian conservatism. Indeed, reforming and improving access to information is the only unfulfilled original campaign promise of the Conservatives – and there appears to be no interest in touching it. Quite the opposite – that political staff now intervene to block and restrict Access to Information Requests – contravening the legislation and policy – is now a well known and documented fact.

Second, this issue is of secondary importance to the public. While everyone will say they care about access to information and open government, then number of people (while growing) still remains small. These types of reports and issues are of secondary importance. This isn’t to say they don’t matter. They do – but generally after something bigger and nastier has come to light and the public begins to smell rot. Then studies like this become the type of thing that hurts a government – it gives legitimacy and language to a sentiment people widely feel.

Third, the public seems confused about who they distrust more – the fact is, however bad the current government is on this issue, the Liberal brand is still badly tarnished on this issue of transparent government due to the scandals from almost a decade ago. Sadly, this means that there will be less burden on this government to act since – every time the issue of transparency and open government arise – rather than act, Government leaders simply point out the other parties failings.

So as the world moves on while Canada remains stuck, its government becoming more opaque, distant and less accountable to the people that elect it.

Interestingly , this also has a real cost to Canada’s influence in the world. It means something when the world turns to you as an expert – as we once were on access to information – minister’s are consulted by other world leaders, your public servants are given access to information loops they might otherwise not know about, there is a general respect, a soft power, that comes from being an acknowledged leader. Today, this is gone.

Indeed, it is worth noting that of the countries survey in the above mentioned study, only Canada and Ireland do not have open data portals which allow for proactive disclosure.

It’s a sign of the times.