Inferring Serial Killers with Data: A Lesson from Vancouver

3 Replies

For those happily not in the know, my home town of Vancouver was afflicted with a serial killer during the 80’s and 90’s who largely targeted marginalized women in the downtown eastside – the city’s (and one of the country’s) poorest neighborhoods.

The murderer – Robert Pickton – was ultimately caught in February 2002 and, in December 2007, was convicted on 6 accounts of second degree murder. He is accused of murdering an additional twenty women, and may be responsible for deaths of a number more.

Presently there is an inquiry going on in Vancouver regarding the failure of the policy to investigate and act earlier on the disappearing women. Up until now, the most dramatic part of the inquiry for me had been heart wrenching testimony from one female officer whose own efforts within the Police Department went largely ignored. But I’ve recently seen a new spat of articles that are more interesting and disturbing.

It turns out that during the late 1990s the Vancouver Policy Department actually had an expert analyzing crime data – particularly regarding the disappearing women – and his assessment was that a serial murder was at work in the city. The expert, Kim Rossmo, advised the police to issue a press release and begin to treat the case more seriously.

He was ignored.

The story is relatively short, but worth the read – it can be found here.

What’s particularly discouraging is looking back at past articles, such as this Canadian Press piece which was published in June 26th, 2001, less than a year before Pickton was caught:

Earlier that day, Hughes stood with six others outside a Vancouver courthouse and told passers-by she believes a serial killer is responsible.

Vancouver police officially reject the suggestion.

But former police officer Kim Rossmo supported it while he was a senior officer. He wanted to warn residents about the possible threat. Rossmo is now involved in a wrongful dismissal trial against the force in B.C. Supreme Court.

Last week, he testified he wanted to issue a public warning in 1998, but other officers strongly objected. The force issued a news release saying police did not believe a serial killer was behind the disappearances.

Indeed, Rossom was not just ignored, other policemen on the force actively made his life difficult. He was harassed and further data that would have helped him engage in his analysis was withheld from him. Of course a few months later the murder was caught, demonstrating that his capture might have happened much earlier, if the force had taken the potential problem seriously.

A few lessons from this:

1) Data matters. In this case, the use of data could have, literally, saved lives. Rossom’s data model is now used by other forces and has become a professor in the United States.

2) The challenge with data is as often cultural as it is technical. As with the Moneyball story, the early advocates of using data to analyze and reassess a problem are often victimized. Their approach threatens entrenched interests and, the work often is conducted by people on the margins. Rossom was the first PhD in Canada to become a police officer – I’m pretty sure that didn’t make him a popular guy. Moreover, his approach implicitly, and then explicitly suggested the police were wrong. Police forces don’t deal with errors well – but nor do many organizations or bureaucracies.

3) Finally, this case study says volumes about police forces capacity to deal with data. Indeed, some of you may remember that the other week I deconstructed the Vancouver Police Department’s misleading press release regarding its support for Bill-C30 which would dramatically increase the police’s power to monitor Canadians online. I find it ironic that the police are seeking access to more data, when they have been unable to effectively use data that they can already legal acquire (or that, frankly is open, such as the number and locations of murder/disappearance victims).

Media Watch: The Globe and Mail’s Shifting Headline

2 Replies

Earlier today the Globe and Mail had one of these truly terrible “balanced” articles about the proposed federal crime bill. The headline screamed: Quebec expert backs Tory crime bill amid U.S. warning on sentencing. (Image below)

So who was this expert you might ask? A university professor with years of research on the subject? Maybe some breakthrough research by a young grad student? How about a researcher from a think tank that has been investigated the issue?

Wrong on all accounts. It was, in fact, former Justice Minister Marc Bellemare from the province of Quebec. Of course, you might say… “being a Justice Minister problem should make you an expert.” If only this were the case. If Minister Vic Toews has taught us anything it is that you definitely don’t have to be an expert in something to become a Minister. Nor does being a Minister make you an expert. But the real kicker is that Marc Bellemare was minister for just under a year. Sworn in on April 29th 2003 he resigned on April 27th 2004. Of course, the article makes no reference to the current Justice Minister of Quebec, Jean-Marc Fournier, who is both opposed to the Crime bill and has been minister since August 11th, 2010. That’s a year and a half longer making him 50% more of an expert than Bellemare!

I suspect one of two things happened (both of which I now know are wrong – see update below). Either the Globe reporter simple used language that came packaged in a press release that referred to Marc Bellemare as an expert or worse, in pursuit of “balance” the journalist felt compelled to label Bellemare’s an expert given the second part focused on how a large number of US republican “tough on crime” legislators who created mandatory minimum sentences in the 90s are trying to role them back because they have been a total failure in addressing crime and a disaster financially.

Of course Globe and Mail readers noticed the problem with the “expert” right away. The most voted for comment was the following one (yes, I voted too, might have been my first time):

And slightly further done was a better comment pointing out some further idiotic ideas the Minister had for reforming the justice system.

More interesting is that sometime in the later afternoon EST the Globe changed its landing page, acknowledging the “expert’s” true credentials.

I think this speaks volumes about the Globe – in a good way. Nobody is perfect, we make mistakes. Sun prides itself on getting facts wrong to tell a story and the Globe is demonstrating that they take the opposite tact. So this post isn’t to say “the globe messed up,” it’s about how newspaper can and should react to feedback from readers. It doesn’t mean you change everything all the time, but there are times when the feedback points to changes that will bring about greater clarity. It also says a lot about the power of the audience.

However, it is worth noting, the headline on the story page… remains unchanged.

7:14pm Update

I’ve made some errors of my own in the above post. I assumed above that the journalist had chosen the headline, this is, in fact, not true. As one of the editors from the Globe has pointed out to me on twitter, it was the editor who made the choice. Any assignment of blame on the journalist is misplaced, I definitely apologize for that on my part.

More on Google Transit and how it is Reshaping a Public Service

1 Reply

Some of you know I’ve written a fair bit on Google transit and how it is reshaping public transit – this blog post in particular comes to mind. For more reading I encourage you to check out the Xconomy article Google Transit: How (and Why) the Search Giant is Remapping Public Transportation as it provides a lot of good details as to what is going on in this space.

Two things about this article:

First, it really is a story about how the secret sauce for success is combining open data with a common standard across jurisdictions. The fact that the General Transit Feed Specification (a structured way of sharing transit schedules) is used by over 400 transit authorities around the world has helped spur a ton of other innovations.

Couple of money quotes include this one about the initial reluctance of some authorities to share their data for free (I’m looking at you Translink board):

“I have watched transit agencies try to monetize schedules for years and nobody has been successful,” he says. “Markets like the MTA and the D.C. Metro fought sharing this data for a very long time, and it seems to me that there was a lot of fallout from that with their riders. This is not our data to hoard—that’s my bottom line.”

and this one about iBart, an app that uses the GTFS to power an app for planning transit trips:

in its home city, San Francisco, the startup’s app continues to win more users: about 3 percent of all trips taken on BART begin with a query on iBART

3%? That is amazing. Last year my home town of Vancouver’s transit authority, Translink, had 211.3 million trips. If the iBart app were ported to here and enjoyed similar success that would man 6.4 million trips planned on iBart (or iTranslink?). That’s a lot of trips made easier to plan.

The second thing I encourage you to think about…

Where else could this model be recreated? What’s the data set, where is the demand from the public, and what is the company or organization that can fulfill the role of google to give it scale. I’d love to hear thoughts.

The Surveillance State – No Warrant Required

9 Replies

Yesterday a number of police organizations came out in support of bill C-30 – the online surveillance bill proposed by Minister Vic Toews. You can read the Vancouver Police Department’s full press release here – I’m referencing theirs not because it is particularly good or bad, but simply because it is my home town.

For those short on time, the very last statement, at the bottom of the post, is by far the worst and is something every Canadian should know. The authors of these press releases would have been wise to read Michael Geist’s blog posts from yesterday before publishing. Geist’s analysis shows that, at best, the police are misinformed, at worst, they are misleading the public.

So let’s look at some of the details of the press release that are misleading:

Today I speak to you as the Deputy Chief of the VPD’s Investigation Division, but also as a member of the Canadian Association of Chiefs of Police, and I’m pleased to be joined by Tom Stamatakis, President of both the Vancouver Police Union and Canadian Police Association.

The Canadian Association of Chiefs of Police (CACP) is asking Canadians to consider the views of law enforcement as they debate what we refer to as “lawful access,” or Bill C-30 – “An Act to enact the Investigating and Preventing Criminal Electronic Communications Act and to amend the Criminal Code and other Acts.”

This Bill was introduced by government last week and it has generated much controversy. There is no doubt that the Bill is complex and the technology it refers to can be complex as well.
I would, however, like to try to provide some understanding of the Bill from a police perspective. We believe new legislation will:

assist police with the necessary tools to investigate crimes while balancing, if not strengthening, the privacy rights for Canadians through the addition of oversight not currently in place

So first bullet point, first problem. While it is true the bill brings in some new process, to say it strengthens privacy rights is misleading. It has become easier, not harder, to gain access to people’s personal data. Before, when the police requested personal information from internet service providers (ISPs) the ISPs could say no. Now, we don’t even have that. Worse, the bill apparently puts a gag on order on these warrantless demands, so you can’t even find out if a government agency has requested information about you.

help law enforcement investigate and apprehend those who are involved in criminal activity while using new technologies to avoid apprehension due to outdated laws and technology

allow for timely and consistent access to basic information to assist in investigations of criminal activity and other police duties in serving the public (i.e. suicide prevention, notifying next of kin, etc.)

This, sadly, is a misleading statement. As Michael Geist notes in his blog post today “The mandatory disclosure of subscriber information without a warrant has been the hot button issue in Bill C-30, yet it too is subject to unknown regulations. These regulations include the time or deadline for providing the subscriber information (Bill C-30 does not set a time limit)…”

In other words, for the police to say the bill will get timely access to basic information – particularly timely enough to prevent a suicide, which would have to be virtually real time access – is flat out wrong. The bill makes no such promise.

Moreover, this underlying justification is itself fairly ridiculous while the opportunities for abuse are not trivial. It is interesting that none of the examples have anything to do with preventing crime. Suicides are tragic, but do not pose a risk to society. And speedily notifying next of kin is hardly such an urgent issue that it justifies warrantless access to Canadians private information. These examples speak volumes about the strength of their case.

Finally, it is worth noting that while the Police (and the Minister) refer to this as “basic” information, the judiciary disagrees. Earlier this month the Saskatchewan Court of Appeal concluded in R v Trapp, 2011 SKCA 143 that an individual has a reasonable expectation of privacy in the IP address assigned to him or her by an internet service provider, a point which appeared not to have been considered previously by an appellate court in Canada

The global internet, cellular phones and social media have all been widely adopted and enjoyed by Canadians, young and old. Many of us have been affected by computer viruses, spam and increasingly, bank or credit card fraud.

This is just ridiculous and is designed to do nothing more than play on Canadians fears. I mean spam? Really? Google Mail has virtually eliminated spam for its users. No government surveillance infrastructure was required. Moreover, it is very, very hard to see how the surveillance bill will help with any of the problems cited about – viruses, spam or bank fraud.

Okay skipping ahead (again you can read the full press release here)

2. Secondly, the matter of basic subscriber information is particularly sensitive.

The information which companies would be compelled to release would be: name, address, phone number, email address, internet protocol address, and the name of the service provider to police who are in the lawful execution of their duties.

Actually to claim that these are going to police who are in the lawful execution of their duties is also misleading. This data would be made available to police who, at best, believe they are in the lawful execution of their duties. This is precisely why we have warrants so that an independent judiciary can assess whether or not the police are actually engaged in the lawful execution of their duties. Strip away that check and there will be abuses. Indeed, the Sun Newspaper phone hacking scandal in the UK serves as a perfect example of the type of abuse that is possible. In this case police officers were able to access “under extraordinary circumstances” without a warrant or oversight, the names and phone numbers of people whose phones they wanted to, or had already, hacked.

While this information is important to police in all types of investigations, it can be critical in cases where it is urgent that police locate a caller or originator of information that reasonably causes the police to suspect that someone’s safety is at risk.

Without this information the police may not be able to quickly locate and help the person who was in trouble or being victimized.

An example would be a message over the internet indicating someone was contemplating suicide where all we had was an email address.

Again, see above. The bill does not stipulate any timelines around sharing this data. This statement is designed to lead readers to believe readers that the bill will grant necessary and instant access so that a situation could be defused in the moment. The bill does nothing of the sort.

Currently, there is no audited process for law enforcement to gain access to basic subscriber information. In some cases, internet service providers (ISPs) provide the information to police voluntarily — others will not, or often there are lengthy delays. The problem is that there is no consistency in providing this information to police nationally.

This, thankfully is a sensible statement.

3. Lastly, and one of the most important things to remember, this bill does NOT allow the police to monitor emails, phone calls or internet surfing at will without a warrant, as has been implied or explicitly stated.

There is no doubt that those who are against the legislation want you to believe that it does. I have read the Bill and I cannot find that anywhere in it. There are no changes in this area from the current legislation.

This is the worst part of the press release as it is definitely not true. See Michael Geist’s – the Ottawa professor most on top of this story – blog post from yesterday, which was written before this press release went out. According to Geist, there is a provision in the law that “…opens the door to police approaching ISPs and asking them to retain data on specified subscribers or to turn over any subscriber information – including emails or web surfing activities – without a warrant. ISPs can refuse, but this provision is designed to remove any legal concerns the ISP might have in doing so, since it grants full criminal and civil immunity for the disclosures.” In other words the Police can conduct warantless surveillance. It just requires the permission of the ISPs. This flat out contradicts the press release.

Transparency isn't a cost – it's a cost saver (a note for Governments and Drummond)

1 Reply

Yesterday Don Drummond – a leading economist hired by the Ontario government to review how the province delivers services in the face of declining economic growth and rising deficits – published his report.

There is much to commend, it lays out stark truths that frankly, many citizens already know, but that government was too afraid to say aloud. It is a report that, frankly, I think many provincial and state governments may look at with great interest since the challenges faced by Ontario are faced by governments across North America (and Europe).

From an IT perspective – particular one where I believe open innovation could play a powerfully transformative role – I found the report lacking. I say this with enormous trepidation, as I believe Drummond to be a man of supreme intellect, but my sense is he (and/or his team) have profoundly misunderstand government transparency and why it should be relevant. In Chapter 16 (no I have not yet read all 700 pages) a few pieces come together to create, what I believe, are problematic conditions. The first relates to the framing around “accountability”:

Accountability is an essential aspect of government operations, but we often treat that goal as an absolute good. Taxpayers expect excellent public-sector management as well as open and transparent procurement practices. However, an exclusive focus on rigorous financial reporting and compliance as the measure of successful management requires significant investments of time, energy and resources. At some point, this investment is subject to diminishing returns.

Remember the context. This section largely deals with how government services – and in particular the IT aspects of these services – could be consolidated (a process that rarely yields the breadth of savings people believe it will). Through this lens the interesting things about the word “accountability” in this section above is that I could replace it with searchability – the capacity to locate pieces of information. I agree with Drummond that there is a granularity around recording items – say tracking every receipt versus offering per diems – that creates unnecessary costs. Nor to I believe we should pay unlimited costs for transparency – just for the sake of transparency. But I do believe that government needs a much, much stronger capacity to search and locate pieces of information. Indeed, I think that capacity, the ability for government to mine its own data intelligently, will be critical. Transparency thus becomes one of the few metrics citizens have into not only how effective a government’s inputs are, but how effective its systems are.

Case in point. If you required every Canadian under the age of 30 to conduct an ATIP request tomorrow, I predict that you’d have a massive collapse in Canadians confidence in government. The length of ATIP requests (and the fact that in many places, they aren’t even online) probably says less about government secrecy to these Canadians than it does about the government’s capacity to locate, identify and process its own data and information. When you can’t get information to me in a timely manner, it strongly suggests that managers may not be able to get timely information either.

If Ontario’s public service is going to be transformed – especially if it is going to fulfill other Drummond report recommendations, such as:

Further steps should be taken to advance partnering with municipal and federal services —efficiencies can be found by working collaboratively with other levels of government. For example, ServiceOntario in Ottawa co-locates with the City of Ottawa and Service Canada to provide services from one location, therefore improving the client experience. Additionally, the new BizPal account (which allows Ontario businesses to manage multiple government requirements from a single account) allows 127 Ontario municipalities (such as Kingston, Timmins, Brampton and Sudbury) to partner with ServiceOntario and become more efficient in issuing business permits and licensing. The creation of more such hubs, with their critical mass, would make it easier to provide services in both official languages. Such synergies in service delivery will improve customer experience and capitalize on economies of scale.

Then it is going to require systems that can be easily queried as well as interface with other systems quickly. Architecting systems in open standards, that can be easily searched and recoded, will be essential. This is particularly true if the recommendation that private sector partners (who love proprietary data models, standards and systems which regularly trap governments in expensive traps) are to be used more frequently. All this is to say, we shouldn’t to transparency for transparencies sake. We should do transparency because it will make Ontario more interoperable, will lower costs, and will enable more accountability.

Accountability doesn’t have to be a cost drive. Quite the opposite, transparency should and can be the bi-product of good procurement strategies, interoperable architecture choices and effective processes.

Let’s not pit transparency against cost savings. Very often, it’s a false dichotomy.

Data Wars: A mini-case study of Southwest Airlines vs. TripIt and Orbitz

1 Reply

As a regular flyer, I’m an enormous fan of TripIt. It’s a simple service in which you forward almost any reservation – airline, hotel, car rental, etc… to plans@tripit.com and their service will scan it, grab the relevant data, and create a calendar of events for you. While it’s a blessing not to have to manually enter my travel plans into my calendar, what’s particularly fantastic is that I give my partner access to the calendar – so she knows when I’m flying out and when I return. With 135,000 miles of travel last year alone, there was a lot of that.

TripIt Pro users, however, have added benefits: they can use TripIt to track how many loyalty points they are gathering. That is, unless you travel on Southwest Airlines. Apparently Southwest sent a legal warning to any company that tracks their members’ loyalty benefits and ordered them to stop doing so. (Award Wallet is another example of an app I use that was affected). In a similar vein, veteran travelers know that Southwest does not appear on many travel search sites like Orbitz.

These are great examples of a data wars – places where a company are fighting over who gets access to customers data. In this case Southwest is using its user license to forbids another company from displaying data Southwest generates, but that its customers might wish to share with others because it is helpful to them. It’s not just that Southwest wants to control its relationship with its customers when it comes to loyalty points, or that it wants to sell them hotels and rental cars though its site. It’s that it wants the data about how you behave, about what choices you make and how you make them. Use another site to access loyalty points and they can’t track or sell to you. Ditto if you use another site to buy airfare for their flights.

Southwest isn’t nuts. But it’s a strategy that won’t work for all companies (and may not work for them) and it has real consequences.

To begin with, they are making it hard for their customers to engage their service. When traveling in the US, I regularly use Kayak and/or other types of airline aggregators – it means I never see Southwest as an option. Nor do I go to their website. The bigger irony of course is that while I frequently find fares on aggregator sites, I often book them on the airline’s site. But again, I don’t go to Southwest because they never appear in any searches I do. Maybe they don’t care about business travelers, but they are making a big trade-off – they get more data about their users and have unique opportunities to sell to them, but I suspect they get far fewer users.

In addition, they may be alienating their customers. I’m not so sure customers will feel like loyalty point data belongs to Southwest. After all, it was their dollars and flying that paid to create the data… why shouldn’t they be able to access a copy of it via an application they find useful?

This was all confirmed by an email from a friend and colleague Gary R., who recently wrote me to say:

While we love Southwest Airlines for its low prices, generous affinity programs and flexibility in changing business trips at the last moment with little consequence, their closed data sharing policy drives up our overall cost of managing travel. Entering flight information manually into TripIt is a pain, yet the service is incredible at keeping one informed during a trip, presenting a palette of options seemingly the instant things go wrong. We have chosen other carriers over Southwest on occasion because they play nicely with Orbitz and TripIt.

I can’t tell if Southwest’s tradeoffs are worth it or not. But any business person must at least recognize there is a tradeoff. That’s the real lesson. You need to find a way to value the data you collect and be able to compare it against the opportunity of a) happier clients and b) potentially accessing more clients. This is particularly true since many customers probably (and rightly) will feel that is data is as much theirs as it is yours. They did co-create it.

Ultimately if you increase the transaction costs of the experience – because you want to shut other actors out – you will lose customers. Southwest already has.

Definitely expect more of these types of legal battles in the future. Your data is now as important as the service you use. This makes it both powerful, and dangerous in the hands of the wrong people.

I Stand for My Rights & Privacy: The Coming Online Police State

6 Replies

“He can either stand with us or with the child pornographers.”

This was Mr. Toews’s, the Minister of Justice, counterattack to a question in the house regarding concerns of letting the police monitor citizens internet use without a warrant.

Apparently this is our choice: a big brother state or child pornography.

This is, of course, ridiculous. Not to mention frightening. But this is the world Canadians will be entering in a few short weeks once the new Conservative Crime bill passes. The provisions that require a warrant, are interesting: the bill forces internet service providers to record and make available, to both police and governments, their customers internet activity such as the websites they visit. Citizen, understand, this now means that Bell, Rogers or anyone else that provides you with internet on your phone or at your home will now be recording every website you visit. Disturbed about that invasion of privacy? It gets worse.

Most disconcerting is that police would be allowed to obtain your email address, your IP addresses (which often identifies you on the Internet – your home, for example, likely has an IP address), or your mobile phone number and other information without a warrant. They just have to demand it. Suddenly a lot of what you can do online can be monitored by the police – again, without a warrant.

It isn’t just opposition members who are concerned. The Federal Privacy Commissioner and provincial counterparts are deeply concerned. They understand what this means. As Jennifer Stoddart, the federal Privacy Commissioner wrote to Minister Toews:

I am also concerned about the adoption of lower thresholds for obtaining personal information from commercial enterprises. The new powers envisaged are not limited to specific, serious offences or urgent or exceptional situations. In the case of access to subscriber data, there is not even a requirement for the commission of a crime to justify access to personal information – real names, home address, unlisted numbers, email addresses, IP addresses and much more – without a warrant.

In a few short weeks, this will be our reality: we will live in a country where the government can gain access to information that enables them to monitor its citizens online without a warrant. Obviously, the opportunities for abuse are astounding. If you are a ~~radical element~~ non-profit advocacy group that disagrees with the government, you’re probably doubly concerned. Of course, if you are an regular citizen I hope you haven’t written any anonymous comments in opposition to the Gateway Pipelines, since this legislation, combined with the government’s new focus on eco-terrorists (they are as much a threat as neo-nazi groups apparently) could make you a “vulnerable individual” and so an obvious target for security forces.

Of course the real irony of all this is that while the government seeks to increase its powers to monitor Canadians online it has used the opposite argument – the fear of government intrusion into citizens lives – to end the long gun registry. Not 6 days ago, Conservative Larry Miller (Bruce–Grey–Owen Sound) expressed his concern about how the gun registry would help foster a police state:

[…] Before I discuss the bill I would like to review how we arrived at where we are today. I would like to share with the House a quote from former Liberal justice minister Allan Rock: “I came to Ottawa last year, with a firm belief that the only people in Canada who should have firearms are police officers and the military.”

Does that sound familiar? Adolf Hitler, 1939.

You know what really reminds me of Adolf Hitler, 1939? A government that seeks to monitor the actions of all its citizens. That ask companies to record their activities in their homes and their places of work and that gives the police the right to access their personal information without a warrant. As a father I agree we need to fight child pornography, but I’m not willing to sign away my – or my children’s – civil rights and online privacy. I suspect most Canadians, as they learn more about this bill, will feel the same way. They don’t want any government, Conservative, Liberal or NDP, recording what they do, or accessing information about them without a warrant from an independent judiciary.

Two Reasons I love blogging: Helping out great communities

The Exciting Launch of Represent and What It Says About Open Data in Canada

6 Replies

Last week a group of volunteer programs from across Canada announced the launch of Represent – a website that tries to map all of Canada’s boundaries. Confused? Don’t be. It’s simple. This is a nifty piece of digital infrastructure – try visiting the website yourself! After identifying where you are located it will then tell you which MP riding, MLA/MPP district and census subdivision you are located in.

So why does this matter?

What’s important about a site like Represent (much like its cousin site Mapit, which offers a similar service in the UK) is that other websites and applications can use it to offer important services, like letting a user know who their MP is, and thus who their complaint email should be sent to, or identify what by-laws are applicable in the place where they are standing. Have you ever visited the site of a ~~radical group~~ non-profit which urged you to write your MP? With Represent that organization can now easily and cheaply create a widget that would figure out where you are, who you MP is, and ensure you had the right address or email address for your letter. This significantly lowers the barrier to advocacy and political mobilization.

This is why I consider sites like Represent to be core digital infrastructure for a 21st century democracy. Critical because the number of useful services that can educate and engage citizens on politics and government is virtually limitless.

But if we accept that Represent is critical, the site’s limits tell us a lot about the state of our democratic institutions in general, and our open data policy infrastructure in particular. In this regard, there are three insights that come to mind.

1) The information limits of Represent

While Represent can locate any of the federal and provincial ridings (along with the elected official in them) there are remarkably few cities for which the service works. Calgary, Charlottetown, Edmonton, Mississauga, Montreal, Ottawa, Stratford, Summerside, Toronto and Windsor are all that are identified. (The absence of Vancouver – my home town – is less alarming as the city does not have wards or boroughs, we elect 10 councillors in an at large system). The main reason you won’t find more cities available is simply because many cities choose not to share their ward boundary data with the public. And of course, things don’t need to stop with just city wards, there is no reason what Represent couldn’t also tell you which school district you are in, or even which specific school catchment area you are in, in say Vancouver, or North Vancouver.

The paucity of data is an indication of how hard it is to get data from most cities and provinces about the communities in which we live in. There has been great success in getting open data portals launched in several cities – and we should celebrate the successes we’ve had – but the reality is, only a tiny fraction of Canadian cities share data about themselves. In the overwhelming majority, useful data about electoral boundaries, elected officials, schools, etc… exists and are sued internally by governments (paid for by our tax dollars) but they are never shared publicly and so cannot help drive democratic engagement.

So here’s a new rule. If your city boundary data isn’t in Represnt – your city is screwing up. It’s a pretty simple metric.

Oh, and Canada Post, you’re the biggest offender of them all. Your data is the default location specific data set in the country – the easiest way to locate where someone is. Being able to map all this data to postal codes is maybe the most important piece of the puzzle, but sadly, Canada Post clings to data our tax dollars subsidize the creation and maintenance of. Of course, in the UK, they made Postal Code data completely open.

2) Lack of Standards

And of course, even when the data does exist, it isn’t standardized. Previously non-profits, think tanks and even companies would have to manage data in various forms from innumerable sources, (or pay people lots of money to organize the data for them). It shouldn’t be this way. While it is great the Represent helps standardize the data, standard data schemas should already exist for things like MPP/MLA/MNA ridings and descriptions. Instead we have to rely on a group of volunteer hackers to solve a problem the countries leading governments are unable, or unwilling to address.

3) Licenses & Legality

However, the real place where Represent shows the short comings in Canada’s open data infrastructure is the way the site struggles to deal with the variety of licenses under which it is allowed to use data from various sources.

The simple fact is, in Canada, most “open data” is in fact not open. Rather that have serious restrictions placed upon them that limit the ability of sites like Rperesent.ca to be useful.

For example, many, many cities still have “share alike” clauses in their licenses, clauses that mean any product created using their data may not have “further restrictions of any kind.” But of course, each city with a “share alike” clause has slightly different restrictions in their license meaning that none of them can be combined. In the end it means that data from Vancouver cannot be used with data from Edmonton or from Montreal. It’s a complete mess.

Other jurisdictions have no license on their data. For example electoral boundary data for British Columbia, Prince Edward Island, Newfoundland and Nova Scotia is unlicensed, leaving users very unclear about their rights. Hint to these and other jurisdictions: just make it open.

What Represent really demonstrates is that there is a need for a single, standard open data license across Canada. It’s something I’m working on. More to report soon I hope.

Despite these hurdles, Represent is a fantastic project and site – and they are looking for others to help them gather more data. If you want to support them (and I strongly encourage you to do so) check out the bottom of their home page. Big congratulations to everyone involved.

Requests for Endorsements: My Application to Attend The Open Government Partnership

16 Replies

Dear friends,

Below (first in english then in a rough french translation – my spoken is much better than my written so I’ve relied on Google translate) is my application letter to attend the April 16-18 Annual Open Government Partnership meeting in Brasilia as a Civil Society Representative.

The first reason I’m posting this is in an effort to make my application as transparent as possible. By posting it, if people have comments, additions, questions,I thought it might be helpful to post it. I can’t promise to engage every issue – I know some people have disagreed with positions I have taken – but I will do my best to engage what I believe is a broad community that cares about everything from open data, to access to information requests, to simply having more accessible MPs and websites.

Another – connected – reason I’m posting my application is I need your help. As part of the process I’m asked to submit organizations that endorse my application. For those of you who already have expressed this support… thank you. I will definitely add you to this letter. If you are willing to support my application and have not yet let me know, please do send me an email (or comment below). I need to add you organization (if any), your name and your email address.

The application deadline is February 6th (yes, it would have been better to get this up sooner) so ideally any feedback would be great to get today or this weekend. That said, I would still love to get feedback after the 6th in case I do go, I’d still like to be able to listen for and act on what you care about.

If you have other questions about or suggestions for the application, please let me know. Thank you!

Dear Open Government Partnership,

I am writing to express my desire to participate as a local civil society member at the 2012 OGP meeting in Brazil.

I have been active in Open Government for the past 7 years working as an advocate, adviser and chronicler of transparency in Canada and around the world.

Background and Engagement in OGP Issues

As an advocate, I’ve spoken about the challenges around Open Government across Canada at the municipal, provincial and federal levels. Internationally, I’ve given the opening keynote at the last two International Open Government Data Camps hosted by the Open Knowledge Foundation, was invited to address the 7^th International Conference of Information Commissioners in Ottawa and Mexico’s Semana Nacional de la Transparencia, and given talks at the Gov 2.0 Expo and Summit in Washington DC (at which the host, Tim O’Reilly, stated, “If you read only one blog in the Gov 2.0 space, you should read eaves.ca”).

In addition to speaking, I’ve tried to actively demonstrate ways open government can improve our communities. My belief is that we need a broad set of ways to engage citizens in open government – some will be motivated by accountability, but others will be engaged by simply having their lives made easier. Consequently, in pursuit of advancing accountability, I worked with a team of developers sponsored by Microsoft to create Emitter.ca, a website that mashes up pollution, politician, and company data to enable citizens to identify heavy polluters in their neighborhoods and region. While trying to find ways to show how open government can promote better services, I worked with friends to create Recollect.net, a simple service that uses open data to remind citizens to take out their recycling and garbage. And as an advocate, when the Federal Government lagged by years behind the US and the UK in creating an open data portal, I created http://www.datadotgc.ca which sought to track open data sets already being shared on various ministry website to demonstrate that, contrary to its position, the government already had a policy infrastructure to do open data. This site helped pressure the government into launching its own formal open data website.

I’ve also worked actively in the Open Government space by engaging with governments directly. In 2009, I co-drafted the open motion for the Mayor and Council of the City of Vancouver. This lead to the first city council motion in the world directing city staff to make open data part of their activities. As a result, Vancouver launched the world’s second municipal open data portal (after Washington DC). I also worked with the city’s IT staff to revise procurement rules to make open data a required specification as well make open source software a permissible option. At the federal level, I worked with NGO and educated key government players to shift Canada – which had been skeptical – into agreeing to participate in the OGP.

I have, however, also been critical when necessary. I’ve written pieces in newspapers and on my blog when governments have failed to be transparent or have taken steps in the wrong direction.

Meeting Contribution and Learnings

The Open Government Partnership provides civil society members with a rare carrot and stick for engaging their government on the issue of open government. Because it requires the government to set clear goals around transparency in an international forum, it provides civil society with leverage to hold the government to account.

While this leverage must be handled responsibly (factually incorrect critiques will erode the public’s confidence in civil society organizations), if properly used it can compel the government to move more aggressively on fixing problems in this area. This is of particular urgency in Canada, where government transparency has been in decline over the past several decades. Once considered cutting age, Canada’s access to information regime is wildly out of date. Access to information requests are handled more slowly than ever and access to government information – with the exception of a dramatic improvement in the area of open data – is becoming more restrictive. My goal at the Open Government Partnership will be to engage other government and NGOs to understand the transparency benchmarks being set by other governments that can be used as a way by which Canadians can judge the progress of their own government. I can also share my own experiences in moving open data policies through local and national governments, as well as some approaches for engaging non-traditional stakeholders in this space.

OGP Outreach Plan

Upon returning from the Open Government Partnership, I commit to aggregating feedback from various actors in an effort to have it directly inform the goals and actions of the Canadian Federal Government with whom I have a critical but cordial relationship. I will also, of course, blog about what I believe are the key benchmarks Canadian civil society actors should be using to pressure and measure the Canadian government against. Finally, I commit to get on the phone with any civil society actor that contacts me and discuss with them what I observed and how I believe it impacts their organization.

Funding

With regard to funding, my hope is that I will be able to find some alternative funding for travel. As a result, I’m looking to have my room and board covered along with some of my travel costs. My hope is that by doing so, it might be possible to use some of the OGP funds to support the travel of others.

The organizations, names and emails of the leaders endorsing my application

Organisation Leader Email

Thank you for considering my application. Please let me know if you have any questions.

Sincerely,

David Eaves

version française

Cher Partenariat gouvernement ouvert,

Je vous écris pour exprimer mon désir de participer en tant que membre de la société civile locale à la réunion de 2012 du OGP au Brésil.

J’ai été actif au sein du gouvernement ouvert pour les 7 dernières années de travail comme un défenseur, de conseiller et chroniqueur de la transparence au Canada et partout dans le monde.

Contexte et engagement dans les questions du OGP

En tant que défenseur, j’ai parlé des défis autour de la transparence du gouvernement partout au Canada aux niveaux municipal, provincial et fédéral. Au niveau international, je vous ai donné le discours d’ouverture lors des deux dernières International Open Camps de données hébergées par le gouvernement de l’Open Knowledge Foundation, a été invité à s’adresser à la 7e Conférence internationale des commissaires à l’information à Ottawa et du Mexique Semana Nacional de la Transparencia, et donné des conférences au l’Gov 2.0 Expo et au Sommet de Washington DC (à laquelle l’hôte, Tim O’Reilly, a déclaré: «Si vous ne voulez lire un blog dans l’espace Gov 2.0, vous devriez lire eaves.ca”).

En plus de parler, j’ai essayé de démontrer activement des façons un gouvernement ouvert pouvons améliorer nos collectivités. Ma conviction est que nous devons un large éventail de moyens pour engager les citoyens dans un gouvernement ouvert, dont certains seront motivés par la reddition de comptes, mais d’autres seront engagés en ayant simplement leur vie plus facile. Par conséquent, dans la poursuite de l’avancement de responsabilité, j’ai travaillé avec une équipe de développeurs parrainés par Microsoft pour créer Emitter.ca, un site Web qui écrase les données de la pollution, homme politique et la société pour permettre aux citoyens d’identifier les pollueurs dans leurs quartiers et de la région. Tout en essayant de trouver des façons de montrer le degré d’ouverture du gouvernement peut favoriser de meilleurs services, j’ai travaillé avec des amis pour créer Recollect.net, un service simple qui utilise les données ouvertes à rappeler aux citoyens de prendre leur recyclage et des ordures. Et en tant que défenseur, lorsque le gouvernement fédéral retardée par des années derrière les Etats-Unis et au Royaume-Uni dans la création d’un portail de données ouverte, j’ai créé http://www.datadotgc.ca qui a cherché à suivre les données ouvertes fixe d’ores et déjà partagé sur le site Web du ministère, pour montrer que, contrairement à sa position, le gouvernement avait déjà une infrastructure politique de faire de données ouvertes. Ce site a contribué pression sur le gouvernement en lançant son propre site web officiel de données ouvert.

J’ai aussi travaillé activement dans l’espace ouvert par gouvernement collaboration avec les gouvernements directement. En 2009, j’ai co-rédigé la motion ouverte pour le maire et le conseil de la ville de Vancouver. Cela a conduit à la motion du Conseil municipal première dans le monde de diriger le personnel municipal pour faire partie des données ouverte de leurs activités. En conséquence, Vancouver a lancé seconde municipale dans le monde Portail de données ouvertes (après Washington DC). J’ai également travaillé avec TI de la Ville de personnel pour réviser les règles de passation des marchés pour rendre les données ouvertes une spécification requise ainsi rendre les logiciels open source d’une option acceptable. Au niveau fédéral, j’ai travaillé avec des ONG et instruits joueurs clés du gouvernement de transférer au Canada – qui avait été sceptique – en acceptant de participer à l’OGP.

J’ai, cependant, a également joué un rôle crucial lorsque cela est nécessaire. J’ai écrit des pièces dans les journaux et sur mon blog où les gouvernements n’ont pas réussi à être transparents ou ont pris des mesures dans la mauvaise direction.

Comment vais-je contribuer à la réunion et ce, je veux apprendre de fréquenter;

Le Partenariat sur la transparence gouvernementale offre aux membres de la société civile avec une carotte et du bâton rare pour engager leur gouvernement sur la question d’un gouvernement ouvert. Parce qu’il oblige le gouvernement à fixer des objectifs clairs à la transparence dans un forum international, il offre à la société civile avec effet de levier pour obliger le gouvernement à rendre compte.

Bien que cet effet de levier doit être géré de façon responsable (dans les faits critiques incorrectes va éroder la confiance du public dans les organisations de la société civile), si elle est correctement utilisée, elle peut contraindre le gouvernement à agir de façon plus agressive sur la résolution des problèmes dans ce domaine. Ceci est d’une urgence particulière au Canada, où la transparence du gouvernement a été en déclin au cours des dernières décennies. Autrefois considéré comme la coupe d’âge, l’accès du Canada au système d’information est follement hors de date. Demandes d’accès à l’information sont traitées plus lentement que jamais et l’accès à l’information du gouvernement – à l’exception d’une amélioration spectaculaire dans la zone de données ouvertes – est de plus en plus restrictive. Mon objectif au sein du Partenariat sur la transparence gouvernementale sera d’engager les autres gouvernements et les ONG pour comprendre les repères de transparence étant fixés par les gouvernements d’autres qui peuvent être utilisés comme un moyen par lequel les Canadiens peuvent juger l’état d’avancement de leur propre gouvernement. Je peux aussi partager mes propres expériences dans le déplacement des politiques d’ouverture des données par les gouvernements locaux et nationaux, ainsi que quelques approches pour engager les acteurs non traditionnels dans cet espace.

Description de mon OGP plan de sensibilisation avec la société civile et d’autres où je rentre chez moi

Au retour de la société en commandite gouvernement ouvert, je m’engage à agréger les commentaires des différents acteurs dans un effort pour faire informer directement les objectifs et les actions du gouvernement fédéral canadien avec qui j’ai une relation critique, mais cordiale. Je vais aussi, bien sûr, blog sur ce que je crois sont les principaux critères canadiens acteurs de la société civile devraient être en utilisant à la pression et de mesurer le gouvernement canadien contre. Enfin, je m’engage à prendre le téléphone avec n’importe quel acteur de la société civile qui communique avec moi et discuter avec eux ce que j’ai observé et comment je crois que son impact sur leur organisation.

Financement

En ce qui concerne le financement, mon espoir est que je serai en mesure de trouver un financement alternatif pour les voyages. En conséquence, je suis à la recherche d’avoir ma chambre et pension comprise avec certains de mes frais de déplacement. Mon espoir est que, ce faisant, il pourrait être possible d’utiliser une partie des fonds du OGP à financer les déplacements des autres.

Les organisations, les noms et les courriels des dirigeants approuvant ma demande

Organisation Leader Courriel

Merci de considérer ma demande. S’il vous plaît laissez-moi savoir si vous avez des questions.

Sincèrement,

David Eaves