Category Archives: commentary

Access to Information, Open Data and the Problem with Convergence

In response to my post yesterday one reader sent me a very thoughtful commentary that included this line at the end:

“Rather than compare [Freedom of Information] FOI legislation and Open Gov Data as if it’s “one or the other”, do you think there’s a way of talking about how the two might converge?”

One small detail:

So before diving in to the meat let me start by saying I don’t believe anything in yesterday’s post claimed open data was better or worse than Freedom of Information (FOI often referred to in Canada as Access to Information or ATI). Seeing FOI and open data as competing suggests they are similar tools. While they have similar goals – improving access – and there may be some overlap, I increasingly see them as fundamentally different tools. This is also why I don’t see an opportunity for convergence in the short term (more on that below). I do, however, believe open data and FOI processes can be complimentary. Indeed, I’m hopeful open data can alleviate some of the burden placed on FOI system which are often slow. Indeed, in Canada, government departments regularly violate rules around disclosure deadlines. If anything, this complimentary nature was the implicit point in yesterday’s post (which I could have made more explicit).

The Problem with Convergence:

As mentioned above, the overarching goals of open data and FOI systems are similar – to enable citizens to access government information – but the two initiatives are grounded in fundamentally different approaches to dealing with government information. From my view FOI has become a system of case by case review while open data is seeking to engage in an approach of “pre-clearance.”

Part of this has to do with what each system is reacting to. FOI was born, in part, out of a reaction to scandals in the mid 20th century which fostered public support for a right to access government information.

FOI has become a powerful tool for accessing government information. But the infrastructure created to manage it has also had some perverse effects. In some ways FOI has, paradoxically made it harder to gain access to government information. I remember talking to a group of retired reporters who talk about how it was easier to gain access to documents in a pre-FOI era since there were no guidelines and many public servants saw most documents as “public” anyways. The rules around disclosure today – thanks in part to FOI regimes – mean that governments can make closed the “default” setting for government information. In the United States the Ashcroft Memo serves as an excellent example of this problem. In this case the FOI legislation actually becomes a tool that helps governments withhold documents, rather than enable citizens to gain legitimate access.

But the bigger problem is that the process by which access to information requests are fulfilled is itself burdensome. While relevant and necessary for some types of information it is often overkill for others. And this is the niche that open data seeks to fill.

Let me pause to stress, I don’t share the above to disparage FOI. Quite the opposite. It is a critical and important tool and I’m not advocating for its end. Nor am I arguing the open data can – in the short or even medium term – solve the problems raised above.

This is why, over the short term, open data will remain a niche solution – a fact linked to its origins. Like FOI Open data has its roots in government transparency. However, it also evolved out of efforts to tear down antiquated intellectual property regimes to the facilitate sharing of data/information (particularly between organizations and governments). Thus the emphasis was not on case by case review of documents, but rather of clearing rights to categories of information, both created and to be created in the future. In other words, this is about granting access to the outputs of a system, not access to individual documents.

Another way of thinking about this is that open data initiatives seek to leverage the benefits of FOI while jettisoning its burdensome process. If a category of information can be pre-clear in advanced and in perpetuity for privacy, security and IP concerns then FOI processes – essential for individual documents and analysis – becomes unnecessary and one can reduce the transaction costs to citizens wishing to access the information.

Maybe, in the future, the scope of these open data initiatives could become broader, and I hope they will. Indeed there is, ample evidence to suggest that technology could be used to pre-clear or assess the sensitivity of any government document. An algorithm that assess a mixture of who the author is, the network of people who review it and a scan of the words would probably allow ascertain if a document could be released to an ATIP request in seconds, rather than weeks. It could at least give a risk profile and/or strip out privacy related information. These types of reforms would be much more disruptive (in the positive sense) to FOI legislation than open data.

But all that said, just getting the current focus of open data initiatives right would be a big accomplishment. And, even if such initiatives could be expanded, there are limits. I am not so naive to believe that government can be entirely open. Nor am I sure that would be an entirely good outcome. When trying to foster new ideas or assess how to balance competing interests in society, a private place to initiate and play with ideas may be essential. And despite the ruminations above, the limits of government IT systems means there will remain a lot of information – particularly non-data information like reports and analysis – that we won’t be able to “pre-clear- for sharing and downloading. Consequently an FOI regime – or something analogous – will continue to be necessary.

So rather than replace or converge with FOI systems, I hope open data will, for the short to medium term actually divert information out of the FOI, not because it competes, but because it offers a simpler and more efficient means of sharing (for both government and citizens) certain types of information. That said, open data initiatives offer none of the protections or rights of FOI and so this legislation will continue to serve as the fail safe mechanism should a government choose to stop sharing data. Moreover, FOI will continue to be a necessary tool for documents and information that – for all sorts of reasons (privacy, security, cabinet confidence, etc…) cannot fall under the rubric of an open data initiative. So convergence… not for now. But co-existence feels both likely and helpful for both.

Calculating the Value of Canada’s Open Data Portal: A Mini-Case Study

11 Replies

Okay, let’s geek out on some open data portal stats from data.gc.ca. I’ve got three parts to this review: First, an assessment on how to assess the value of data.gc.ca. Second, a look at what are the most downloaded data sets. And third, some interesting data about who is visiting the portal.

Before we dive in, a thank you to Jonathan C sent me some of this data to me the other day after requesting it from Treasury Board, the ministry within the Canadian Government that manages the government’s open data portal.

1. Assessing the Value of data.gc.ca

Here is the first thing that struck me. Many governments talk about how they struggle to find methodologies to measure the value of open data portals/initiatives. Often these assessments focus on things like number of apps created or downloaded. Sometimes (and incorrectly in my mind) pageviews or downloads are used. Occasionally it veers into things like mashups or websites.

However, one fairly tangible value of open data portals is that they cheaply resolve some access to information requests – a point I’ve tried to make before. At the very minimum they give scale to some requests that previously would have been handled by slow and expensive access to information/freedom of information processes.

Let me share some numbers to explain what I mean.

The Canada Government is, I believe, only obligated to fulfill requests that originate within Canada. Drawing from the information in the charts later in this post, let’s say assume there were a total of 2200 downloads in January and that 1/3 of these originated from Canada – so a total of 726 “Canadian” downloads. Thanks to some earlier research, I happen to know that the office of the information commissioner has assessed that the average cost of fulfilling an access to information request in 2009-2010 was $1,332.21.

So in a world without an open data portal the hypothetical cost of fulfilling these “Canadian” downloads as formal access to information requests would have been $967,184.46 in January alone. Even if I’m off by 50%, then the cost – again, just for January – would still sit at $483,592.23. Assuming this is a safe monthly average, then over the course of a year the cost savings could be around $11,606,213.52 or $5,803,106.76 – depending on how conservative you’d want to be about the assumptions.

Of course, I’m well aware that not every one of these downloads would been an information request in a pre-portal world – that process is simply to burdensome. You have to pay a fee, and it has to be by check (who pays for anything by check any more???) so many of these users would simply have abandoned their search for government information. So some of these savings would not have been realized. But that doesn’t mean there isn’t value. Instead the open data portal is able to more cheaply reveal latent demand for data. In addition, only a fraction of the government’s data is presently on the portal – so all these numbers could get bigger still. And finally I’m only assessing downloads that originated inside Canada in these estimates.

So I’m not claiming that we have arrived at a holistic view of how to assess the value of open data portals – but even the narrow scope of assessment I outline above generates financial savings that are not trivial, and this is to say nothing of the value generated by those who downloaded the data – something that is much harder to measure – or of the value of increased access to Canadians and others.

2. Most Downloaded Datasets at data.gc.ca

This is interesting because… well… it’s just always interesting to see what people gravitate towards. But check this out…

Data sets like the Anthropogenic disturbance footprint within boreal caribou ranges across Canada may not seem interesting, but the ground breaking agreement between the Forest Products Association of Canada and a coalition of Environmental Non-Profits – known as the Canadian Boreal Forest Agreement (CBFA) – uses this data set a lot to assess where the endangered woodland caribou are most at risk. There is no app, but the data is critical in both protecting this species and in finding a way to sustainably harvest wood in Canada. (note, I worked as an adviser on the CBFA so am a) a big fan and b) not making this stuff up).

It is fascinating that immigration and visa data tops the list. But it really shouldn’t be a surprise. We are of course, a nation of immigrants. I’m sure that immigration and visa advisers, to say nothing of think tanks, municipal governments, social service non-profits and English as a second language schools are all very keen on using this data to help them understand how they should be shaping their services and policies to target immigrant communities.

There is, of course, weather. The original open government data set. We made this data open for 100s of years. So useful and so important you had to make it open.

And, nice to see Sales of fuel used for road motor vehicles, by province and territory. If you wanted to figure out the carbon footprint of vehicles, by province, I suspect this is a nice dataset to get. Probably is also useful for computing gas prices as it might let you get a handle on demand. Economists probably like this data set.

All this to say, I’m less skeptical than before about the data sets in data.gc.ca. With the exception of weather, these data sets aren’t likely useful to software developers – the group I tend to hear most from – but then I’ve always posited that apps were only going to be a tiny part of the open data ecosystem. Analysis is king for open data and there does appear to be people out there who are finding data of value for analyses they want to make. That’s a great outcome.

Here are the tables outlining the most popular data sets since launch and (roughly) in February.

Top 10 most downloaded datasets, since launch

	DATASET	DEPARTMENT	DOWNLOADS
1	Permanent Resident Applications Processed Abroad and Processing Times (English)	Citizenship and Immigration Canada	4730
2	Permanent Resident Summary by Mission (English)	Citizenship and Immigration Canada	1733
3	Overseas Permanent Resident Inventory (English)	Citizenship and Immigration Canada	1558
4	Canada – Permanent residents by category (English)	Citizenship and Immigration Canada	1261
5	Permanent Resident Applicants Awaiting a Decision (English)	Citizenship and Immigration Canada	873
6	Meteorological Service of Canada (MSC) – City Page Weather	Environment Canada	852
7	Meteorological Service of Canada (MSC) – Weather Element Forecasts	Environment Canada	851
8	Permanent Resident Visa Applications Received Abroad – English Version	Citizenship and Immigration Canada	800
9	Water Quality Indicators – Reports, Maps, Charts and Data	Environment Canada	697
10	Canada – Permanent and Temporary Residents – English version	Citizenship and Immigration Canada	625

Top 10 most downloaded datasets, for past 30 days

	DATASET	DEPARTMENT	DOWNLOADS
1	Permanent Resident Applications Processed Abroad and Processing Times (English)	Citizenship and Immigration Canada	481
2	Sales of commodities of large retailers – English version	Statistics Canada	247
3	Permanent Resident Summary by Mission – English Version	Citizenship and Immigration Canada	207
4	CIC Operational Network at a Glance – English Version	Citizenship and Immigration Canada	163
5	Gross domestic product at basic prices, communications, transportation and trade – English version	Statistics Canada	159
6	Anthropogenic disturbance footprint within boreal caribou ranges across Canada – As interpreted from 2008-2010 Landsat satellite imagery	Environment Canada	102
7	Canada – Permanent residents by category – English version	Citizenship and Immigration Canada	98
8	Meteorological Service of Canada (MSC) – City Page Weather	Environment Canada	61
9	Sales of fuel used for road motor vehicles, by province and territory – English version	Statistics Canada	52
10	Government of Canada Core Subject Thesaurus – English Version	Library and Archives Canada	51

3. Visitor locations

So this is just plain fun. There is not a ton to derive from this – especially as IP addresses can, occasionally, be misleading. In addition, this is page view data, not download data. But what is fascinating is that computers in Canada are not the top source of traffic at data.gc.ca. Indeed, Canada’s share of the traffic is actually quite low. In fact, in January, just taking into account the countries in the chart (and not the long tail of visitors) Canada accounted for only 16% of the traffic to the site. That said, I suspect that downloads were significantly higher from Canadian visitors – although I have no hard evidence of this, just a hypothesis.

•Total visits since launch: 380,276 user sessions

Attack of the Drones – How Surveillance May Change our Culture

14 Replies

I’ve been following the rise of do it yourself (DIY) drones for a few years now, ever since Chris Anderson, the editor of Wired magazine, introduced me to the topic in a podcast. And yes, I’m talking about flying drones… Like those the US Air Force uses to monitor – and attack – enemy forces in Afghanistan. Except, in the case of DIY drones, they are smaller, cheaper, and are being built by a growing legion of hobbyists, companies and enthusiasts all around the world, many of whom are sharing open source UAV plans that can be downloaded off the internet.

You many not know it, but there could be drones in your neighborhood. And this has real implications.

Take, for example, a story that really grabbed my attention a few weeks ago. An animal rights group called SHARK chose to deploy a drone to monitor a live pigeon shoot taking place on private land. It turns out the mere presence of their drone caused the shoot to be cancelled. To begin with, that says a lot of drone’s effectiveness. But what was really interesting was how, in frustration, one or some of the shooters then hid and shot the drone out of the sky.

Think about this.

Here you have a group using what is essentially a mini-helicopter to monitor an activity taking place on what is private property. Then, in response, the other party fires live rounds at the drone and causes it to crash. And all of playing out near a US highway (not a major one, but still, a public road). This is a privacy, legal, and public safety nightmare. The policy and societal implications are significant.

And this is not an isolated use. As the Economist pointed out in its excellent write up on civilian drones in this week’s Quarterly Technology Review, drones are already being used by an environmental group to locate and track Japanese whalers. In the US several police forces already operate drones – including one in Texas which, frighteningly, has the capacity to launch grenades. George Clooney funds a non-profit that uses satellites to monitor Sudan in an effort to prevent atrocities through transparency. Can drones be far behind?

I share all this because, these days, people are often most frightened by the state’s growing interest to monitor what we do online. Here in Canada for example, the government has proposed a law that would require telecommunications firm have the ability to record, and save, everyone’s online activities. But technology to monitor people offline, in the physical world, is also evolving. More importantly, it is becoming available to ordinary citizens. This will have real impacts.

As my friend Luke C. pointed out the other day, it is entirely conceivable that, in 5-7 years, there could be drones that would follow your child as he walks to school. You can of course, already choose to monitor your child by giving them a cell phone and tracking the GPS device within it, but a drone would have several advantages. It would be harder for someone to destroy or “disconnect” from your child. It could also record and save remotely everything that is going on – in order to prevent anyone from harassing or bullying them. It might even remind them to look both ways before crossing the street, in case they forget. Or, because of its high vantage point, it could pick out and warn your child of cyclists and cars they failed to observe. Once your kid is safely at school the drone could whiz home and recharge in time to walk them home at the end of the day. This may all seem creepy to you, but if such a drone cost $100 dollars, how many parents do you think would feel like it was “the responsible thing to do.” I suspect a great deal. Even if it was only 5% of parents… that would be a lot of drones.

And of course there are thousands of other uses. Protestors might want a drone observing them, just so that any police brutality could be carefully recorded for later. Cautious adults may want one hovering over them, especially when going into an unfamiliar or unsafe neighborhoods. Or maybe you’ll want one for your elderly parents… just in case something happens to them? It’s be good to be able to pull them up on a live feed, from anywhere.

If you think back 20 years ago and told someone you were going to give them a device that would enable their government to locate them within a few feet at any given moment, they would likely have imagined some Orwellian future. But this is, functionally, what any smart phone can do. Looking forward 20 years, I ask myself: would my child feel monitored if he has a drone helping him get to school? Or maybe he will he feel unsafe without it? Or maybe it will feel like his Hogwart’s owl, a digital pet? Or maybe all of these outcomes? I’m not sure the answer is obvious.

My larger point is that the pressure to create the surveillance society isn’t going to come exclusively from the state. Indeed, we may find ourselves in a surveillance society not because the state demands it, but because we want the tools for our own useful and/or selfish ends. Some people may argue that this may level the playing field between citizens and the state or powerful organizations. I hope that is true. But maybe the mass adoption of such tools will simply normalize surveillance in our society and culture. That might, in turn, make it easier for the state, or other organizations, or just everyone else, to monitor us.

What I do know is that our government, our police forces, our neighborhoods are wholly unprepared for this. That’s okay, they have some time. But it is coming. At some point we will be living in a society where the technology will exist to enable anyone to deploy a drone that can observe anyone else in a public space, and maybe even in a private space. The challenges and complexities for policy makers are significant, and the implications for our communities, probably even more so. Either way it’s going to make many people’s lives a lot more complicated.

Note, I suspect there are typos in this, but it is 2 am and wordpress already deleted the first draft of this post killing a couple hours of work… so my capacity and patience is low. I hope you’ll forgive me a little.

Let's Hack data.gc.ca

6 Replies

In just under two weeks data.gc.ca will celebrate its one year anniversary. This will also mark the period that the pilot project is officially supposed to end.

Looking at data.gc.ca three things stand out. First, the license has improved a great deal since its launch. Second, a LOT of data has been added to the site over the last year. And finally, the website is remarkably bad at searching for data and enabling a community of users.

Indeed, I believe that a lot of people have stopped visiting the site and don’t even know what data is available. My suspicion is that almost none of us know what is actually available since a) there is a lot, b) much of it is not sexy and c) it is very hard to search.

Let’s do something about that.

I have managed to create, and upload to buzzdata, a list of all the data sets in data.gc.ca – both geographic and non-geographic data sets.

I’m proposing that we go through the data.gc.ca data sets and find what is interesting to each of us, and on March 15th, find a way to highlight it or talk about it so that other people find out about it. Maybe you tweet about it (use the hashtah #gcdata) or blog about it.

Even more interesting would be if we could find a way to do it collaboratively – have a way of collectively marking what data sets are interesting (in say, a piratepad somewhere). If someone had a clever proposal about how to go through all the datasets, I’d love for us to collectively highlight the high value datasets (if there are any) available in data.gc.ca.

Speaking with the great community of open data activists in Ottawa, we brainstormed about organizing an event after work on the 15th where people might get together and do this. We could call it “The Big Search” – an effort in any city where people are interested to gather and comb through the data. All with the goal of signaling to developers, non-profits, journalists and others, what, if any, data in data.gc.ca might be of interest for analysis, applications, or other uses. In addition, this exercise would also help us write supportive and critical comments about the government’s open data trial.

Finally, and most ambitiously, I’ve heard some people say they’d like to design an alternative data portal – I’m definitely game for that and am happy to offer up the datadotgc.ca url for that too.

So, I’m throwing this out there. If there is interest, please comment below. Would love to hear your thoughts and hope we can maybe organize some events on March 15th, or at least posts data sets in blogs, on facebook and on twitter, that people think are interesting.

The Ground War for Control of Public Space

1 Reply

My partner stumbled on the quote below attributed to Banksy (wikipedia), the English street artist who many consider to be one of the great a modern day satirists.

It’s an interesting take on advertising in public space – for some it will be a familiar call to arms, to others it will feel revolutionary. I find it particularly interesting given the age of customized advertising is coming our way. One of the big questions we’ll soon be asking is, to what degree can a company target an add, to you specifically, in a public space.

Some people find customized Google ads to be eerie, whereas I find them useful, and frankly, understand them to be part of the social contract I accept when I use Google. But using Google is a choice. Walking down a public street and suddenly being hit by an ad that is customized for me? One I did not even implicit ask to see… I don’t know how I feel about that yet. It may feel helpful. But it may feel deeply invasive.

However, I know how Banksy is going to feel. And he’s got some powerful points.

By the way, if you have not seen the documentary about/by Banksy entitled Exit Through the Gift Shop it is very, very much worth catching. It’s on Netflix for those who have it (Which by the way, has a decent documentary film selection).

Here’s the Banksy quote. It’s all over the internet, but I’ve yet to find an attributing link, which would have been nice.

“People are taking the piss out of you everyday. They butt into your life, take a cheap shot at you and then disappear. They leer at you from tall buildings and make you feel small. They make flippant comments from buses that imply you’re not sexy enough and that all the fun is happening somewhere else. They are on TV making your girlfriend feel inadequate. They have access to the most sophisticated technology the world has ever seen and they bully you with it. They are “The Advertisers”and they are laughing at you.

You, however, are forbidden to touch them. Trademarks, intellectual property rights and copyright law mean advertisers can say what they like wherever they like with total impunity.

Fuck that. Any advert in a public space that gives you no choice whether you see it or not is yours. It’s yours to take, re-arrange and re-use. You can do whatever you like with it. Asking for permission is like asking to keep a rock someone just threw at your head.

You owe the companies nothing. Less than nothing, you especially don’t owe them any courtesy. They owe you. They have re-arranged the world to put themselves in front of you. They never asked for your permission, don’t even start asking for theirs.”

Media Watch: The Globe and Mail’s Shifting Headline

2 Replies

Earlier today the Globe and Mail had one of these truly terrible “balanced” articles about the proposed federal crime bill. The headline screamed: Quebec expert backs Tory crime bill amid U.S. warning on sentencing. (Image below)

So who was this expert you might ask? A university professor with years of research on the subject? Maybe some breakthrough research by a young grad student? How about a researcher from a think tank that has been investigated the issue?

Wrong on all accounts. It was, in fact, former Justice Minister Marc Bellemare from the province of Quebec. Of course, you might say… “being a Justice Minister problem should make you an expert.” If only this were the case. If Minister Vic Toews has taught us anything it is that you definitely don’t have to be an expert in something to become a Minister. Nor does being a Minister make you an expert. But the real kicker is that Marc Bellemare was minister for just under a year. Sworn in on April 29th 2003 he resigned on April 27th 2004. Of course, the article makes no reference to the current Justice Minister of Quebec, Jean-Marc Fournier, who is both opposed to the Crime bill and has been minister since August 11th, 2010. That’s a year and a half longer making him 50% more of an expert than Bellemare!

I suspect one of two things happened (both of which I now know are wrong – see update below). Either the Globe reporter simple used language that came packaged in a press release that referred to Marc Bellemare as an expert or worse, in pursuit of “balance” the journalist felt compelled to label Bellemare’s an expert given the second part focused on how a large number of US republican “tough on crime” legislators who created mandatory minimum sentences in the 90s are trying to role them back because they have been a total failure in addressing crime and a disaster financially.

Of course Globe and Mail readers noticed the problem with the “expert” right away. The most voted for comment was the following one (yes, I voted too, might have been my first time):

And slightly further done was a better comment pointing out some further idiotic ideas the Minister had for reforming the justice system.

More interesting is that sometime in the later afternoon EST the Globe changed its landing page, acknowledging the “expert’s” true credentials.

I think this speaks volumes about the Globe – in a good way. Nobody is perfect, we make mistakes. Sun prides itself on getting facts wrong to tell a story and the Globe is demonstrating that they take the opposite tact. So this post isn’t to say “the globe messed up,” it’s about how newspaper can and should react to feedback from readers. It doesn’t mean you change everything all the time, but there are times when the feedback points to changes that will bring about greater clarity. It also says a lot about the power of the audience.

However, it is worth noting, the headline on the story page… remains unchanged.

7:14pm Update

I’ve made some errors of my own in the above post. I assumed above that the journalist had chosen the headline, this is, in fact, not true. As one of the editors from the Globe has pointed out to me on twitter, it was the editor who made the choice. Any assignment of blame on the journalist is misplaced, I definitely apologize for that on my part.

The Surveillance State – No Warrant Required

9 Replies

Yesterday a number of police organizations came out in support of bill C-30 – the online surveillance bill proposed by Minister Vic Toews. You can read the Vancouver Police Department’s full press release here – I’m referencing theirs not because it is particularly good or bad, but simply because it is my home town.

For those short on time, the very last statement, at the bottom of the post, is by far the worst and is something every Canadian should know. The authors of these press releases would have been wise to read Michael Geist’s blog posts from yesterday before publishing. Geist’s analysis shows that, at best, the police are misinformed, at worst, they are misleading the public.

So let’s look at some of the details of the press release that are misleading:

Today I speak to you as the Deputy Chief of the VPD’s Investigation Division, but also as a member of the Canadian Association of Chiefs of Police, and I’m pleased to be joined by Tom Stamatakis, President of both the Vancouver Police Union and Canadian Police Association.

The Canadian Association of Chiefs of Police (CACP) is asking Canadians to consider the views of law enforcement as they debate what we refer to as “lawful access,” or Bill C-30 – “An Act to enact the Investigating and Preventing Criminal Electronic Communications Act and to amend the Criminal Code and other Acts.”

This Bill was introduced by government last week and it has generated much controversy. There is no doubt that the Bill is complex and the technology it refers to can be complex as well.
I would, however, like to try to provide some understanding of the Bill from a police perspective. We believe new legislation will:

assist police with the necessary tools to investigate crimes while balancing, if not strengthening, the privacy rights for Canadians through the addition of oversight not currently in place

So first bullet point, first problem. While it is true the bill brings in some new process, to say it strengthens privacy rights is misleading. It has become easier, not harder, to gain access to people’s personal data. Before, when the police requested personal information from internet service providers (ISPs) the ISPs could say no. Now, we don’t even have that. Worse, the bill apparently puts a gag on order on these warrantless demands, so you can’t even find out if a government agency has requested information about you.

help law enforcement investigate and apprehend those who are involved in criminal activity while using new technologies to avoid apprehension due to outdated laws and technology

allow for timely and consistent access to basic information to assist in investigations of criminal activity and other police duties in serving the public (i.e. suicide prevention, notifying next of kin, etc.)

This, sadly, is a misleading statement. As Michael Geist notes in his blog post today “The mandatory disclosure of subscriber information without a warrant has been the hot button issue in Bill C-30, yet it too is subject to unknown regulations. These regulations include the time or deadline for providing the subscriber information (Bill C-30 does not set a time limit)…”

In other words, for the police to say the bill will get timely access to basic information – particularly timely enough to prevent a suicide, which would have to be virtually real time access – is flat out wrong. The bill makes no such promise.

Moreover, this underlying justification is itself fairly ridiculous while the opportunities for abuse are not trivial. It is interesting that none of the examples have anything to do with preventing crime. Suicides are tragic, but do not pose a risk to society. And speedily notifying next of kin is hardly such an urgent issue that it justifies warrantless access to Canadians private information. These examples speak volumes about the strength of their case.

Finally, it is worth noting that while the Police (and the Minister) refer to this as “basic” information, the judiciary disagrees. Earlier this month the Saskatchewan Court of Appeal concluded in R v Trapp, 2011 SKCA 143 that an individual has a reasonable expectation of privacy in the IP address assigned to him or her by an internet service provider, a point which appeared not to have been considered previously by an appellate court in Canada

The global internet, cellular phones and social media have all been widely adopted and enjoyed by Canadians, young and old. Many of us have been affected by computer viruses, spam and increasingly, bank or credit card fraud.

This is just ridiculous and is designed to do nothing more than play on Canadians fears. I mean spam? Really? Google Mail has virtually eliminated spam for its users. No government surveillance infrastructure was required. Moreover, it is very, very hard to see how the surveillance bill will help with any of the problems cited about – viruses, spam or bank fraud.

Okay skipping ahead (again you can read the full press release here)

2. Secondly, the matter of basic subscriber information is particularly sensitive.

The information which companies would be compelled to release would be: name, address, phone number, email address, internet protocol address, and the name of the service provider to police who are in the lawful execution of their duties.

Actually to claim that these are going to police who are in the lawful execution of their duties is also misleading. This data would be made available to police who, at best, believe they are in the lawful execution of their duties. This is precisely why we have warrants so that an independent judiciary can assess whether or not the police are actually engaged in the lawful execution of their duties. Strip away that check and there will be abuses. Indeed, the Sun Newspaper phone hacking scandal in the UK serves as a perfect example of the type of abuse that is possible. In this case police officers were able to access “under extraordinary circumstances” without a warrant or oversight, the names and phone numbers of people whose phones they wanted to, or had already, hacked.

While this information is important to police in all types of investigations, it can be critical in cases where it is urgent that police locate a caller or originator of information that reasonably causes the police to suspect that someone’s safety is at risk.

Without this information the police may not be able to quickly locate and help the person who was in trouble or being victimized.

An example would be a message over the internet indicating someone was contemplating suicide where all we had was an email address.

Again, see above. The bill does not stipulate any timelines around sharing this data. This statement is designed to lead readers to believe readers that the bill will grant necessary and instant access so that a situation could be defused in the moment. The bill does nothing of the sort.

Currently, there is no audited process for law enforcement to gain access to basic subscriber information. In some cases, internet service providers (ISPs) provide the information to police voluntarily — others will not, or often there are lengthy delays. The problem is that there is no consistency in providing this information to police nationally.

This, thankfully is a sensible statement.

3. Lastly, and one of the most important things to remember, this bill does NOT allow the police to monitor emails, phone calls or internet surfing at will without a warrant, as has been implied or explicitly stated.

There is no doubt that those who are against the legislation want you to believe that it does. I have read the Bill and I cannot find that anywhere in it. There are no changes in this area from the current legislation.

This is the worst part of the press release as it is definitely not true. See Michael Geist’s – the Ottawa professor most on top of this story – blog post from yesterday, which was written before this press release went out. According to Geist, there is a provision in the law that “…opens the door to police approaching ISPs and asking them to retain data on specified subscribers or to turn over any subscriber information – including emails or web surfing activities – without a warrant. ISPs can refuse, but this provision is designed to remove any legal concerns the ISP might have in doing so, since it grants full criminal and civil immunity for the disclosures.” In other words the Police can conduct warantless surveillance. It just requires the permission of the ISPs. This flat out contradicts the press release.

Transparency isn't a cost – it's a cost saver (a note for Governments and Drummond)

1 Reply

Yesterday Don Drummond – a leading economist hired by the Ontario government to review how the province delivers services in the face of declining economic growth and rising deficits – published his report.

There is much to commend, it lays out stark truths that frankly, many citizens already know, but that government was too afraid to say aloud. It is a report that, frankly, I think many provincial and state governments may look at with great interest since the challenges faced by Ontario are faced by governments across North America (and Europe).

From an IT perspective – particular one where I believe open innovation could play a powerfully transformative role – I found the report lacking. I say this with enormous trepidation, as I believe Drummond to be a man of supreme intellect, but my sense is he (and/or his team) have profoundly misunderstand government transparency and why it should be relevant. In Chapter 16 (no I have not yet read all 700 pages) a few pieces come together to create, what I believe, are problematic conditions. The first relates to the framing around “accountability”:

Accountability is an essential aspect of government operations, but we often treat that goal as an absolute good. Taxpayers expect excellent public-sector management as well as open and transparent procurement practices. However, an exclusive focus on rigorous financial reporting and compliance as the measure of successful management requires significant investments of time, energy and resources. At some point, this investment is subject to diminishing returns.

Remember the context. This section largely deals with how government services – and in particular the IT aspects of these services – could be consolidated (a process that rarely yields the breadth of savings people believe it will). Through this lens the interesting things about the word “accountability” in this section above is that I could replace it with searchability – the capacity to locate pieces of information. I agree with Drummond that there is a granularity around recording items – say tracking every receipt versus offering per diems – that creates unnecessary costs. Nor to I believe we should pay unlimited costs for transparency – just for the sake of transparency. But I do believe that government needs a much, much stronger capacity to search and locate pieces of information. Indeed, I think that capacity, the ability for government to mine its own data intelligently, will be critical. Transparency thus becomes one of the few metrics citizens have into not only how effective a government’s inputs are, but how effective its systems are.

Case in point. If you required every Canadian under the age of 30 to conduct an ATIP request tomorrow, I predict that you’d have a massive collapse in Canadians confidence in government. The length of ATIP requests (and the fact that in many places, they aren’t even online) probably says less about government secrecy to these Canadians than it does about the government’s capacity to locate, identify and process its own data and information. When you can’t get information to me in a timely manner, it strongly suggests that managers may not be able to get timely information either.

If Ontario’s public service is going to be transformed – especially if it is going to fulfill other Drummond report recommendations, such as:

Further steps should be taken to advance partnering with municipal and federal services —efficiencies can be found by working collaboratively with other levels of government. For example, ServiceOntario in Ottawa co-locates with the City of Ottawa and Service Canada to provide services from one location, therefore improving the client experience. Additionally, the new BizPal account (which allows Ontario businesses to manage multiple government requirements from a single account) allows 127 Ontario municipalities (such as Kingston, Timmins, Brampton and Sudbury) to partner with ServiceOntario and become more efficient in issuing business permits and licensing. The creation of more such hubs, with their critical mass, would make it easier to provide services in both official languages. Such synergies in service delivery will improve customer experience and capitalize on economies of scale.

Then it is going to require systems that can be easily queried as well as interface with other systems quickly. Architecting systems in open standards, that can be easily searched and recoded, will be essential. This is particularly true if the recommendation that private sector partners (who love proprietary data models, standards and systems which regularly trap governments in expensive traps) are to be used more frequently. All this is to say, we shouldn’t to transparency for transparencies sake. We should do transparency because it will make Ontario more interoperable, will lower costs, and will enable more accountability.

Accountability doesn’t have to be a cost drive. Quite the opposite, transparency should and can be the bi-product of good procurement strategies, interoperable architecture choices and effective processes.

Let’s not pit transparency against cost savings. Very often, it’s a false dichotomy.

Data Wars: A mini-case study of Southwest Airlines vs. TripIt and Orbitz

1 Reply

As a regular flyer, I’m an enormous fan of TripIt. It’s a simple service in which you forward almost any reservation – airline, hotel, car rental, etc… to plans@tripit.com and their service will scan it, grab the relevant data, and create a calendar of events for you. While it’s a blessing not to have to manually enter my travel plans into my calendar, what’s particularly fantastic is that I give my partner access to the calendar – so she knows when I’m flying out and when I return. With 135,000 miles of travel last year alone, there was a lot of that.

TripIt Pro users, however, have added benefits: they can use TripIt to track how many loyalty points they are gathering. That is, unless you travel on Southwest Airlines. Apparently Southwest sent a legal warning to any company that tracks their members’ loyalty benefits and ordered them to stop doing so. (Award Wallet is another example of an app I use that was affected). In a similar vein, veteran travelers know that Southwest does not appear on many travel search sites like Orbitz.

These are great examples of a data wars – places where a company are fighting over who gets access to customers data. In this case Southwest is using its user license to forbids another company from displaying data Southwest generates, but that its customers might wish to share with others because it is helpful to them. It’s not just that Southwest wants to control its relationship with its customers when it comes to loyalty points, or that it wants to sell them hotels and rental cars though its site. It’s that it wants the data about how you behave, about what choices you make and how you make them. Use another site to access loyalty points and they can’t track or sell to you. Ditto if you use another site to buy airfare for their flights.

Southwest isn’t nuts. But it’s a strategy that won’t work for all companies (and may not work for them) and it has real consequences.

To begin with, they are making it hard for their customers to engage their service. When traveling in the US, I regularly use Kayak and/or other types of airline aggregators – it means I never see Southwest as an option. Nor do I go to their website. The bigger irony of course is that while I frequently find fares on aggregator sites, I often book them on the airline’s site. But again, I don’t go to Southwest because they never appear in any searches I do. Maybe they don’t care about business travelers, but they are making a big trade-off – they get more data about their users and have unique opportunities to sell to them, but I suspect they get far fewer users.

In addition, they may be alienating their customers. I’m not so sure customers will feel like loyalty point data belongs to Southwest. After all, it was their dollars and flying that paid to create the data… why shouldn’t they be able to access a copy of it via an application they find useful?

This was all confirmed by an email from a friend and colleague Gary R., who recently wrote me to say:

While we love Southwest Airlines for its low prices, generous affinity programs and flexibility in changing business trips at the last moment with little consequence, their closed data sharing policy drives up our overall cost of managing travel. Entering flight information manually into TripIt is a pain, yet the service is incredible at keeping one informed during a trip, presenting a palette of options seemingly the instant things go wrong. We have chosen other carriers over Southwest on occasion because they play nicely with Orbitz and TripIt.

I can’t tell if Southwest’s tradeoffs are worth it or not. But any business person must at least recognize there is a tradeoff. That’s the real lesson. You need to find a way to value the data you collect and be able to compare it against the opportunity of a) happier clients and b) potentially accessing more clients. This is particularly true since many customers probably (and rightly) will feel that is data is as much theirs as it is yours. They did co-create it.

Ultimately if you increase the transaction costs of the experience – because you want to shut other actors out – you will lose customers. Southwest already has.

Definitely expect more of these types of legal battles in the future. Your data is now as important as the service you use. This makes it both powerful, and dangerous in the hands of the wrong people.

Two Reasons I love blogging: Helping out great communities

eaves.ca

if writing is a muscle, this is my gym

Category Archives: commentary

Access to Information, Open Data and the Problem with Convergence

Calculating the Value of Canada’s Open Data Portal: A Mini-Case Study

3. Visitor locations

Attack of the Drones – How Surveillance May Change our Culture

Let's Hack data.gc.ca

The Ground War for Control of Public Space

Media Watch: The Globe and Mail’s Shifting Headline

The Surveillance State – No Warrant Required

Transparency isn't a cost – it's a cost saver (a note for Governments and Drummond)

Data Wars: A mini-case study of Southwest Airlines vs. TripIt and Orbitz

Two Reasons I love blogging: Helping out great communities