Lawful, Neutral and Chaotic: mapping the drivers behind social media, companies and states in the public sphere

My own view of this space was initially shaped in 2004, after I first watched Laurence Lessig’s Free Culture OSCON talk. I became more conscious of the earlier debates (which continue today) around freedom and the online sphere.

Back then, I took comfort knowing we had a map of the terrain; we knew the players and their power. On the one side sat organizations like Mozilla and the leverage provided it by Firefox. On the other was Microsoft with its platform, marketing budget and customer base. As it turns out, a small, and effectively self-organized subset of the public (Mozilla & others) was able to tackle a de facto monopoly (Microsoft) and win.

Snowden (and Evgeny Morozov had much to say on this before Snowden) crystallized in the public’s mind that the real challenge to freedom on the internet is not companies, but states. China and Russia (and even the UK) offer a view into a dystopian world of persistent surveillance, while the NSA’s activities (and the US government’s response to his leaks) have destroyed people’s confidence in it as a model — particularly for non-Americans using US-based online services. In my mind, the biggest consequence of Snowden is that the United States functionally legitimized mass surveillance. While this prompted some governments to (publicly) recoil in horror, I suspect many more said “I’d like that too, thank you.” The post-Snowden era is defined by the state’s overt efforts to reassert control over the online space, particularly via company proxies. In the battle of the internet versus the state, today states want to win, and are comfortable saying as much publicly.

Probably not the right breakdown…

Given all this, for those of us who believe the internet should be a place where people can work, debate, shop and live without the threat of persistent surveillance, it is easy to assume certain actors are inherently good or evil. Governments must be evil — and the online public sphere, those organizing via social media must be good.

But nothing is ever that simple. And the good/neutral/evil moniker is almost never a fixed thing for a group of actors. So what is a more helpful frame?

No map is perfect, but as a simple tool to tease these differences out, I’ve enjoyed leveraging a current internet meme about good, neutral and evil. There are an endless number of these (the computer geek and The Office examples are particularly fun), but as I enjoy The Princess Bride, here’s an example:

Recreating a chart like this in the online space, I see countries as (generally) concerned with control, companies with money, and social media as chaotic. This description, albeit still imperfect, is at least accurate around what I believe to be the fixed variable:

Lawful Neutral and ChaoticThis in turn helps provide a frame for a conversation around how these actors behave. Take social media (a nebulous actor at best, I know). Several years back, Ethan Zuckerman, while talking about his then upcoming book, described the online social media environment as a giant unpredictable ball which people try to push or temporarily sway (at the time the Stop Kony movement was an example). I see him now as describing the public sphere, with its diverse interests, opinions, goals and ideas constantly clashing and in flux, as inherently chaotic. While this was true of an old bazaar, one element that make the online public sphere particularly chaotic is its ability to scale up in ways that were previously more difficult, and at a much faster pace.

However, there is nothing about the discussions in the public sphere — in person or online—that are inherently good. A public sphere conversation can be as much about promoting racism or doxxing activists as it can be about ending tyranny.

So here are some illustrative examples of where some actors are currently positioned in relation to some of the policy debates I think about.

Screen Shot 2016-05-04 at 12.39.18 PMSome observations

First, I’m sure people will find reasons to disagree with the choices above — although the X axis is relatively fixed in my eyes, the Y axis is both more fluid and subjective. My goal is less about accuracy than to help provide a framework for thinking about this space and prompting conversation. If it causes you to debate where actors fall, or maybe where your own employer or country falls, great.

Second, it also outlines the challenge around some policy debates. In 2005 the focus of those who cared for freedom, privacy and self-expression on the internet focused (more) on companies. Post-Snowden, companies matter, but the real challenge is the state. There are legitimate reasons why a state would want to surveil a specific target: child pornography, terrorism, etc… what I think people fear about is the capacity for institutionalized mass-surveillance.

And by legitimizing such mass-surveillance, the US has prompted more actors in the “state/lawful” column to move downward towards (what I would define as) evil. More critically, these state actors are asserting significant pressure on the companies (the neutral column) to move downward to help them fulfill that objective.

The current strategy, as I see it, is to grow the “chaotic good” group as a way to try to first counterbalance the state and its efforts to coerce companies, but to ultimately shift the state upward to at least a neutral and ideally a “good” place. There is a world of lawful, limited surveillance. The question is how are we going to ensure it is created.

Third, your allies are probably not other actors in your column — so people who look, organize or act like you. Rather your allies are people in your row. So how do you connect, empower, leverage and enable them?

Fourth, one thing I like about the above is that social media and self-organization genuinely is chaotic. It is also often far better at protesting and “tearing something down” than it is at building something. Stopping a proposed law or seeking to destabilize a government is something social media has been effective at. Creating an alternative law, or forming a coherent government in waiting, much less so.

Finally, I am deeply conscious of several actors who aren’t listed and not sure where I would put them. EFF (lawful good?), Mozilla (Chaotic good?), W3C (true neutral — although the DRM stuff…?), so none of this is perfect.

Again, I’m playing around with this to try to build tools that simplify some complex thinking around players, organize the online sphere, and make it easier for people to access the conversation. Feedback, public or private, is always great.

Open Mandate Letters

The newly elected Government of Canada made its ministerial mandate letters available to the public last week. They are absolutely worth checking out both for their content and as a example of public disclosure/communication. I’ll talk about that latter part in a second, but let me first let’s discuss some background information and context.

From a content perspective the mandate letters are filled with a number of ambitious goals. Tzeporah Berman outlines a range of initiatives Ministers have been tasked to attend to from an environmental perspective (this may require access to Facebook). There are other goals to be excited by beyond the environment – the Justice Minister’s mandate letter in particular is worth looking at in detail. Again, it is an ambitious and exciting set of goals.

Some of you may ask: What IS a ministerial mandate letter? In Canada’s parliamentary system they have been how the Prime Minister and the Premier formally articulate the goals, priorities, specific task as well as general tone, for their ministers. In other words, they’re a way for the leader to tell cabinet ministers what he or she expects them to do (and presumably, how they will evaluate their performance).

Interestingly, while this is the first time I’m aware of that the federal government has made mandate letters open, it is actually part of an emerging trend. I believe this first happened in British Columbia under the current Premier (Premier Clark). They also formed part of the recommendations my colleagues and I proposed as part of the Open By Default report I helped draft at the request of Premier Wynn of Ontario. Premier Wynn, after winning the subsequent election, made her ministerial mandate letters public as well.

What I find fascinating is that mandate letters have not been made public before. Making them public has a number of advantages, both to the government, but also for more effective governance in general.

Making them public should help focus the government. It makes everybody, from the Minister, their advisors, down to every employee in the department that minister oversees aware of the minister’s goals. Such clarity is likely quite helpful and probably the most compelling reason for making them open. Governments are super tankers. The more the crew knows which way the ship is supposed to be going, the more individual decisions can be made to help ensure it’s course is accurate. And of course, any crew members that were planning on trying to push a ship in the direction they wanted to go, were likely to do so with or without seeing the mandate letters. At least now they can’t plead ignorance.

It also effectively communicates the government’s goals to the public. This makes it easier for actors outside of government – individual voters, NGO’s, industry groups and others – to better see where their priorities fit into the governments priorities. Some will be happy, others will be frustrated, but at least they know where they stand in a specific tangible and reference-able document.

Finally, mandate letters matters from a governance perspective. Mandate letters that veer far from campaign promises or from the political mandate the government received from voters will make for good fodder by opposition leaders. In addition, Ministers ability to execute against the goals and tasks laid out in their mandate letters are more visible to the public and, of course, the opposition. This form of accountability is likely to be helpful, spurring ministers to be more effective and on task while also granting opposition parties greater ability to point out problems.

Is it possible that a Minister could receive an additional “secret” mandate letter? Absolutely. But having mandate letters be completely secret strikes me as little better. And if a minister is working to complete some of their “secret” tasks, at least the public and opposition can hammer away at the minister asking them why they are not working on completing their stated “public” goals as outlined in the public mandate letter.

Moving to Harvard

Hi friends.

Just a brief note to say that I’ve been invited to come to the Kennedy School of Government to be a Research Fellow in the Science, Technology and Public Policy Program (STPP) at the Belfer Center for Science and International Affairs at the Kennedy School of Government at Harvard University.  I’ve also been invited to be an Adjunct Lecturer in Public Policy at the Kennedy School and to teach on Technology, Policy and Government.

A number of other changes flow out of this news!

  1. I’ll be moving to be Boston in the New Year. Looking forward to reconnecting with old friends from a previous life there, and making new friends. There’s a wonderful community of people there that includes the likes of Nigel Jacobs, Debbie Chachra, Nick Sinai, Susan Crawford, Nick Grossman, Colin Maclay, Mitchell Weiss and many others that I hope I get to see more of.
  2. As I’ll be teaching, please send me any ideas, cases, readings you think I should include as course materials. I’m working on my initial course – an introduction to technology and government – which will focus on how technology can and is changing the ways we deliver services, organize government and make policy. Opportunities and challenges around user-centric design, collaboration vs. cooperation, influence and power, and open data/methodologies will all figure prominently. It is also about bringing what I tried to impart on fellows during the boot camps at Code for America and the Presidential Innovation Fellows Program.
  3. Finally, with my children (only somewhat, but nonetheless…) a little older and these new academic responsibilities, I hope to write more again. Or maybe, more precisely, I hope to write more here again. This blog enables me to organize and structure thoughts (and, on occasion, is a place to get things off my chest). With luck it, along with the classroom, will be that again. While I’ve been silent and busy, the last few years have been an incredible time of learning – particularly in the civic startup space, but also vis-a-vis technology and government more generally – and I look forward to sharing more of what I’ve gleaned.

Leaving Vancouver is hard; I’ve got wonderful roots, family and friends here, and a community and city I care about enormously. But I’m also excited about engaging with colleagues and students at the Kennedy School.

Looking forward to it all.

Government Procurement Failure: BC Ministry of Education Case Study

Apologies for the lack of posts. I’ve been in business mode – both helping a number of organizations I’m proud of and working on my own business.

For those interested in a frightening tale of inept procurement, poor judgement and downright dirty tactics when it comes to software procurement and government, there is a wonderfully sad and disturbing case study emerging in British Columbia that shows the lengths a government is willing to go to shut out open source alternatives and ensure that large, expensive suppliers win the day.

The story revolves around a pickle that the province of British Columbia found itself in after a previous procurement disaster. The province had bought a student record management system – software that records elementary and secondary students’ grades and other records. Sadly, the system never worked well. For example, student records generally all get entered at the end of the term, so any system must be prepared to manage significant episodic spikes in usage. The original British Columbia Electronic Student Information System (BCeSIS) was not up to the task and frequently crashed and/or locked out teachers.

To make matters worse, after spending $86M over 6 years it was ultimately determined that BCeSIS was unrecoverably flawed and, as the vendor was ending support, a new system needed to be created.

Interestingly, one of the Province’s school districts – the District of Saanich – decided it would self-fund an open source project to create an alternative to BCeSIS. Called OpenStudent, the system would have an open source license, would be created using locally paid open source developers, could be implemented in a decentralized way but still meet the requirements of the province and… would cost a fraction of that proposed by large government vendors.  The Times Colonist has a simple article that covers the launch of OpenStudent here.

Rather than engage Saanich, the province decided to take another swing at hiring a multinational to engage in a IT mega-project. An RFP was issued to which only companies with $100M in sales could apply. Fujitsu was awarded a 12 year contract with costs of up to $9.4M a year.

And here are the kickers:

So in other words, the province sprung some surprise requirements on the District of Saanich that forced it to kill an open source solution that could have saved tax payers millions and employed British Columbians, all while exempting a multinational from meeting the same requirements. It would appear that the province was essentially engaged in a strategy to kill OpenStudent, likely because any success it enjoyed would have created an ongoing PR challenge for the province and threatened its ongoing contract with Fujitsu.

While I don’t believe that any BC government official personally profited from this outcome, it is hard – very hard indeed – not to feel like the procurement system is deeply suspect or, at worst, corrupted. I have no idea if it is possible, but I do hope that these documents can serve as the basis for legal action by the District of Saanich against the Province of British Columbia to recapture some of their lost expenses. The province has clearly used its purchasing power to alter the marketplace and destroy competitors; whether this is in violation of a law, I don’t know. I do know, however, that it is in violation of good governance, effective procurement and general ethics. As a result, all BC tax payers have suffered.

Addendum: It has been suggested to me that that one reason the BC government may be so keen to support Fujitsu and destroy competing suppliers is because it needs to generate a certain amount of business for the company in order for it to maintain headcount in the province. Had OpenStudent proved viable and cheaper (it was estimated to cost $7-10 per student versus $20 for Fujistu’s service), Fujistu might have threatened to scale back operations which might have hurt service levels for other contracts. Unclear to me if this is true or not. To be clear I don’t hold Fujistu responsible for anything here – they are just a company trying to sell their product and offer the best service they can. The disaster described above has nothing to do with them (they may or may not offer amazing products, I don’t know); rather, it has everything to do with the province using its power to eliminate competition and choice.

Canada’s Opaque Transparency – An Open Data Failure

Yesterday, at the Prospectors and Developers Association of Canada’s (PDAC) Canada Minister of Natural Resource, Joe Oliver, announced with great fanfare a new initiative to compel mining companies to disclose payments of over $100,000’s to foreign and domestic governments.

On the surface this looks like a win for transparency, particularly for a sector that is of great importance to Canada: mining.

And this issue matters since not only do extractive industries represent an important part of Canada’s economy, but the sector has been dogged with controversy. Indeed the Toronto Star just uncovered today a report commissioned (and buried) by the Prospectors and Developers Association of Canada (PDAC) that showed Canadian mining companies have the worst record when it comes to environmental standard and human rights.

Forcing mining companies to account for their payments to foreign and domestic governments won’t solve every problem, but it can help curb corruption. Indeed the issue was seen as so important that at the last G8 summit, the leaders agreed that companies should be compelled to disclose these payments.

Happily, there is a legitimate global movement to make government payments by extractive industry companies more transparent. It is called the Extractive Industries Transparency Iniative (EITI). It has set a series of standards for disclosing such payments so that they are easier to track across borders. In fact EITI is seen as so important it is actually the only organization mentioned by name in the last G8 summit communique. This is the same EITI program about which last year the Minister’s press secretary boasted:

Since 2007 Canada has also been a supporting country of the Extractive Industries Transparency Initiative, and is now the second largest financial donor to the initiative, providing $12.65 million to the World Bank’s Extractive Industries Transparency Initiative Multi-donor Trust Fund…

Which brings us to Minister Oliver’s important announcement.

Did the government announce that it was joining 42 other countries, including its G8 partners the United States, the United Kingdom, France, Germany and Italy to join the standard it has been a major funder of?

No. It did not.

Apparently EITI is good enough to fund so that others can implement it. When it comes to actually doing what is effective… the government balked. Canada, apparently, is going to adhere to its own “unique” approach.

And it gets worse.

Read the Minister’s statement more closely, particularly this line:

“We want to make it as easy as possible, so we will not create a central database. Instead, we would require that reports be posted to company websites, with the government and public notified.”

So unlike EITI, which offers a centralized repository where records can quickly be downloaded and compared, Canada’s “compliance” will involve each company to maintain their own records “somewhere” and will require anyone interested if actually figuring out what is going on to go and track down each one individually.

We call this secrecy by obscurity. It makes a mockery of the notion of transparency.

We have a global infrastructure designed to make disclosure cheap, easy and effective. Infrastructure our own government has poured $12.7M into it. And we turn around and ignore it all.

Canada claims it wants to be a leader in open data. But if it can’t even get something basic like this right… such claims sounds increasingly silly here at home, among our G8 partners and, well, among the rest of the world.

Addendum: It gets worse still. Few people have noticed yet, but Canada recently (and quietly) stopped reporting the names of corporate directors in the public database of the country’s firms. This is a major step backwards and makes those who benefit from one of the most important benefits society can confer – limited liability – invisible to the public who confers that right. This is a major step backwards. Read this wonderful Economist article on why. More on this to come.

Great Hacks from the Open Data in Vancouver

Last weekend I helped host an Open Data Day in Vancouver. With the generous support of Domain7, who gave us a place to host talks and hack, over 30 Vancouverites braved the sleet and snow to spend the day sharing ideas and working on projects.

We had opening comments from Andy Yan – whose may be the most prolific user of Open Data in Vancouver, possibly Canada. I encourage you to check out his work here. We were also incredibly lucky to have Jeni Tennison – the Technical Director of the Open Data Institute – onsite to talk to participants about the ODI.

After the opening talks, people simply shared what they hoped to work on and people just found projects to contribute to. Minimal organization was involved… and here a taste of the awesome projects that got worked on! Lots of ideas here for other communities.

1. Open Data Licenses Resource: JSON + search + compatibility check = Awesome.

Kent Mewhort, who recently moved to Vancouver from Ottawa (via the Congo) updated his ongoing CLIPol project by adding some of the recently published licenses. If you’ve not seen CLIPol it is… awesome. It allows you to easily understand and compare the restrictions and rights of many open government licenses.

CLIPol Data

Better still CLIPol also lets you to see how compatible a license is (see example here). Possibly the best tool of all is one that allows you to determine what license you can apply to your re-mixed work in a way that is compliant with the original licenses (check out that tool here – screenshot below).

CLIPol compatibility

CLIPol is just such a fantastic tool – can’t recommend it enough and encourage people to add more licenses to it.

2. Vancouver in MineCraft

I have previously written about how Minecraft is being used to help in public consultations and urban planning – I love how the game becomes a simple tool that enables anyone to shape the environment.

So I was crazy excited I heard that Ryan Smith (aka Goldfish) had used the City of Vancouver’s open elevation data to recreate much of the city in Minecraft.

Below is a photo of Ryan presenting at the end of the day. The projection behind him shows Stanley park, near Siwash Rock. The flat feature at the bottom is the sea wall. Indeed Ryan notes that the sea wall makes for one of the clearest features since it creates almost perfectly flat structure along the city’s coast.

Mincraft Data

3. Vancouver’s Capital Budget Visualized in Where Does my Money Go

It is hard to imagine a project going better. I’m going to do a separate blog post on it.

This is a project I’ve always wanted to do – create a bubble tree visualization with Where Does my Money Go. Fortunately two developers – Alexandre Dufournet and Luc Lussier – who had never hacked on open data jumped on the idea. With help from City of Vancouver’s staff who were on site, I found a PDF of the capital budget which we then scraped.


The site is not actually live, but for developers who are interested in seeing this work (hint, hint City of Vancouver staff) you can grab their code from github here.

4. Monitoring Vancouver’s Bike Accident Data – Year 3

Eric Promislow has been coming to Open Data Hack-a-thons ever since Luke Closs and I started organizing them in 2009. During the first Open Data Day in 2011 you can read in my wrap up post about a bike accident monitoring website Eric created that day which Eric would eventual name Bent Frame. Well, Bent Frame has been live ever since and getting bigger. (Eric blogs about it here)

Each open data day, Eric updates Bent Frame with new data from ICBC – the province’s insurance monopoly. With over 6 years of data now in Eric is starting to be able to analyze trends – particularly around the decline of bike accidents along many roads with bike lanes, and an increase in accidents where the bike lanes end.

Bike Data

I initially had conversations with ICBC to persuade them to share their data with Eric and they’ve been in touch with him ever since, passing along the data on a regular basis. It is a real example of how an active citizen can change an organization’s policies around sharing important data that can help inform public policy debates.

5. – Making government information easier to search

Kevin McArthur is the kind of security guy most governments dreads having around but should actually love (example his recent post on e-voting).  He continued to hack on one of his side projects: The site is a sort of front end for open data sets, making it easier to do searches based on people or companies. Thus, want to find all the open data about a specific minister… proactive disclosure organizes it for you.

Proactive Data

Kevin and a small team of players uploaded more data into their site and allowed it to consume unstructured data. Very cool stuff.

6. Better Open Data Search

Herb Lainchbury – another fantastic open data advocate – worked on a project in which he tried to rethink what an open data search engine would look like. This is a topic that I think matters A LOT. There is simply not a lot of good ways to find data that you are interested in.

Herb’s awesome insight was invert the traditional way of thinking about data search. He created a search engine that didn’t search for the data set keywords or titles, but rather searched the meta data exclusively.

One interesting side outcome of this approach is that it made related data sets easier and, made locating identical data sets but from different years a snap. As Herb notes the meta data becomes a sort of “finger print” that makes it easy to see when it has been duplicated. (Quick aside rant: I loath it when governments releases 20 data files of the same data set – say crime data – with each file representing a different year and then claiming that it is 20 unique data sets in their catalogue. No. It is one data set. You just have 20 years of it. Sigh).

7. School Performance Chart

Two local video game programers – Louie Dinh and Raymond Huang – with no experience in open data looked around the BC Government Open Data catalogue and noticed the data on test scores. Since they attended school here in British Columbia they thought it might be interesting to chart the test scores to see how their own schools had preformed over time.

They were able to set up a site which graphed how a number of elementary schools had performed over time by looking at the standardized test scores.

Test SCore Data

This is just a great example of data as a gateway to learning. Here a simple hackathon project become a bridge for two citizens to dive into a area of public policy and learn more about it. No one is claiming that there chart is definitive, rather it is the start of a learning process around what matters and what doesn’t and what can be measured and what can’t in education.

Congratulations to everyone who participated in the day – thank you for making it such an amazing success!