I’ve been observing the Globe Policy Wiki with enormous interest. I’m broadly supportive of all of Mathew Ingram’s experiments and efforts to modernize the Globe. That said, my sense is that this project project faces a number of significant challenges. Some from the technology, others around how it is managed. Understanding and cataloging the ups and downs of such this effort is essential. At some point (I suspect in the not too distant future) wikis will make their way into the government’s policy development process – the more we understand the conditions under which they flourish, the more likely such experiments will be undertaken successfully.
Here are some lessons I’ve taken away:
1. The problem of purpose: accuracy vs. effectiveness
Wiki’s are clearly effective at spreading concrete, specific knowledge. Software manuals, best practice lists and Wikipedia works because – more often than not – they seek to share a concrete, objective truth. Indeed “the goal” of Wikipedia is to strive for verifiable accuracy. Consequently, a Wikipedia article on Mohandas Karamchand Gandhi can identify that he was born on October 2nd, 1869. We can argue whether or not this is true, but he was born on a specific day, and people will eventually align around the most accurate answer. Same with a software wiki – a software bug is caused by a specific, verifiable, set of circumstances. Indeed, because the article is an assemblage of facts its contributors have an easier time pruning or adding to the article.
Indeed, it is where there is debate and subjective interpretation that things become more complicated in Wikipedia. Did George Bush authorize torture? I’ll bet Wikipedia has hosted a heady debate on the subject that, as yet, remains unresolved.
A policy wiki however, lives in this complicated space. This is because the goal of a policy wiki is not accuracy. Policies are are not an assemblage of facts whose accuracy can be debated. A policy is a proposal – an argument – informed by a combination of assumptions, evidence and values. How does one edit an argument? If we share different values, what do I edit? If I have contradictory evidence, what do I change? Can or should one edit a proposal they simply don’t agree with? In Wikipedia or in online software manual the response would be: “Does it make the piece more accurate?” If the answer is yes, the you should.
But with is the parallel guiding criteria for a policy wiki? “Does it make the policy more effective?” Such a question is significantly more open to interpretation. Who defines effective?
It may be that for a policy wiki will only work within communities that share a common goal, or that at least have a common metric for assessing effectiveness. More likely, Wikis in areas such as public policy may require an owner who ultimately acts as an arbiter deciding which edits stand and which edits will get deleted.
2. Combining voting with editing is problematic.
The goal of having people edit and improve a policy proposal runs counter to those of having them vote on a proposal. A wiki is, by definition, dynamic. Voting – or any preference system – implies that what is being voted on is static and unchanging; a final product that different people can assess. How can a user vote in favour of something if, the next day, if it can be changed into something I may disagree with it? By allowing simultaneously for voting and editing I suspect the wiki discourages both. Voters are unsure if what they are voting for will stay the same, editors were likely wary of changing anything too radically because the voting option suggests proposals shouldn’t change too much – undermining the benefits of the wiki platform.
3. While problematic for editing, the Policy Wiki could be a great way to catalog briefs
One thing that is interesting about the wiki is that anyone can post their ideas. If the primary purpose were to create a catalogue of ideas the policy wiki could be a great success. Indeed, given that people are discouraged from radically altering policy notes this is effectively what the Policy Wiki is (is it still a wiki?). Presently the main obstacle to this success is the layout. The policy briefs currently appear in a linear order based on when they were submitted. This means a reader must scroll through them one by one. There is no categorization or filtering mechanism to help readers find policies they specifically care about. A search feature would enable readers to find briefs with key words. Also, enabling users to “tag” briefs would allow readers to filter the briefs in more useful ways. One could, for example, ask to see briefs tagged “environment,” or “defense” taking you to the content you want to see faster.
Such filtering approaches might distribute readers more accurately based on their interests. In a recent blog post Ingram notes that the Flat Tax briefing note received the most page views. But this should hardly come as a surprise (and probably should not be interpreted as latent interest in a flat tax). The flat tax brief was the first brief on the list. Consequently, casual observers showing up on the site to see what it was all about were probably just clicking on the first brief to get a taste.
eaves.ca 
Your observations about the differences between wikis for policy and software documentation were good one. Wikis are great for many things on software projects: documentation, standards and policies, meeting minutes and status reports. They're not good for defect (bug) management, however.Instead, I think policy discussion shares many things in common with the negotiation that accompanies discussion related to the features to be incorporated into subsequent releases. Joel Spolsky has an insightful piece about this on his site: http://www.joelonsoftware.com/articles/SetYourP…I wonder if policy negotiation could (or already does?) incorporate similar strategies. I'm certain that your friend Mike Beltzner could talk about ways that might work.Other ideas could be staring us right in the face (assuming we're looking at eaves.ca). What about the DISQUS system. It incorporates similarities to both Slashdot and Digg. Conversations are threaded based on an idea (a policy brief?) and then voted up or down by the community.
I'm as excited as much as you are about the potential of the G&M Wiki.However there's two things I might comment on in your analysis.Firstly, to channel Clay Shirky, one must be careful to disentangle how a software tool can be used from how a software tool is being used. Wiki's, from a functional perspective, are extraordinarily primitive (they're essentially a carefully broken publishing tool). Wikipedia's culture is a question of internal policy, not a constraint of the tools. Have a look at the other WikiMedia projects, or even other projects implementing MediaWiki – including some that document formalized debates.The strength of the wiki is to expose the underlying reasoning behind policy – which is often opaque and consequently assumed in other contexts. When the foundational syllogisms and ontological presumptions for a policy are laid bare it becomes factual discourse rather entrenched “arguments of principle”.Policy discussions, I would imagine are as often centered around differences or confusions about factual information and its interpretation as they are around notional “values”.Wikis and other tools will go a long way to expose blind knee-jerk ideological hackery, that ultimately will be their biggest value.
An excellent post, David. I appreciate your constructive criticism — in part because you successfully described many of my own mixed feelings about the project. When we started the wiki, there was a lot of discussion about whether a wiki was the correct tool for what we wanted, and whether something like a forum wouldn't be better, or a simple list with voting features.I pushed for the wiki for a couple of reasons: one was that I simply like how easy wiki tools make it to edit a site — not just for participants or contributors, but for me and the other moderators. It just makes it really easy to manage, and without a huge amount of manpower or time, that was a pretty important consideration. But I also thought that a wiki was the right tool for many people as well, perhaps not for editing but at least for easy creation of briefing notes.We (or rather I) decided to add voting and comments and forums — none of which are really true wiki style, as I've been reminded repeatedly by many people — because I assumed (rightly I think) that very few people would be prepared to take the time to actually edit or create briefing notes, or to learn how to use the wiki tools. I wanted to make the site as inclusive as possible, and so I thought we should have tiers of access for different interest levels: if you're in a hurry, you can just vote; if you just want to comment, that's fine too. Perhaps it cuts down on the actual editing of the notes, but I think that's an acceptable risk.I completely agree with many of your other points — the numbering system, the way the notes are just listed on a page and aren't categorized or searchable. In true wiki fashion, of course, there are several people who feel the same and have actually started re-structuring the wiki to make that happen :-) It is first and foremost an experiment, and we are (hopefully) learning from it. In any case, I really appreciate your thoughtful input. I'd be happy to talk about it more sometime.
I followed the Globe's policy wiki with interest over these past couple weeks, as I think that Canada could do a much better job of engaging the population in political discourse through the use of technology. I would like to add a couple observations about the particulars of the Globe's wiki experiment that may have discouraged its use.Firstly, potential contributors to the wiki could rightfully question the value of their contribution because the long-term future of the wiki is uncertain. It is not clear that the Globe has made any commitment to publicly host the wiki for a defined period of time. In contrast, editing Wikipedia is attractive because of the potential for permanence: wikis are fluid and changeable, yes, but Wikipedia's mission also includes hosting articles until the end of time. A user who puts in a great deal of effort on a Wikipedia article can be reasonably assured that his work will still be there in three years, though others may have built upon it. On the Globe's wiki, users are not reasonably assured that their work will still be publicly available in 3 weeks.A second concern (and this poses a difficult problem) is that the results of the wiki don't make it out to the Globe's main site except as filtered through Mr. Ingram's blog posts and articles. Thus, the author of one of the 30 briefs (especially one of the briefs created near the end of the experiment) could not have the expectation of communicating directly to the Globe's audience… they must merely hope that Mr. Ingram, as the gatekeeper, sees fit to mention their efforts. Of course, I feel that Mr. Ingram did an excellent and fair-minded job in this regard; however, the uncertainty behind relying on a gatekeeper to summarize the efforts on the wiki also serves to discourage serious effort by potential contributors.
Thanks for that, Jared — those are both excellent points. I am doing my best to strengthen the link between the Wiki and the paper itself (Web and/or print version) — we are still in the early days of this experiment. Baby steps :-)
Just want to note that these are precisely the types of discussions I hope this blog can generate. Always great when the comments are better than the original post. Thanks everyone – especially Mathew.