Category Archives: media

How the Media Should have Responded to Peter Thiel

Much ink has been spilled about Peter Thiel’s funding of various cases against Gawker. However, the discussion of whether he should or shouldn’t mostly miss the point. Nor do the responses give me much confidence in the media, who seem focused on playing victim, rather than focused on the incredible power they have.

Let me lay some groundwork, before articulating what I wish someone in the media would do.

First, Peter Thiel had every right to do what he did.

Second, if Gawker is guilty it suggests they did violated an individuals right and should pay. (I suspect they will win on appeal). There is such a thing as harassment and bullying. Being a “journalist” doesn’t give you a pass to print or say anything. Figuring out the balance is a debate we should continue to have in society.

Third, having third parties pay for lawsuits doesn’t strike me as an issue. There are real benefits to this and, it should not affect the material facts of the case and thus the outcome.

Fourth, everybody seems to dump on Gawker. I don’t read them, but can see why many people don’t like them. That said, they are important. I remember them for being the only media company willing to print a story about how Rob Ford, the former Mayor of Toronto, was smoking crack. This was very much in the public interest and it appears the Canadian media was too cautious to print it. There is value in this type of journalism.

Finally, while Peter Thiel had the right to fund these lawsuits, I think he’ll come to regret it. We may look back on May 25th as the moment the broader media began to recognize the powerful people in Silicon Valley are the elites of a new Gilded Age who increasingly shape and influence our lives.

So rather than bemoan and write articles questioning the ethics of Peter Thiel’s actions here’s what I wish the media would say:

“We agree with Peter Thiel, there should be smart, thoughtful and professional reporting. In addition, it is also clear that Silicon Valley is one, if not the, new centre of economic power in the world. Given its prominence in shaping our world and the power its elites have, we, the undersigned editors, are each assigning one of our most veteran news reporters to a news beat which will critically examine the people and companies reshaping the lives of America and the world. We look forward to the debate and conversations this reporting sparks about the direction of our economy, and its impact on our lives and democracy.”

At least the Guardian made the commitment before all this happened.

Some Nice Journalistic Data Visualization – Global’s Crude Awakening

Over at Global, David Skok and his team have created a very nice visualization of the over 28,666 crude oil spills that have happened on Alberta pipelines over the last 37 years (that’s about two a day). Indeed, for good measure they’ve also visualized the additional 31,453 spills of “other” substance carried by Alberta pipeline (saltwater, liquid petroleum, etc..)

They’ve even created a look up feature so you can tackle the data geographically, by name, or by postal code. It is pretty in depth.

Of course, I believe all this data should be open. Sadly, they have to get at it through a complicated Access to Information Request that appears to have consumed a great deal of time and resources and that would probably only be possible by a media organizations with the  dedicated resources (legal and journalistic) and leverage to demand it. Had this data been open there would have still been a great deal of work to parse, understand and visualize it, but it would have helped lower the cost of development.

In fact, if you are curious about how they got the data – and the sad, sad, story it involved – take a look at the fantastic story they wrote about the creation of their oilspill website. This line really stood out for me:

An initial Freedom of Information request – filed June 8, 2012, the day after the Sundre spill – asked Alberta Environment and Sustainable Resource Development for information on all reported spills from the oil and gas industry, from 2006 to 2012.

About a month later, Global News was quoted a fee of over $4,000 for this information. In discussions with the department, it turned out this high fee was because the department was unable to provide the information in an electronic format: Although it maintained a database of spills, the departmental process was to print out individual reports on paper, and to charge the requester for every page.

So the relevant government department has the data in a machine readable form. It just chooses to only give it out in a paper form. Short of simply not releasing the data at all it is hard to imagine a more obstructionist approach to preventing the public from accessing environmental data their tax dollars paid to collect and that is supposed to be in the public interest. You essentially look at thousands of pieces of paper and re-enter tens, if not hundreds of thousands, of data points into spreadsheets. This is a process designed to prevent you from learning anything and frustrating potential users.

Let’s hope that when the time comes for the Global team to update this tool and webpage there will be open data they can download and access to the task is a little easier.

 

The Beneficial Impact of Newspaper Paywalls on Users

There continues to be fierce debate about the cost/benefits of newspaper paywalls, a debate Mathew Ingram has been helping drive with a great deal of depth and with excellent links.

It is interesting to watch Ingram take on, and have to rebut, the problematic thinking that seems to so frequently comes out of the Columbia Journalism Review which, sadly, as America’s most important journal on the industry and trainer of next generation journalist, is probably the most conservative voice in the debate. That said, while its contributions are defensive and disappointing, they are understandable. And well, it makes for fascinating reading of how people deal with an industry in decline/collapse/[insert whatever word you prefer here]. Someone should package these and make it mandatory stuff for MBA types.

Personally, I’m somewhat indifferent. If paywalls can save newspapers (every indication suggests that they cannot) then great! If they can’t… okay. The sky will not fall and the world will not end. We will have to figure out some new models for thinking about how we dispense news and discuss issues. Or, perhaps better still, might rethink what “news” and “media” is and means (psst… already happening).

I don’t want to join the holy war around paywalls, so I’m not trying to add to that debate directly – you should read Ingram and the others yourself.

What I do want to contribute is what I’m experiencing as a consumer. And what I can say is that I’ve found paywalls to be profoundly beneficial to me as an avid and significant news consumer, but in ways that I’m pretty sure the news industry is going to find disturbing and disappointing.

tweets-on-paper

So, what do I mean by that?

Well, in Canada several newspapers have thrown up paywalls – including the Globe and Mail (the main national newspaper) as well as the main “traditional” newspaper in my market – the Vancouver Sun. Each offers something like 20 free articles a month – akin to what the New York Times does. I also assume (but confess I don’t actually know) that if you arrive at an article via facebook or twitter, it does not count towards that total.

Now, every morning when I visit the home page of the Globe and the Sun and I see an article that might be interesting, I hover my mouse over it and ask myself the same question I suspect tens of thousands of others ask in that moment: “Is reading this article worth burning one my of my 20 free articles this month?” (or, framed another way: “is this article, sponge worthy?”)

And you know what? About 90% of the time, more actually, the answer is no.

And you know what else?

I’m grateful that this is the choice I’m being forced to make.

The internet is filled with news and articles that are wonderfully distracting and that I should probably not spend time reading. It turns out that by “internet” I also mean “pieces of news and articles in the pages of the Globe and the Sun.

What pay walls are reminding me of is that time is my most valuable (or scarce) resource, not access to content. By putting a price on their content the Globe, the Sun and everyone else with a paywall is simultaneously helping me put a “value” on my time. And that is a real service.

Interestingly, it makes getting a subscription is even less interesting to me. A subscription is basically a license for the Globe or Sun to eat up an even bigger chunk my time (which is scarce) with content (which is not scarce), much of which I really don’t need. This is not to say that there is not good content in either newspaper. There is. Sometimes some VERY good content. But about 99% of their content is either not relevant (or basically a form of entertainment) or worse, a kind of unhelpful distraction – what Clay Johnson would call “junk information” in his book, The Information Diet: A Case for Conscious Consumption (my review here). So why would I pay to get access behind a paywall when only 1% of the content is valuable to me and the other 99% is likely an unwanted time suck? Is the 1% worth it… so far… no. Indeed, if you are going to do paywalls, newspapers had better REALLY up their game.

And, well, if I just want to burn time and read fun things… well, the internet is the best thing in the world for that! There is almost unlimited quantities of high quality articles about things I don’t urgently need to know scattered all over it!

So I’m loving the paywall because it has made me more judicious about what I read. But that happens to mean less from these newspapers. In the end, if many people are like me (and, I’m a pretty avid news reader, so not really representational) I can’t see this boding well for the industry. Maybe some big papers, like the Guardian and the New York Times can survive on this model. I can’t see local newspapers, with their value chain and costs, surviving.

Indeed the other thing I’m paying for… the editorial piece, also becomes a starker choice to me. I may be an avid news reader, but the editor of these papers is not my editor – their audience is someone else (probably a boomer). Twitter continues to be my main news editor, particularly my “thought-leaders” list of people I respect (but don’t always agree with) give me a constant stream of interesting content.

I also want to make clear, I’m also not a “it must be free” kind of guy. I mean… I like free. But I don’t worship at its altar.

I pay for books all the time. I pay for my subscription to the Economist. There I feel about 60% of the content is really good. Of course – bigger caveat – I only get the digital edition, and I really care (e.g. pay a premium for) the fact that they read every edition to me. This means I can listen while exercising (running) or walking through airports. As a result the Economist doesn’t compete for my time in the same way the Globe or Sun is – but that is clever of them! They worked themselves into my less valuable “workout” and “commuting time”, not my more valuable “productivity” time. It means I also digest almost the entire thing every week.

So my sense is that pay walls can be a good thing for thoughtful users. I’m sure it will drive many to lower quality free content, but for many others it will help them think more judicious about how they spend their time. Frankly, not reading half the stuff I read and spending that time writing, reading something else, or spending time with my son would probably be a big boost to both my productivity and quality of life.

But that doesn’t solve any problems for newspapers.

Why Banning Anonymous Comments is Bad for Postmedia and Bad for Society

Last night I discovered that my local newspaper – the Vancouver Sun – was going to require users log in with Facebook to comment. It turns out that this will be true of all Postmedia newspapers.

I’m stunned that a newspaper ownership would make such a move. Even more so that editors and journalists would support it. We should all be disappointed when the fourth estate is unable to recognize it is dis-empowering those who are most marginalized. Especially when there are better alternatives at ones disposal. (For those interested in this I also recommend reading Mathew Ingram’s post, Anonymity Has Value, In Comments and Elsewhere from over a year ago.)

So what’s wrong with forcing users to sign in via Facebook to comment?

First, you have to be pretty privileged to believe that forcing people to use their real names will improve comments. Yes, there are a lot of people who use anonymity to troll or say stupid things, but there are also many people who – for very legitimate reasons – don’t want to use their real name.

What supporters of banning anonymity are saying is not just that they oppose trolls (I do too!) but that, for the sake of “accountability” we must also know the name of recovering sexual abuse victim who wants to share their personal perspective on a story in the comments. Or that we (and thus also their boss) should get to know the name of an employee who wants to share information about illegal or unethical practices they have seen at their work in a comment. It also means that a comment you make, ten years hence, can be saved on a newspapers website, traced back to your Facebook account and so used by a prospective employer to decide if you should get a job.

What ending anonymity is really about is power. Now, those who can comment will (even more so) be disproportionately those who have the income and social security to know they can voice their concern in public, safely. So I’m confident that this move will reduce trolls – but it will also snuff out the voices of those who are most marginalized. And journalists clearly understand the power dynamics of our society and the important role anonymity plays in balancing them  this is why they use anonymous sources to get scoops and dig up stories. So how newspapers as an institution, and journalists as a profession see narrowing the opportunity for those most marginalized to challenge power and authority in the comments section as being consistent with their mission, I cannot explain.

There are, of course, far better ways of handling comments. The CBC does a quite decent job of letting people vote up and down comments – this means I rarely see the worst trolls and many thoughtful comments rise to the top. The Globe does an adequate job at this as well. Mechanisms such as these are far less draconian the “outlawing” anonymity and preserve room for those most impacted or marginalized.

But let me go further. Journalists and editors often complain about the comments section as being wild. Well how often to they take even the tiniest bit of energy to engage their commentators? There are plenty of sites that allow anonymous comments with fantastic results – see flickr or reddit – but this is because those sites invested in creating norms and engaging their users. When has a journalist or commentator in this country ever decided to invest themselves in engaging their readers and commenters on a regular and ongoing basis in the comments section? While I’m sure there are important exceptions, by and large the answer is almost never. Indeed, I’m always stunned by the number of journalists and commentators I talk to who more or less hold much of their audience in contempt – seeing them as wild. No wonder the comment section has run amok – we can pretend otherwise but the commenters know you don’t respect them. If newspapers are not happy with their comment sections, they really have no one to blame but themselves. This is after all, the community they created, the norms they fostered, the result of investments that they made. Shluffing it all off to Facebook both runs counter to their mission but is also a shirking of responsibility (and business opportunity) of the highest order.

Of course, handing the problem to Facebook won’t solve it either. It was suggested, at last count, that over 80 million facebook accounts are fake. Expect that number to go up. But of course, the people who will be most happy to create that fake account are going to be the trolls who want to use it regularly, not the lone commentator who has an important perspective about a story but doesn’t want to tell the world who they are out of fear of social stigma or worse.

What’s worse, Postmedia has now essentially farmed its privacy policy out to Facebook. Presently that means that, in theory, you can’t be anonymous. But what will it mean in the future? Postmedia can’t tell you. They can’t even influence it.

For an organization managing discussions as sensitive as newspapers do – that is a pretty shocking stance to take. Who knows what future decisions about privacy Facebook is going to make. But here’s what I do know, I trusted the National Post a hell of a lot more to manage my comments and identity than I do Facebook because their missions are totally different. In the end, this could be bad not just for comments, but for Postmedia. Many people are already pretty uncomfortable with Facebook’s policies. I expect more will become so. Even if they don’t comment, I suspect readers will be drawn to sites that engage them more effectively – a newspapers that has outsourced its engagement to Facebook will probably lose out.

I get that Postmedia believes its job of managing comments will become easier because it has outsourced identity management to Facebook – but it has come at a real cost, one that I think is unacceptable for a newspaper. In the end, I think the quality of engagement and of discussion at Postmedia will suffer. That will be bad for it, but it will also be bad for society in general.

And that is sad news for all of us.

Added @ 9:27am PST. Note: Some Postmedia journalists want to make clear that this decision was a corporate one, not theirs.

Media Watch: The Globe and Mail’s Shifting Headline

Earlier today the Globe and Mail had one of these truly terrible “balanced” articles about the proposed federal crime bill. The headline screamed: Quebec expert backs Tory crime bill amid U.S. warning on sentencing. (Image below)

So who was this expert you might ask? A university professor with years of research on the subject? Maybe some breakthrough research by a young grad student? How about a researcher from a think tank that has been investigated the issue?

Wrong on all accounts. It was, in fact, former Justice Minister Marc Bellemare from the province of Quebec. Of course, you might say… “being a Justice Minister problem should make you an expert.” If only this were the case. If Minister Vic Toews has taught us anything it is that you definitely don’t have to be an expert in something to become a Minister. Nor does being a Minister make you an expert.  But the real kicker is that Marc Bellemare was minister for just under a year. Sworn in on April 29th 2003 he resigned on April 27th 2004. Of course, the article makes no reference to the current Justice Minister of Quebec, Jean-Marc Fournier, who is both opposed to the Crime bill and has been minister since August 11th, 2010. That’s a year and a half longer making him 50% more of an expert than Bellemare!

I suspect one of two things happened (both of which I now know are wrong – see update below). Either the Globe reporter simple used language that came packaged in a press release that referred to Marc Bellemare as an expert or worse, in pursuit of “balance” the journalist felt compelled to label Bellemare’s an expert given the second part focused on how a large number of US republican “tough on crime” legislators who created mandatory minimum sentences in the 90s are trying to role them back because they have been a total failure in addressing crime and a disaster financially.

Of course Globe and Mail readers noticed the problem with the “expert” right away. The most voted for comment was the following one (yes, I voted too, might have been my first time):

comment-1

And slightly further done was a better comment pointing out some further idiotic ideas the Minister had for reforming the justice system.

comment-two

More interesting is that sometime in the later afternoon EST the Globe changed its landing page, acknowledging the “expert’s” true credentials.

GM-landing-page

I think this speaks volumes about the Globe – in a good way. Nobody is perfect, we make mistakes. Sun prides itself on getting facts wrong to tell a story and the Globe is demonstrating that they take the opposite tact. So this post isn’t to say “the globe messed up,” it’s about how newspaper can and should react to feedback from readers. It doesn’t mean you change everything all the time, but there are times when the feedback points to changes that will bring about greater clarity. It also says a lot about the power of the audience.

However, it is worth noting, the headline on the story page… remains unchanged.

7:14pm Update

I’ve made some errors of my own in the above post. I assumed above that the journalist had chosen the headline, this is, in fact, not true. As one of the editors from the Globe has pointed out to me on twitter, it was the editor who made the choice. Any assignment of blame on the journalist is misplaced, I definitely apologize for that on my part.

Not Brain Candy: A Review of The Information Diet by Clay Johnson

My body no longer kills me when I come back from the gym. However, I had a moment of total humiliation today: theoretically my ideal body weight is 172 pounds and I weigh 153 Ibs. The woman at the gym calibrated my fat/water/meat/bone ratios, made an inward gasp and I asked her what was wrong. She said (after a tentative, you-have-cancer pause), “You’re what’s technically known as a ‘thin fat person.’ ”

– Douglas Copeland, Microserfs

We know that healthy eating – having a good, balanced diet – is the most important thing we can do for our physical health. What if the same is true of our brains?  This is the simple but powerful premise that lies at the heart of Clay Johnson’s excellent book The Information Diet.

It’s also a timely thesis.

Everyone seems worried about how we consume information, about what it is doing to our brains and how it impacts society. Pessimists believe Google and social media are creating a generation of distracted idiots unable or unwilling to steep themselves in any deep knowledge. From the snide ramblings of Andrew Keen in The Cult of the Amateur to alarmed New York Times executive editor Bill Keller – who equates letting his daughter join Facebook to passing her a crystal meth pipe – the internet and the type of information it creates are apparently destroying our minds, our society and, of course, our children.

While I disagree with the likes of Keen and Keller, your humble author admits he’s an information addict. I love reading the newspaper or my favourite columnists/bloggers; I’m regularly distracted by both interesting and meaningless articles via Twitter and Facebook; and I constantly struggle to stay on top of my email inbox. I’m a knowledge worker in an information society. If anyone should be good at managing information, it should be me. Reading The Information Diet forces me to engage with my ability in a way I’ve not done before.

What makes The Information Diet compelling is that Johnson embraces the concerns we have about the world of information overload – from those raised by New York Magazine authors and celebrated pundits to the challenges we all feel on a day to day basis – and offers the best analysis to date of its causes, and what we can do about it. Indeed, rather than being a single book, The Information Diet is really three. It’s an analysis of what is happening to the media world; it’s a self-help book for information-age workers, consumers and citizens; and it’s a discussion about the implications of the media environment on our politics.

InfoDietIt is in its first section that the book shines the brightest. Johnson is utterly persuasive in arguing that the forces at play in the food industry are a powerful mirror for our media environment. Today the main threat to Americans (and most others living in the developed world) is not starvation; it’s obesity. Our factory farms are so completely effective at pumping out produce that it isn’t a lack of food the kills us, it’s an overabundance of it. And more specifically, it’s the over-consumption of food that we choose to eat, but that isn’t good for us in anything greater than small quantities.

With information, our problem isn’t that we consume too much – Johnson correctly points out that physically, this isn’t possible. What’s dangerous is consuming an overabundance of junk information – information that is bad for us. Today, one can choose to live strictly on a diet of ramen noodles and Mars bars. Similarly, it’s never been easier to restrict one’s information consumption to that which confirms our biases. In an effort to better serve us, everywhere we go, we can chomp on a steady diet of information that affirms and comforts rather than challenges – information devoid of knowledge or even accuracy; cheaply developed stories by “big info” content farms like Demand Media or cheaply created opinion hawked by affirmation factories like MSNBC or FOX News; even emails and tweets that provide dopamine bursts but little value. In small quantities, these information sources can be good and even enjoyable. In large quantities, they deplete our efficiency, stress us out, and can put us in reality bubbles.

And this is why I found The Information Diet simultaneously challenging, helpful and worrying.

Challenging, because reading The Information Diet caused me to think of my own diet. I like to believe I’m a healthy consumer, but reflecting on what I read, where I get my information and who I engage with, in parts of my life, I may be that dreaded thin-fat person. I look okay, but probe a little deeper and frankly, there are a few too many confirmation biases, too many common sources, leaving my brain insufficiently challenged and becoming a shade flabby. I certainly spend too much time on email, which frankly is a type of information fix that really does sap my productivity.

Helpful, because in part The Information Diet is a 21st-century guide to developing and honing critical thinking and reasoning skills. At its most basic, it’s a self-help book that provides some solid frameworks and tools for keeping these skills sharp in a world where the opportunities for distraction and confirmation bias remain real and the noise-to-signal ratio can be hard to navigate.  To be clear, none of this advice is overly refined, but Johnson doesn’t pretend it is. You can’t download critical thinking skills – no matter what Fox News’s slogan implies. In this regard, the book is more than helpful – it’s empowering. Johnson, correctly I believe, argues that much like the fast food industry – which seeks to exploit our body’s love of salty, fatty food – many media companies are simply indulging our desire for affirming news and opinion. It’s not large companies that are to blame. It’s the “secret compact” (as Johnson calls it) that we make with them that makes them possible. We are what we consume. In this regard, for someone that those on the right might consider (wrongly) to be a big government liberal, The Information Diet has an strong emphasis on personal responsibility.

There is, of course, a depressing flip side to this point: one that has me thinking about the broader implications of his metaphor. In a world of abundant food, we have to develop better discipline around dieting and consumption.

But the sad fact is, many of us haven’t. Indeed, almost a majority has not.

As someone who believes in democratic discourse, I’ve always accepted that as messy as our democratic systems may be, over time good ideas – those backed by evidence and effective track records – will rise to the top. I don’t think Johnson is suggesting this is no longer true. But he is implying that in a world of abundant information, the basic ante of effective participation is going up. The skills are evolving and the discipline required is increasing. If true, where does that leave us? Are we up for the challenge? Even many of those who look informed may simply be thin fat people. Perhaps those young enough to grow up in the new media environment will automatically develop the skills Clay says we need to explicitly foster. But does this mean there is a vulnerable generation? One unable to engage critically and so particularly susceptible to the siren song of their biases?

Indeed, I wish this topic were tackled more, and initially it felt like it would be. The book starts off as a powerful polemic on how we engage in information; it is then a self-help book, and towards the end, an analysis of American politics. It all makes for fascinating reading. Clay has plenty of humour, southern charm and self-deprecating stories that the pages flow smoothly past one another. Moreover, his experience serves him well. This is man who worked at Ask Jeeves in its early days, helped create the online phenomenon of the Howard Dean campaign, and co-founded Blue State Digital – which then went on to create the software that powered Obama’s online campaign.

But while his background and personality make for compelling reading, the last section sometimes feels more disconnected from the overall thesis. There is much that is interesting and I think Clay’s concerns about the limits of transparency are sound (it is a prerequisite to success, but not a solution). Much like most people know Oreos are bad for them, they know congressmen accept huge bundles of money. Food labels haven’t made America thinner, and getting better stats on this isn’t going to magically alter Washington. Labels and transparency are important tools for those seeking to diet. Here the conversation is valuable. However, some of the arguments, such as around scalability problems of representation, feel less about information and more about why politics doesn’t work. And the chapter closes with more individual advice. This is interesting, but his first three chapters create a sense of crisis around America’s information diet. I loved his suggestions for individuals, but I’d love to hear some more structural solutions, or if he thinks the crisis is going to get worse, and how it might affect our future.

None of this detracts from the book. Quite the opposite – it left me hungry for more.

And I suspect it will do the same for anyone interested in participating as a citizen or worker in the knowledge economy. Making The Information Diet part of your information diet won’t just help you rethink how you consume information, live and work. It will make you think. As a guy who knows he should eat more broccoli but doesn’t really like the taste, it’s nice to know that broccoli for your brain can be both good for you and tasty to read. I wish I had more of it in my daily diet.

For those interested you can find The Information Diet Blog here – this has replaced his older well known blog – InfoVegan.com.

Full disclosure: I should also share that I know Clay Johnson. I’ve been involved in Code for America and he sits on the Advisory Board. With that in mind, I’ve done my best to look at his book with a critical eye, but you the reader, should be aware.

Open Source Data Journalism – Happening now at Buzz Data

(there is a section on this topic focused on governments below)

A hint of how social data could change journalism

Anyone who’s heard me speak in the last 6 months knows I’m excited about BuzzData. This week, while still in limited access beta, the site is showing hints its potential – and it still has only a few hundred users.

First, what is BuzzData? It’s a website that allows data to be easily uploaded and shared among any number of users. (For hackers – it’s essentially github for data, but more social). It makes it easy for people to copy data sets, tinker with them, share the results back with the original master, mash them up with other data sets, all while engaging with those who care about that data set.

So, what happened? Why is any of this interesting? And what does it have to do with journalism?

Exactly a month ago Svetlana Kovalyova of Reuters had her article – Food prices to remain high, UN warns – re-published in the Globe and Mail.  The piece essentially outlined that food commodities were getting cheaper because of local conditions in a number of regions.

Someone at the Globe and Mail decided to go a step further and upload the data – the annual food price indices from 1990-present – onto the BuzzData site, presumably so they could play around with it. This is nothing complicated, it’s a pretty basic chart. Nonetheless a dozen or so users started “following” the dataset and about 11 days ago, one of them, David Joerg, asked:

The article focused on short-term price movements, but what really blew me away is: 1) how the price of all these agricultural commodities has doubled since 2003 and 2) how sugar has more than TRIPLED since 2003. I have to ask, can anyone explain WHY these prices have gone up so much faster than other prices? Is it all about the price of oil?

He then did a simple visualization of the data.

FoodPrices

In response someone from the Globe and Mail entitled Mason answered:

Hi David… did you create your viz based on the data I posted? I can’t answer your question but clearly your visualization brought it to the forefront. Thanks!

But of course, in a process that mirrors what often happens in the open source community, another “follower” of the data shows up and refines the work of the original commentator. In this case, an Alexander Smith notes:

I added some oil price data to this visualization. As you can see the lines for everything except sugar seem to move more or less with the oil. It would be interesting to do a little regression on this and see how close the actual correlation is.

The first thing to note is that Smith has added data, “mashing in” Oil Price per barrel. So now the data set has been made richer. In addition his graph quite nice as it makes the correlation more visible than the graph by Joerg which only referenced the Oil Price Index. It also becomes apparent, looking at this chart, how much of an outlier sugar really is.

oilandfood

Perhaps some regression is required, but Smith’s graph is pretty compelling. What’s more interesting is not once is the price of oil mentioned in the article as a driver of food commodity prices. So maybe it’s not relevant. But maybe it deserves more investigation – and a significantly better piece, one that would provide better information to the public – could be written in the future. In either case, this discussion, conducted by non-experts simply looking at the data, helped surface some interesting leads.

And therein lies the power of social data.

With even only a handful of users a deeper, better analysis of the story has taken place. Why? Because people are able to access the data and look at it directly. If you’re a follower of Julian Assange of wikileaks, you might call this scientific journalism, maybe it is, maybe it isn’t, but it certainly is a much more transparent way for doing analysis and a potential audience builder – imagine if 100s or 1000s of readers were engaged in the data underlying a story. What would that do to the story? What would that do to journalism? With BuzzData it also becomes less difficult to imagine a data journalists who spends a significant amount of their time in BuzzData working with a community of engaged pro-ams trying to find hidden meaning in data they amass.

Obviously, this back and forth isn’t game changing. No smoking gun has been found. But I think it hints at a larger potential, one that it would be very interesting to see unlocked.

More than Journalism – I’m looking at you government

Of course, it isn’t just media companies that should be paying attention. For years I argued that governments – and especially politicians – interested in open data have an unhealthy appetite for applications. They like the idea of sexy apps on smart phones enabling citizens to do cool things. To be clear, I think apps are cool too. I hope in cities and jurisdictions with open data we see more of them.

But open data isn’t just about apps. It’s about the analysis.

Imagine a city’s budget up on Buzzdata. Imagine, the flow rates of the water or sewage system. Or the inventory of trees. Think of how a community of interested and engaged “followers” could supplement that data, analyze it, visualize it. Maybe they would be able to explain it to others better, to find savings or potential problems, develop new forms of risk assessment.

It would certainly make for an interesting discussion. If 100 or even just 5 new analyses were to emerge, maybe none of them would be helpful, or would provide any insights. But I have my doubts. I suspect it would enrich the public debate.

It could be that the analysis would become as sexy as the apps. And that’s an outcome that would warm this policy wonk’s soul.