For those that missed them two of my favourite authors – Clay Shirky and Steven Johnson – posted brilliant pieces on the future of the news industry this week. I’ve pulled some of the best lines from both so you can glimpse at why these to writers are models for me. These relevant paragraphs also reveal a further analysis, one I think both authors stop shy of but that both pieces hint at: the Death of Journalism.

…in the long run, we’re going to look back at many facets of old media and realize that we were living in a desert disguised as a rain forest. Local news may be the best example of this. When people talk about the civic damage that a community suffers by losing its newspaper, one of the key things that people point to is the loss of local news coverage… I adore the City section of the New York Times, but every Sunday when I pick it up, there are only three or four stories in the whole section that I find interesting or relevant to my life – out of probably twenty stories total. And yet every week in my neighborhood there are easily twenty stories that I would be interested in reading: a mugging three blocks from my house;

But of course, that’s what the web can do. That’s one of the main reasons we created outside.in, because I found myself waking up in the morning and turning to local Brooklyn bloggers like Brownstoner, who were suddenly covering local news with a granularity that the Times had never attempted. Two years later, there are close to a thousand bloggers writing about Brooklyn: there are multiple blogs devoted to the Atlantic Yards real estate development; dozens following the Brooklyn foodie scene; music blogs, politics blogs, parenting blogs. [A veritable rain forest of information where there was once a desert]

– Steven B Johnson

When someone demands to know how we are going to replace newspapers, they are really demanding to be told that we are not living through a revolution. They are demanding to be told that old systems won’t break before new systems are in place. They are demanding to be told that ancient social bargains aren’t in peril, that core institutions will be spared, that new methods of spreading information will improve previous practice rather than upending it. They are demanding to be lied to.

– Clay Shirky

Both Shirky and Johnson’s pieces acknowledge that the trends hitting the news industry are hitting every part of society but – because they have written articles and not books – they deal with the changes to the news industry ecosystem in isolation. As a result, their analyses account for the death of the newspaper in its current form. However, both shy away from explicitly looking over that bigger cliff – are we seeing the death of Journalism? I for one, hope so, as it will mean a more profound change may be upon us.

Step back and look at the relationship between news gathering institutions and the organizations they report on. A large piece of “investigative news” has been about one set of opaque institutions – the news organization – covertly gathering information on another set of opaque institutions – government, corporations or non-profits – so as to shine a light on some malfeasance.

What if it isn’t just the business model newspapers and TV news that is faulty. What if it is the underlying structure and values are eroding not only among news institutions but also among the entities they normally cover? What if the belief in objectivity and trust in opaque structures are dying? This would mean that the public’s confidence in products, ideas, services, policies and news created behind a curtain – within any opaque institutions – is slowly crumble. In his Bertha Bassam lecture, this is precisely what David Weinberger brilliantly argues is already taking place:

“Wikipedia is far more credible because it shows us how the sausage is made makes Wikipedia far more credible. Yet this is exactly the stuff that the Britannica wont show us because they think it would make them look amateurish and take away from their credibility. But in fact transparency – which is what this is – is the new objectivity. We are not going to trust objectivity, we are not going to trust objectivity unless we can see the discussion that lead to it.”

Such a transformation, a reshaping of credibility from objectivity to transparency, would have profound implications for every organization – corporate, non-profit and governmental – in our society.

The trends Shirky and Johnson describe as killing newspapers – the fact there are more eyes, able to create more information, that is able to flow faster, and freer than ever before – may be making openness and transparency a strategically salient choice for an increasing number of organizations. Firstly, it is simply becoming harder and harder to keep secrets. More and more organizations may decide that, rather than devote energy to hiding secrets that will inevitable see the light of day, why not devote energy to solving the underlying problems that are creating them? More importantly, by being transparent allows these organizations to access the long tail of analyses an additional powerful incentive to being open. Those who share information and invite criticism and analysis may be better positioned to survive crises and challenges than those who don’t. Many eyes makes the bugs in any institution more likely to be shallow.

As a result we may see an organizational ecosystem emerging that strongly favours transparency. This is not to say that every organization will instantaneously become “open” overnight… but there would be increasing pressure, and more powerfully the adoption of the naked corporation as the default model in the new system.

Such a shift would forever change journalism. The first is that opaque news entities – those that don’t make clear the bargain with their readers, that fail to spell out their editorial decisions and philosophy and allow readers to hold them to account, will themselves be at risk. I suspect this will be true even if some magical financial solution (like the terrible idea of subsidizing news with an internet access tax) were to emerge. The problem would simply shift from being a financial crisis to a credibility crisis. If journalism prides itself on objectivity – then it had better find ways to be transparent. This means news sites had better engage with legitimate critics: and this means doing more than having columnists who ignore commenters that poke large holes in their arguments or electing to publish retractions on the bottom corner of page 8 or on some lost webpage.

More profound however may be that journalism in a transparent ecosystem could look very different than it does today. If investigative journalism has been about uncovering the dirty secrets within opaque institutions – what does it do if an increasing number of institutions have no secrets?

I suspect the ideal of good journalism will shift from being what Gladwell calls puzzle solving to mystery solving. In the former you must find a critical piece of the puzzle – one that is hidden to you – in order to explain an event. This is the Woodward and Bernstein model of journalism – the current ideal. But in a transparent landscape where huge amounts of information about most organizations is being generated and shared the critical role of the journalist will be that of mystery solving – figuring out how to analyze, synthesize and discover the mystery within the vast quantity of information. As Gladwell recounts this was ironically the very type of journalism that brought down Enron (an organization that was open, albeit deeply  flawed). All of the pieces of that lead to the story that “exposed” Enron were freely, voluntarily and happily given to reports by Enron. It’s just a pity it didn’t happen much, much sooner.

I for one would celebrate the rise of this mystery focused style of “journalism.” It has been sorely needed over the past few years. Indeed, the housing crises that lead to the current financial crises is a perfect example of case where we needed mystery solving not puzzle solving, journalism. The fact that sub-prime mortgages were being sold and re-packaged was not a secret, what was lacking was enough people willing to analyze and write about this complex mystery and its dangerous implications.

Interestingly this is precisely what many blogs – alone or as part of an emergent network – already do. They take large complex stories, break them down and, by linking back and forth to one another, create a collective analysis that slowly allows the mystery to be decoded. I hope this post is part of such a mystery solving exercise – I’m trying to build off of, and extend, the brilliant analyses of Johnson and Shirky.

Does this mean the death of journalism? Well, in a world where everybody can be a journalists… is anyone a journalist? I don’t know. I’m sure there will always be some professional journalists, but in a world where people distrust opaque institutions I’m not certain they will reside in organizations that look even remotely like the news institutions of today. Most importantly, in a world of mysteries perhaps citizen journalist and bloggers, and their role in the news ecoysystem, will be less frightening than the one most present day pundits (especially newspaper columnists) would have us believe.