I’ve been doing a lot of thinking about access to information, and am working on a longer analysis, but in the short term I wanted to share two graphs – graphs that outline why Access to Information (Freedom of Information in the United States) is unsustainable and will, eventually, need to be radically rethought.
First, this analysis is made possible by the enormous generosity of the Canadian Federal Information Commissioners Office which several weeks ago sent me a tremendous amount of useful data regarding access to information requests over the past 15 years at the Treasury Board Secretariat (TBS).
The first figure I created shows both the absolute number of Access to Information Requests (ATIP) since 1996 as well as the running year on year percentage increase. The dotted line represents the average percentage increase over this time. As you can see the number of ATIP requests has almost tripled in this time period. This is very significant growth – the kind you’d want to see in a well run company. Alas, for those processing ATIP requests, I suspect it represents a significant headache.
That’s because, of course, such growth is likely unmanageable. It might be manageable if say, the costs of handling each requests was dropping rapidly. If such efficiencies were being wrestled out of the system of routing and sorting requests then we could simply ignore the chart above. Sadly, as the next chart I created demonstrates this is not the case.
In fact the costs of managing these transactions has not tripled. It has more than quadrupled. This means that not only are the number of transactions increasing at about 8% a year, the cost of fulfilling each of those transactions is itself rising at a rate above inflation.
Now remember, I’m not event talking about the effectiveness of ATIP. I’m not talking about how quickly requests are turned around (as the Information Commissioner has discussed, it is broadly getting worse) nor am I discussing less information is being restricted (it’s not, things are getting worse). These are important – and difficult to assess – metrics.
I am, instead, merely looking at the economics of ATIP and the situation looks grim. Basically two interrelated problems threaten the current system.
1) As the number of ATIP requests increase, the manpower required to answer them also appears to be increasing. At some point the hours required to fulfill all requests sent to a ministry will equal the total hours of manpower at that ministry’s disposal. Yes that day may be far off, but they day where it hits some meaningful percentage – say 1%, 3% or 5% of total hours worked at Treasury Board, may not be that far off. That’s a significant drag on efficiency. I recall talking to a foreign service officer who mentioned that during the Afghan prisoner scandal an entire department of foreign service officers – some 60 people in all – were working full time on assessing access to information requests. That’s an enormous amount of time, energy and money.
2) Even more problematic than the number of work hours is the cost. According to the data I received, Access to Information requests costs The Treasury Board $47,196,030 last year. Yes, that’s 47 with a “million” behind it. And remember, this is just one ministry. Multiply that by 25 (let’s pretend that’s the number of ministries, there are actually many more, but I’m trying to be really conservative with my assumptions) and it means last year the government may have spent over $1.175 Billion fulfilling ATIP requests. That is a staggering number. And its growing.
Transparency, apparently, is very, very expensive. At some point, it risks becoming too expensive.
Indeed, ATIP reminds me of healthcare. It’s completely unsustainable, and absolutely necessary.
To be clear, I’m not saying we should get rid of ATIP. That, I believe, to be folly. It is and remains a powerful tool for holding government accountable. Nor do I believe that requesters should pay for ATIP requests as a way to offset costs (like BC Ferries does) – this creates a barrier that punishes the most marginalized and threatened, while enabling only the wealthy or well financed to hold government accountable.
I do think it suggests that governments need to radical rethink how manage ATIP. More importantly I think it suggests that government needs to rethink how it manages information. Open data, digital documents are all part of a strategy that, I hope, can lighten the load. I’ve also felt that if/as government’s move their work onto online platforms like GCPEDIA, we should simply make non-classified pages open to the public on something like a 5 year timeline. This could also help reduce requests.
I’ve more ideas, but at its core we need a system rethink. ATIP is broken. You may not know it yet, but it is. The question is, what are we going to do before it peels off the cliff? Can we invent something new and better in time?
eaves.ca 
Beyond the economics of the system, the human dynamics of it haven’t been properly considered either. This example in Wisconsin is representative of the issues: http://www.nytimes.com/2011/03/28/opinion/28mon3.html
Although open inforamtion is useful, the costs (both monetary and otherwise) can sometimes be too much.
David -thanks for this – as a consultant working with the federal government on information management solutions i have been harboring a sneaking suspicion that the transformation that we are pursuing in how we will manage our digital information (many of the decisions being made at a very practical level and often fueled by economic realities) will eventually require a rethinking of expectations and guidelines as they relate to Atip. For example few IM groups will be managing retention and disposition at the granular levels we saw in the paper world meaning that we will be keeping a lot more information for a lot longer than we previously have. Consider also the challenge of confidently expressing that a particular piece of information is no longer available, owing to it’s “official” disposal when the likelihood of a given electronic resource existing obly as a single version within an organization is is about as slim as my being the next prime minister. We see that TBS is rethinking many atip tools (such as infosource) in order to better serve Canadians in this digital age but perhaps there is a real need to step back and consider at a higher level the realities and practicalities of continuing to pursue and conduct ATIP requests in our traditional manner given the changing IM landscape and realities of IM in the GC.
Lindsay Fraser – Systemscope
It would be interesting to know how many requests are duplicate, and how efficiently duplicates are managed. Worst case scenario is each request is fulfilled without consulting whether a similar/identical request has already been made. I think making the system open (publishing both requests and responses) will lighten the load if there is significant duplication.
“First, this analysis is made possible by the enormous generosity of the Canadian Federal Information Commissioners Office which several weeks ago sent me a tremendous amount of useful data regarding access to information requests over the past 15 years at the Treasury Board Secretariat (TBS).” – Any chance at this data being released for general consumption?
Definitely. Will post it when in back at my computer. Likely tomorrow.
—
http://www.eaves.ca
@daeaves
David,
$47,196,030 is the cost for ATI across all 280+ departments that comply with ATI:
“These figures are based on Statistical Reports provided by government institutions subject to the Access to Information Act.”
http://www.infosource.gc.ca/bulletin/2010/b/bulletin33b/bulletin33b02-eng.asp
Also what needs to be included in estimates of the economics of ATI are the benefits brought to government. How much waste has been identified or adverted because of ATI? How has the government benefit from businesses making more competitive bids by using ATI? How have costly tragedies been avoided?
And of course, how much of the current costs of handling ATI are avoidable costs? The Office of the Information Commissioner recently released a report about the process known as “contentious issue management” or by journalists “amber lighting.” This is a process where ATIs are given a sensitivity label and special handling. The results of the investigation include:
“we found that institutions that label access requests as ‘sensitive,’ ‘of interest’ or ‘amber light,’ or with some other marker indicating special handling, tend to delay requests for unacceptably long periods. We also found that the media are not the only ones to encounter such delays. Requests from parliamentarians, organizations, academics and lawyers are also delayed. ”
http://www.oic-ci.gc.ca/eng/syst-inv_inv-syst_do_secret_rules_lead_to_delays.aspx
(For a great article on amber lighting see: http://www.atkinsonfoundation.ca/publications/Atkinson1_Ann_Rees.pdf)
To hell with a five year timeline. Non-sensitive information (i.e. how may people used highway 401 on a particular day) should be released as soon AS IT IS AVAILABLE. Weather information is released this way, in fact weather information is released in advance due to advanced prediction technology.
The rest of the government should follow Environment Canada’s lead on this. Unless there is a compelling reason (privacy for example) the data is immediately released. No five year wait. Immediately. And the government should also provide a database of databases, to act as a guide for citizens to find the data they need.
It’s possible some ‘sensitive’ information may need to be held for varying periods of time. Military and security operations come to mind. However full information should be released on those too. All too often, we, the Canadian people, find that secrecy is just an excuse to hide mistakes. That we will not tolerate.
Wayne
“At some point the hours required to fulfill all requests sent to a ministry will equal the total hours of manpower at that ministry’s disposal.” This assumes the number of requests will reach that point. I don’t think that’s a fair assumption. It will likely plateau much sooner than that, before even 1% of time is spent on FOI. Also, 60 people, in the Government of Canada, is not a lot. There are over a quarter million civil servants.
This, along with Mark’s comment, suggests that FOI is not broken at an economic level. It’s broken in other respects, no doubt.
Thanks for this great insight into public data availability. Why do we need people to retrieve the data and/or decide whether we can see it? I don’t see the point.
The big change in society today, which I call the Fourth Revolution, will see a significant change in the role of government. Why? In the Industrial Age, information and communication management was power. One of the legitimacies of the state government was that it was one of the few organizations able to create and manipulate enormous (at the time) amounts of data. And it had large privileges over communication networks. Remember the first computers were used for census.
Today, information management and communication is more than abundant: it is ubiquitous. Why should the justification of government still be the management of data? Any private company nowadays, or even a consortium of volunteers, can do better for cheaper.
Definitely government need to change their model and leave data management to others.
Another inefficiency that has been introduced is the apparent need for departments to consult with central agencies and other departments when preparing to release information. Because TBS is such a central agency, in fact, I suspect it spends a lot more than other departments (relatively speaking).
Just read a related and interesting blog post: http://infovegan.com/2011/03/23/the-costs-of-foia
The US doesn’t spend even half a billion on FOIA. I doubt Canada spends over a billion.
Pingback: A partial answer to “Open Data: who profits?” | Stop
Pingback: Cost of not opening PSI is increasing | Stop