Tag Archives: FOIA

Access to Information, Technology and Open Data – Keynote for the Commissioners

Leave a reply

On October 11th I was invited by Elizabeth Denham, the Access to Information and Privacy Commissioner for British Columbia to give a keynote at the Privacy and Access 20/20 Conference in Vancouver to an audience that included the various provincial and federal Information Commissioners.

Below is my keynote, I’ve tried to sync the slides up as well as possible. For those who want to skip to juicier parts:

  • 7:08 – thoughts about the technology dependence of RTI legislation
  • 12:16 –  the problematic approach to RTI implementation that results from these unsaid assumptions
  • 28:25 – the need and opportunity to bring open data and RTI advocates together

Some acronyms used:

Why Journalists Should Support Putting Access to Information Requests Online Immediately

Leave a reply

Here’s a headline you don’t often expect to see: “Open-Government Laws Fuel Hedge-Fund Profits.”

It’s a fascinating article that opens with a story about SAC Capital Advisors LP – a hedge fund. Last December SAC Capital used Freedom of Information Laws (FOIA) to request preliminary results on a Vertex Pharmaceuticals drug being tested by the US Food and Drug Administration. The request revealed there were no “adverse event reports,” increasing the odds the drug might be approved. SAC Capital used this information – according to the Wall Street Journal – to snatch up 15,000 shares and 25,000 options of Vertex. In December – when the request was made – the stock traded around $40. Eight months later it peaked at $89 and still trades today at around $75. Thus, clever usage of government access to information request potentially netted the company a cool ROI of 100% in 9 months and a profit of roughly 1.2 million dollars (assuming they sold around $80).

This is an interesting story. And I fear it says a lot about the future of access to information laws.

This is because it contrasts sharply with the vision of access to information the media likes to portray: Namely, that access requests are a tool used mainly by hardened journalists trying to uncover dirt about a government. This is absolutely the case… and an important use case. But it is not the only usage of access laws. Nor was it the only intended use of the law. Indeed, it is not even the main usage of the law.

In my work on open data I frequently get pulled into conversations about access to information laws and their future. I find these conversations are aggressively dominated by media representatives (e.g. reporters) who dislike alternative views. Indeed, the one-sided nature of the conversation – with some journalists simply assuming they are the main and privileged interpreters of the public interest around access laws – is deeply unhealthy. Access to information laws are an important piece of legislation. Improving and sustaining them requires a coalition of actors (particularly including citizens), not just journalists. Telling others that their interests are secondary is not a great way to build an effective coalition. Worse, I fear the dominance of a single group means the conversation is often shaped by a narrow view of the legislation and with a specific set of (media company) interests in mind.

For example, many governments – including government agencies in my own province of British Columbia – have posted responses to many access to information requests publicly. This enrages (and I use that word specifically) many journalists who see it as a threat. How can they get a scoop if anyone can see government responses to their requests at the same time? This has led journalists to demand – sometimes successfully – that the requestor have exclusive access to government responses for a period of time. Oy vey. This is dangerous.

For certain types of stories I can see how complete transparency of request responses could destroy a scoop. But most stories – particularly investigative stories – require sources and context and understanding. Such advantages, I suspect, are hard to replicate and are the real source of competitive advantage (and if they aren’t… shouldn’t they be?).

It also suggests that a savvy public – and the media community – won’t be able to figure out who always seems to be making the right requests and reward them accordingly. But let’s put issues of a reputation economy and the complexity of reporting on a story aside.

First, it is worth noting that it is actually in the public interest to have more reporters cover a story and share a piece of news – especially about the government. Second, access to information laws were not created to give specific journalists scoops – they were designed to maximize the public’s capacity to access government information. Protecting a media company’s business model is not the role of access laws. It isn’t even in the spirit of the law.

Third, and worst, this entire debate fails to discuss the risks of such an approach. Which brings me back to the Wall Street Journal article.

I have, for years, warned that if public publication of access to information requests results are delayed so that one party (say, a journalist) has exclusive access for a period of time, then the system will also be used by others in pursuit of interests that might not be in the public good. Specifically, it creates a strong incentive for companies and investors to start mining government to get “exclusive” rights to government information they can put to use in advancing their agenda – making money.

As the SAC Capital Case outlined above underscores, information is power. And if you have exclusive access to that information, you have an advantage over others. That advantage may be a scoop on a government spending scandal, but it can also be a stock tip about a company whose drug is going to clear a regulatory hurdle, or an indication that a juicy government contract is about to be signed, or that a weapons technology is likely to be shelved by the defence department. In other words – and what I have pointed out to my journalist friends – exclusivity in access to information risks transforming the whole system into a giant insider information generation machine. Great for journalists? Maybe. (I’ve my doubts – see above.) But great for companies? The Wall Street Journal article shows us it already is. Exclusivity would make it worse.

Indeed, in the United States, the private sector is already an enormous generator of access requests. Indeed one company, that serves as a clearing house for requests, accounts for 10% of requests on its own:

The precise number of requests from investors is impossible to tally because many come from third-party organizations that send requests on behalf of undisclosed clients—a thriving industry unto itself. One of them, FOI Services Inc., accounted for about 10% of the 50,000 information requests sent to the FDA during the period examined by the Journal. Marlene Bobka, a senior vice president at Washington-based FOI Services, says a “huge, huge reason people use our firm is to blind their requests.”

Imagine what would happen if those making requests had formal exclusive rights? The secondary market in government information could become huge. And again, not in a way that advances the public interest.

In fact, given the above-quoted paragraph, I’m puzzled by the fact that journalists don’t demand that every access to information request be made public immediately. All told, the resources of the private sector (to say nothing of the tens of thousands of requests made by citizens or NGOs) dwarf those of media companies. Private companies may start (or already are) making significantly more requests than journalists ever could. Free-riding on their work could probably be a full time job and a successful career for at least a dozen data journalists. In addition, by not duplicating this work, it frees up media companies’ capacity to focus on the most important problems that are in the public good.

All of this is to say… I fear for a world where many of the journalists I know – by demanding changes that are in their narrow self-interest – could help create a system that, as far as I can tell, could be deeply adverse to the public interest.

I’m sure I’m about to get yelled at (again). But when it comes to access to information requests, we are probably going to be better off in a world where they are truly digitized. That means requests can be made online (something that is somewhat arriving in Canada) and – equally importantly – where results are also published online for all to see. At the very minimum, it is a conversation that is worth having.

On Being Misquoted – Access Info Europe and Freedominfo.org

2 Replies

I’ve just been alerted to a new post out on Freedominfo.org has quotes of mine that are used in way that is deeply disappointing. It’s never fund to see your ideas misused to make it appear that you are against something that you deeply support.

The most disappointing misquote comes from Helen Darbishire, a European FOI expert at Access Info Europe. Speaking about the convergence between open data and access to information laws (FOIA) she “lamented that comments like Eaves’ exacerbate divisions at a time when  “synergies” are developing at macro and micro levels.” The comment she is referring to is this one:

“I just think FOIA is broken; the wait time makes it broken….” David Eaves, a Canadian open government “evangelist,” told the October 2011 meeting of International information commissioners. He said “efforts to repair it are at the margins” and governments have little incentive for reform.

I’m not sure if Darbishire was present at the 7th International Conference of Information Commissioners where I made this comment in front of a room of mostly FOI experts but the comment actually got a very warm reception. Specifically, I was talking about how the wait times of access to information requests – not theidea of Access to Information. The fact is, that for many people waiting 4-30 weeks for a response from a government for a piece of information makes the process broken. In addition, I often see the conversation among FOIA experts focus on how to reduce that time by a week or a few days. But for most people, that will still leave them feeling like the system is too slow and so, in their mind, broken, particularly in a world where people are increasingly used to getting the information they want in about .3 seconds (the length of a Google search).

What I find particularly disappointing about Darbishire’s comments is that I’ve been advocating for for the open data and access to information communities to talk more to one another – indeed long before I find any reference of her calling for it. Back in April during the OGP meeting I wrote:

There remain important and interest gaps particularly between the more mature “Access to Information” community and the younger, still coalescing “Gov2.0/OpenGov/Tech/Transparency” community. It often feels like members of the access to information community are dismissive of the technology aspects of the open government movement in general and the OGP in particular. This is disappointing as technology is likely going to have a significant impact on the future of access to information. As more and more government work gets digitized, how way we access information is going to change, and the opportunities to architect for accessibility (or not) will become more important. These are important conversations and finding a way to knit these two communities together more could help the advance everyone’s thinking.

And of course, rather than disparage Access to Information as a concept I frequently praise it, such as during this article about the challenges of convergence between open data and access to information:

Let me pause to stress, I don’t share the above to disparage FOI. Quite the opposite. It is a critical and important tool and I’m not advocating for its end. Nor am I arguing the open data can – in the short or even medium term – solve the problems raised above.

That said, I’m willing to point out the failures of both Open Data and Access to information. But to then cherry pick my comments about FOIA and paint me as someone who is being unhelpful strikes me as problematic.

I feel doubly that way since, not only have I advocated for efforts to bridge the communities, I’ve tried to make efforts to make it happen. I was the one who suggested that Warren Krafchik – the Civil Society co-chair of the Open Government Partnership be invited to the Open Knowledge Festival to help with a conversation around helping bring the two communities together and reached out to him with the invitation.

If someone wants to label me as someone who is opinionated in the space, that’s okay – I do have opinions about what works and what doesn’t work and try to share them, sometimes in a constructive way, and sometimes – such as when on a panel – in a way that helps spur discussion. But to lay the charge of being divisive, when I’ve been trying for several years to bridge the conversation and bring the open data perspective into the FOIA community, feels unfair and problematic.

Access to Information, Open Data and the Problem with Convergence

12 Replies

In response to my post yesterday one reader sent me a very thoughtful commentary that included this line at the end:

“Rather than compare [Freedom of Information] FOI legislation and Open Gov Data as if it’s “one or the other”, do you think there’s a way of talking about how the two might converge?”

One small detail:

So before diving in to the meat let me start by saying I don’t believe anything in yesterday’s post claimed open data was better or worse than Freedom of Information (FOI often referred to in Canada as Access to Information or ATI). Seeing FOI and open data as competing suggests they are similar tools. While they have similar goals – improving access – and there may be some overlap, I increasingly see them as fundamentally different tools. This is also why I don’t see an opportunity for convergence in the short term (more on that below). I do, however, believe open data and FOI processes can be complimentary. Indeed, I’m hopeful open data can alleviate some of the burden placed on FOI system which are often slow. Indeed, in Canada, government departments regularly violate rules around disclosure deadlines. If anything, this complimentary nature was the implicit point in yesterday’s post (which I could have made more explicit).

The Problem with Convergence:

As mentioned above, the overarching goals of open data and FOI systems are similar – to enable citizens to access government information – but the two initiatives are grounded in fundamentally different approaches to dealing with government information. From my view FOI has become a system of case by case review while open data is seeking to engage in an approach of “pre-clearance.”

Part of this has to do with what each system is reacting to. FOI was born, in part, out of a reaction to scandals in the mid 20th century which fostered public support for a right to access government information.

FOI has become a powerful tool for accessing government information. But the infrastructure created to manage it has also had some perverse effects. In some ways FOI has, paradoxically made it harder to gain access to government information. I remember talking to a group of retired reporters who talk about how it was easier to gain access to documents in a pre-FOI era since there were no guidelines and many public servants saw most documents as “public” anyways. The rules around disclosure today – thanks in part to FOI regimes – mean that governments can make closed the “default” setting for government information. In the United States the Ashcroft Memo serves as an excellent example of this problem. In this case the FOI legislation actually becomes a tool that helps governments withhold documents, rather than enable citizens to gain legitimate access.

But the bigger problem is that the process by which access to information requests are fulfilled is itself burdensome. While relevant and necessary for some types of information it is often overkill for others. And this is the niche that open data seeks to fill.

Let me pause to stress, I don’t share the above to disparage FOI. Quite the opposite. It is a critical and important tool and I’m not advocating for its end. Nor am I arguing the open data can – in the short or even medium term – solve the problems raised above.

This is why, over the short term, open data will remain a niche solution – a fact linked to its origins. Like FOI Open data has its roots in government transparency. However, it also evolved out of efforts to tear down antiquated intellectual property regimes to the facilitate sharing of data/information (particularly between organizations and governments). Thus the emphasis was not on case by case review of documents, but rather of clearing rights to categories of information, both created and to be created in the future. In other words, this is about granting access to the outputs of a system, not access to individual documents.

Another way of thinking about this is that open data initiatives seek to leverage the benefits of FOI while jettisoning its burdensome process. If a category of information can be pre-clear in advanced and in perpetuity for privacy, security and IP concerns then FOI processes – essential for individual documents and analysis – becomes unnecessary and one can reduce the transaction costs to citizens wishing to access the information.

Maybe, in the future, the scope of these open data initiatives could become broader, and I hope they will. Indeed there is, ample evidence to suggest that technology could be used to pre-clear or assess the sensitivity of any government document. An algorithm that assess a mixture of who the author is, the network of people who review it and a scan of the words would probably allow ascertain if a document could be released to an ATIP request in seconds, rather than weeks. It could at least give a risk profile and/or strip out privacy related information. These types of reforms would be much more disruptive (in the positive sense) to FOI legislation than open data.

But all that said, just getting the current focus of open data initiatives right would be a big accomplishment. And, even if such initiatives could be expanded, there are limits. I am not so naive to believe that government can be entirely open. Nor am I sure that would be an entirely good outcome. When trying to foster new ideas or assess how to balance competing interests in society, a private place to initiate and play with ideas may be essential. And despite the ruminations above, the limits of government IT systems means there will remain a lot of information – particularly non-data information like reports and analysis – that we won’t be able to “pre-clear- for sharing and downloading. Consequently an FOI regime – or something analogous – will continue to be necessary.

So rather than replace or converge with FOI systems, I hope open data will, for the short to medium term actually divert information out of the FOI, not because it competes, but because it offers a simpler and more efficient means of sharing (for both government and citizens) certain types of information. That said, open data initiatives offer none of the protections or rights of FOI and so this legislation will continue to serve as the fail safe mechanism should a government choose to stop sharing data. Moreover, FOI will continue to be a necessary tool for documents and information that – for all sorts of reasons (privacy, security, cabinet confidence, etc…) cannot fall under the rubric of an open data initiative. So convergence… not for now. But co-existence feels both likely and helpful for both.