In response to my post yesterday one reader sent me a very thoughtful commentary that included this line at the end:
“Rather than compare [Freedom of Information] FOI legislation and Open Gov Data as if it’s “one or the other”, do you think there’s a way of talking about how the two might converge?”
One small detail:
So before diving in to the meat let me start by saying I don’t believe anything in yesterday’s post claimed open data was better or worse than Freedom of Information (FOI often referred to in Canada as Access to Information or ATI). Seeing FOI and open data as competing suggests they are similar tools. While they have similar goals – improving access – and there may be some overlap, I increasingly see them as fundamentally different tools. This is also why I don’t see an opportunity for convergence in the short term (more on that below). I do, however, believe open data and FOI processes can be complimentary. Indeed, I’m hopeful open data can alleviate some of the burden placed on FOI system which are often slow. Indeed, in Canada, government departments regularly violate rules around disclosure deadlines. If anything, this complimentary nature was the implicit point in yesterday’s post (which I could have made more explicit).
The Problem with Convergence:
As mentioned above, the overarching goals of open data and FOI systems are similar – to enable citizens to access government information – but the two initiatives are grounded in fundamentally different approaches to dealing with government information. From my view FOI has become a system of case by case review while open data is seeking to engage in an approach of “pre-clearance.”
Part of this has to do with what each system is reacting to. FOI was born, in part, out of a reaction to scandals in the mid 20th century which fostered public support for a right to access government information.
FOI has become a powerful tool for accessing government information. But the infrastructure created to manage it has also had some perverse effects. In some ways FOI has, paradoxically made it harder to gain access to government information. I remember talking to a group of retired reporters who talk about how it was easier to gain access to documents in a pre-FOI era since there were no guidelines and many public servants saw most documents as “public” anyways. The rules around disclosure today – thanks in part to FOI regimes – mean that governments can make closed the “default” setting for government information. In the United States the Ashcroft Memo serves as an excellent example of this problem. In this case the FOI legislation actually becomes a tool that helps governments withhold documents, rather than enable citizens to gain legitimate access.
But the bigger problem is that the process by which access to information requests are fulfilled is itself burdensome. While relevant and necessary for some types of information it is often overkill for others. And this is the niche that open data seeks to fill.
Let me pause to stress, I don’t share the above to disparage FOI. Quite the opposite. It is a critical and important tool and I’m not advocating for its end. Nor am I arguing the open data can – in the short or even medium term – solve the problems raised above.
This is why, over the short term, open data will remain a niche solution – a fact linked to its origins. Like FOI Open data has its roots in government transparency. However, it also evolved out of efforts to tear down antiquated intellectual property regimes to the facilitate sharing of data/information (particularly between organizations and governments). Thus the emphasis was not on case by case review of documents, but rather of clearing rights to categories of information, both created and to be created in the future. In other words, this is about granting access to the outputs of a system, not access to individual documents.
Another way of thinking about this is that open data initiatives seek to leverage the benefits of FOI while jettisoning its burdensome process. If a category of information can be pre-clear in advanced and in perpetuity for privacy, security and IP concerns then FOI processes – essential for individual documents and analysis – becomes unnecessary and one can reduce the transaction costs to citizens wishing to access the information.
Maybe, in the future, the scope of these open data initiatives could become broader, and I hope they will. Indeed there is, ample evidence to suggest that technology could be used to pre-clear or assess the sensitivity of any government document. An algorithm that assess a mixture of who the author is, the network of people who review it and a scan of the words would probably allow ascertain if a document could be released to an ATIP request in seconds, rather than weeks. It could at least give a risk profile and/or strip out privacy related information. These types of reforms would be much more disruptive (in the positive sense) to FOI legislation than open data.
But all that said, just getting the current focus of open data initiatives right would be a big accomplishment. And, even if such initiatives could be expanded, there are limits. I am not so naive to believe that government can be entirely open. Nor am I sure that would be an entirely good outcome. When trying to foster new ideas or assess how to balance competing interests in society, a private place to initiate and play with ideas may be essential. And despite the ruminations above, the limits of government IT systems means there will remain a lot of information – particularly non-data information like reports and analysis – that we won’t be able to “pre-clear- for sharing and downloading. Consequently an FOI regime – or something analogous – will continue to be necessary.
So rather than replace or converge with FOI systems, I hope open data will, for the short to medium term actually divert information out of the FOI, not because it competes, but because it offers a simpler and more efficient means of sharing (for both government and citizens) certain types of information. That said, open data initiatives offer none of the protections or rights of FOI and so this legislation will continue to serve as the fail safe mechanism should a government choose to stop sharing data. Moreover, FOI will continue to be a necessary tool for documents and information that – for all sorts of reasons (privacy, security, cabinet confidence, etc…) cannot fall under the rubric of an open data initiative. So convergence… not for now. But co-existence feels both likely and helpful for both.
eaves.ca 
Pingback: (23:41 09-03-2012) Noticias más populares de #opendata en las ultimas 24 horas | Tuits de Software Libre
Pingback: (23:43 09-03-2012) Noticias más populares de #opengov en las ultimas 24 horas | Tuits de Software Libre
Pingback: (04:20 10-03-2012) Noticias más populares de #opendata en las ultimas 24 horas | Tuits de Software Libre
Pingback: (04:21 10-03-2012) Noticias más populares de #opengov en las ultimas 24 horas | Tuits de Software Libre
Pingback: (08:41 10-03-2012) Noticias más populares de #opengov en las ultimas 24 horas | Tuits de Software Libre
Pingback: (08:40 10-03-2012) Noticias más populares de #opendata en las ultimas 24 horas | Tuits de Software Libre
David,
I still think Freedom of Information Legislation and Open Government Data can converge. (In fact, I think they are so similar, their convergence seems obvious.)
I think you make some mistakes in framing your blog above.
First, you assume the access method defined by FOI laws uniquely have a burdensome “pre-clearance” phase. The reason FOI laws have this is because they are trying to protect classes of information that if accessed would harm some public good (e.g., personal privacy). But Open Government Data has a pre-clearance phase too. The data files must still be stripped of any information that could harm a public good if accessed (e.g. personal information must be removed; information that could harm a criminal investigation must be removed). So Open Government Data also has “a burdensome process” that just goes on before the data gets posted on the portal.
Second, about “tearing down antiquated intellectual property regimes”. Are you assuming that Open Government Data uniquely aspires to this? In some jurisdictions, the Open Government Licenses apply to information acquired through FOI. According to a spokesperson from the National Archives in the UK, “Once material has been released under [freedom of information]/[Environmental Information Regulations] then in most cases it can be re-used under the Open Government Licence.” (June 2011).
I may be wrong but are you imagining Open Government Data as providing a continual stream of data while FOI laws require people to constantly order info from the government? If so, it’s important to note that some FOI laws (e.g., Ontario and Alberta) have “continual access clauses” which states “The applicant may indicate in the request that it shall, if granted,
continue to have effect for a specified period of up to two years.” So I could say to an Ontario Hospital “Every three months for the next two years, send me CSV statistical reports of C. difficle infections” and they would have a legal obligation to do so. If they post the info on-line for everyone, then we have nice data-portal.
As you said, what FOI legislation uniquely offers are legal rights. And it also offers an independent third-party (e.g., the Information Commissioner of Canada) to investigate allegations of government non-compliance with their legal obligations. It also offers recourse to the courts for judicial review, if necessary.
So I actually think FOI legislation and Open Government Data are siblings, which is why they need to be compared more.
Mark
Pingback: (12:42 10-03-2012) Noticias más populares de #opendata en las ultimas 24 horas | Tuits de Software Libre
Pingback: (13:25 10-03-2012) Noticias más populares de #opengov en las ultimas 24 horas | Tuits de Software Libre
Hi Mark – I feel like you’ve altered the terms of the debate here. Your
initial question wasn’t about whether or not FOI and Open Data are
siblings (they are). Or if they should be compared more (I think they
should). It wasn’t even if there is overlap (there is). The question
was, Will they converge? That is a very different question. (One to
which, apparently, the answer is obvious, and I got it wrong).
Your concluding statement – “So I actually think FOI legislation and
Open Government Data are siblings, which is why they need to be
compared more” – implies I believe otherwise, when in reality my posts
are about a) how the two systems support one another, and b) teasing out
what makes them different. Implicit in such an analysis is that there
is a great deal that makes them similar. Most notably, as I note early
on, they share a common goal.
As a result it feels like a lot of
what I wrote has been misconstrued in your response. I never said that
FOI has a uniquely burdensome pre-clearance phase; rather, I was noting
that with FOI information, once a document changes, it may have to be
re-cleared, whereas data in an open data portal is permanently cleared. I
am noting there is a difference between a process principally designed
to clear specific documents versus one designed to
clear a system or process. FOI tends to focus on
the former, open data on the latter. I make no claim about the
difficulty or burden of pre-clearance in either case.
There are clearly places of overlap – as in the examples of Ontario and
Alberta that you mention. But even here, I think there are important
differences. The Open Data movement is about fostering processes and
culture in government that encourage systems to be able to output data
by design. E.g. The default is that data created by systems should be
open. Whereas the culture and process that many people feel FOI has
fostered is a culture where the default is closed until someone
expressly and accurately (since FOI requests are all about getting the
wording right) asks for the right piece of information. To be clear, I’m
not claiming that open data will succeed in its effort – but it is an
important distinction. A great example of this might be public servant
salaries: every year, this piece of information gets requested, and
every year it must be vetted to ensure it complies with privacy and
other policies/legislation. But once it is understood that that
information is going to appear on an open data portal on an annual (or
more frequent) basis, then the design question shifts. Rather than,
“Let’s clear this information for release” the question becomes, how do
we redesign our systems make it easy to regularly (e.g. cheaply) extract
this information? Such thinking can, and ultimately will (I hope)
permeate into the management of documents as well (I’ve advocated in the
past that I’d love GCPEDIA edits to become public on a 5 year delay)
but they will still likely need to be reviewed individually.
There
is, of course, nothing that would make me happier than to have FOI
requests responded to with a decision to have the relevant data added to
an Open Data portal. In part because it would improve access, but more
importantly, because it would cause the affected department to rethink
its information architecture.
So I think we are in strong
agreement, which is why it is hard to read that you think I’m mistaken.
Maybe this is all due to a difference in understanding of the word
convergence.
Hi David,
I don’t see this as a debate but rather a dialogue. I’m sorry for misconstruing your position as I try to come to a better understanding. I hope readers can be sympathetic that dialogue isn’t often perfect.
I agree that “converging” is a vague word. What I am saying is that the momentum of language won’t let us maintain arbitrary barriers between FOI and OGD.
What I see coming through in your writing are descriptions of differences between FOI & OGD but I think those differences aren’t really there – or at least don’t have to be there. For example, you wrote: “I am noting there is a difference between a process principally designed to clear specific documents versus one designed to clear a system or process. FOI tends to focus on the former, open data on the latter.”
First, let me take one step back to clear up an assumption: FOI legislation is not necessarily “document” oriented. According to the Access to Information Act, it is “record”-oriented. And the courts require “records” to be interpreted broadly. People do order databases via FOI. (I recently got a CSV dump of a database at Library and Archives Canada.)
If the principal distinction you are making is “clearing” information for general access, I understand you to be saying that FOI focuses on clearing records whereas OGD focuses on clearing systems or subsystems so their records or information can be accessed publicly. But if a system or a sub-system has no information that could reasonably cause harm to a public good, then FOI provides a legal framework that says that the information is accessible (without regard to top-down policies.)
And with Ontario/Alberta’s FOI laws and their “continuing access”
clauses, people can create a scheduled flow of information that accesses on-going information about institutional processes for up to two years. Ontario’s continuing access clause is section 24(4) http://www.e-laws.gov.on.ca/html/statutes/english/elaws_statutes_90f31_e.htm#BK33
And Alberta’s continuing access clause is section 9 http://www.qp.alberta.ca/574.cfm?page=F25.cfm&leg_type=Acts&isbncln=9780779761951&display=html
So
Second, FOI doesn’t require on-going inspection of everything. If someone get’s a dataset through the method of access defined by a FOI law that clearly has no exemptible information, then the bureaucracy shouldn’t be spending time on reviewing its contents as it would an internal 500 page report. If they do, then it’s the fault of the institution and not the legislation.
You also wrote “the Open Data movement is about fostering processes and culture in government that encourage systems to be able to output data by design. E.g. The default is that data created by systems should be open. Whereas the culture and process that many people feel FOI has fostered is a culture where the default is closed until someone expressly and accurately (since FOI requests are all about getting the wording right) asks for the right piece of information.”
Again, I think you are mistaken in suggesting that FOI has fostered ‘a culture where the default is closed’. FOI legislation creates a legal framework that sets information held by the Gov’t to open by default (section 4 of the Access to Informaiton Act: http://laws-lois.justice.gc.ca/eng/acts/A-1/page-2.html#h-6) and limits that general right of access in exceptional circumstances where general access would cause harm – these are the exemptions: sections 13-23: http://laws-lois.justice.gc.ca/eng/acts/A-1
If institutions are engaging in a practice of withholding information, I don’t see how that is the fault of the FOI legislation passed by Parliament rather than the information and communication policies set by Cabinet.
Moreover some FOI legislation also have clauses that require departments to proactively make classes of available. In the United Kingdom it is called a “publication scheme”. Here is the clause in the Scottish FOI law:
23 Publication schemes(1)A Scottish public authority must—
(a)adopt and maintain a scheme (in this Act referred to as a “publication scheme”) which relates to the publication of information by the authority and is approved by the Commissioner;
(b)publish information in accordance with that scheme; and
(c)from time to time review that scheme.
(2)A publication scheme must specify—
(a)classes of information which the authority publishes or intends to publish;
http://www.legislation.gov.uk/asp/2002/13/part/1/crossheading/publication-schemes
The Access to Information Act, which was passed in 1982, has a very limited “proactive publication scheme” (section 5; http://laws-lois.justice.gc.ca/eng/acts/A-1/page-3.html#h-7
Mark Weiler
Pingback: Open Data Movement is a Joke? | eaves.ca