Tag Archives: procurement

Government, Digital Services & IT Procurement Reform

1 Reply

Next semester I’ll be teaching a course on why healthcare.gov initially turned into a disaster. Why? Because sadly, the failure of healthcare.gov was not special. Conservative estimates suggest over half of all IT projects are not completed on time or on budget. Others suggest the numbers are higher still. What did make healthcare.gov special was that its failure was so spectacularly public. As a result, unlike so many IT failures that stay hidden from public view, healthcare.gov is, somewhat paradoxically, safe to talk about. This makes it a wonderful case study to explore why so many large IT projects end in disaster.

Thinking about the course has me diving deeper into government IT procurement. Many of my colleagues in the civic tech space are convinced that procurement reform is critical to improving government IT services and fostering a better eco-system of IT vendors. I agree there is a great deal of good work that could be done to make procurement simpler and more effective. However, procurement is not a silver bullet. Indeed, I suspect that procurement reform, on its own, will have a limited impact, because it is not the central problem.

What procurement thinks it is trying to solve

There are two broad goals of government IT procurement policies that I think often get conflated.

One set of rules try to constrain and/or force people to make good technicaldecisions. Is the solution accessible? Is it secure? Does it protect privacy? etc…

The second goal is to constrain and/or force people to make good processdecisions. This is about ensuring fairness, accountability and broadly to prevent corruption. Did you get three bids? Have you laid out the specifications? Are you related to any of the vendors? Have the vendors donated money to any politicians? etc…

Both sets of rules have unintended consequences that make procurement slow and more difficult (although for many governments this can be a feature, not a bug, and making spending more difficult can be a net positive at a system level even if frustrating at the unit level).

The underlying assumption

Unpack these two goals — and particularly the first one — and you discover two underlying assumptions:

  1. IT implementations are treated like technical not adaptive challenges

The first assumption is that IT should be commoditized. While some IT purchases may be similar to buying a pencil, most are not. And if you are talking about an IT purchase that is custom build or the cost is north of $1 million, this is almost definitely not the case. Large IT implementations are complex with significant unknowns. It is impossible (despite organizations efforts) to spec out how all the problems will be solved in advance. Moreover this work takes place in dynamic environments where assumptions about the tech it will operate on, how users will interface with it and even what it product will need to do are dynamic. We treat IT projects like they are technical challenges — that we know exactly what must be done and that it can be all mapped out in advance — but they are almost always adaptive problems, where we don’t even know what all the problems are at the beginning of the process.

2. Effective process can force good decision making (or at least constrain bad ones)

Think of all the questions an enterprise — government or for profit — needs to assess: Will a solution work? Does it have a good user experience for our users? Can that UX evolve? Can the vendor adapt to unknown problems? Can they integrate with our current environment? To our future environment? Plus hundreds of other issues, all of which require nuance and knowledge.

But any set of procurement rules is about standardization process — so that the process can be evaluated, not the outcome. And this makes it harder to bring to bear these nuanced decision and knowledge because nuance, by definition, is hard to standardize. I fear that implicit in procurement reform is the belief that a good set of policies can design a process that, irregardless of who runs it, will prevent disaster and possibly even ensure an optimal outcome. If we assume procurement problems are technical problems for which the appropriate solution must merely be identified, then with the right “code” the machinery of procurement — regardless of who is manning it — should be able to select the optimal solution.

Both assumptions are, of course, patently ridiculous.

This is why I’m not confident that tinkering with the rules of procurement in the IT sector will generate much change. I certainly don’t think it will foster a new eco-system of service providers who will provide solutions that don’t fail 50% of the time. All the tinkering in the world won’t change the underlying issues — which is more than anything else: more than rules, oversight or the size of the ecosystem, is capacity of the people of running the procurement to evaluate technology and vendors capacity that matters.

I’ve seen terrible, terrible IT choices made by organizations and inspired decisions (like California’s Child Welfare Services RFP on which I’m writing a case study) be produced by virtually identical procurement rules and policies. The current rule set can allow a determined actor in government to make good choices. Could we do a lot better? Absolutely. But it is not the defining barrier.

This is, again, why I think USDS and similar digital service groups that try to attract talent that has worked on mass market consumer technology matter. Recruiting top technology talent into government is the single best strategy for ensuring better procurement. Deeply experienced people stuck with an okay rule set will be better than less experienced people stuck with an okay rule set.

And vendors generally agree. As a vendor, what you want more than anything is a capable, intelligent and knowledgeable buyer, not a set of rules that must be adhered to no matter the context.

And, to be clear, this is how the private sector does it. Ask Peter Thiel, Mark Zuckerberg, Elon Musk or Paul Graham (especially Paul Graham). None of them would outsource the technology choices for their start up or $1B dollar unicorn to a group of procurement officers equipped with even the most perfect set of procurement rules. Quite the contrary. They spend millions in stock options and money hiring amazing talent to make highly informed decisions to meet needs in a fluid and evolving environment.

So we should do Procurement Reform?

We should. We definitely should. But let’s recognize the limits. We should have rules that prevent corruption, nepotism or competitive practices. But unshackling people of rules may equip them to make more bad decisions as it does good ones. Let’s make rules that help people buy software and don’t make the process a burden, but our defense against poor outcomes shouldn’t be more rules, it should be an investment in pulling the brightest and best minds into government.

This is a problem I think digital service groups are trying to solve — creating a pipeline of talent that has worked with the latest tech and delivered solutions that have served hundreds of millions of users, flowing through government. So let’s just ensure that we invest in both. Procurement reform without the right people won’t get us there.

Government Procurement Reform – It matters

1 Reply

Earlier this week I posted a slidecast on my talk to Canada’s Access to Information Commissioners about how, as they do their work, they need to look deeper into the government “stack.”

My core argument was how decisions about what information gets made accessible is no longer best managed at the end of a policy development or program delivery process but rather should be embedded in it. This means monkeying around and ensuring there is capacity to export government information and data from the tools (e.g. software) government uses every day. Logically, this means monkeying around in procurement policy (see slide below) since that is where the specs for the tools public servants use get set. Trying to bake “access” into processes after the software has been chosen is, well, often an expensive nightmare.

Gov stack

Privately, one participant from a police force, came up to me afterward and said that I was simply guiding people to another problem – procurement. He is right. I am. Almost everyone I talk to in government feels like procurement is broken. I’ve said as much myself in the past. Clay Johnson is someone who has thought about this more than others, here he is below at the Code for America Summit with a great slide (and talk) about how the current government procurement regime rewards all the wrong behaviours and often, all the wrong players.

Clay Risk profile

So yes, I’m pushing the RTI and open data community to think about procurement on purpose. Procurement is borked. Badly. Not just from a wasting tax dollars money perspective, or even just from a service delivery perspective, but also because it doesn’t serve the goals of transparency well. Quite the opposite. More importantly, it isn’t going to get fixed until more people start pointing out that it is broken and start contributing to solving this major bottle neck of a problem.

I highly, highly recommend reading Clay Johnson’s and Harper Reed’s opinion piece in today’s New York Times about procurement titled Why the Government Never Gets Tech Right.

All of this becomes more important if the White House’s (and other governments’ at all levels) have any hope of executing on their digital strategies (image below).  There is going to be a giant effort to digitize much of what governments do and a huge number of opportunities for finding efficiencies and improving services is going to come from this. However, if all of this depends on multi-million (or worse 10 or 100 million) dollar systems and websites we are, to put it frankly, screwed. The future of government isn’t to be (continue to be?) taken over by some massive SAP implementation that is so rigid and controlled it gives governments almost no opportunity to innovate. And this is the future our procurement policies steer us toward. A future with only a tiny handful of possible vendors, a high risk of project failure and highly rigid and frail systems that are expensive to adapt.

Worse there is no easy path here. I don’t see anyone doing procurement right. So we are going to have to dive into a thorny, tough problem. However, the more governments that try to tackle it in radical ways, the faster we can learn some new and interesting lessons.

Open Data WH

Here's a prediction: A Canadian F-35 will be shot down by a drone in 2035

1 Reply

One of the problems with living in a country like Canada is that certain people become the default person on certain issues. It’s a small place and the opportunity for specialization (and brand building) is small, so you can expect people to go back to the same well a fair bit on certain issues. I know, when it comes to Open Data, I can often be that well.

Yesterday’s article by Jack Granastein – one of the country’s favourite commentator’s on (and cheerleaders of) all things military – is a great case in point. It’s also a wonderful example of an article that is not designed to answer deep questions, but merely reassure readers not to question anything.

For those not in the know, Canada is in the midst of a scandal around the procurement of new fighter jets which, it turns out, the government not only chose to single source, but has been caught lying misleading the public about the costs despite repeated attempts by both the opposition and the media to ask for the full cost. Turns out the plans will cost twice as much as previously revealed, maybe more. For those interested in reading a case study in how not to do government procurement Andrew Coyne offers a good review in his two latest columns here and here. (Granastein, in the past, has followed the government script, using the radically low-ball figure of $16 billion, it is now accepted to be $26 billion).

Here is why Jack Granastein’s piece is so puzzling. The fact is, there really aren’t that many articles about whether the F-35 is the right plane or not. People are incensed about being radically mislead about the cost and the sole source process – not that we chose the F-35. But Granastein’s piece is all about assuring us that a) a lot of thought has gone into this choice and b) we shouldn’t really blame the military planners (nor apparently, the politicians). It is the public servants fault. So, some thoughts.

These are some disturbing and confusing conclusions. I have to say, it is very, very depressing to read someone as seasoned and knowledgeable as Granastein write:

But the estimates of costs, and the spin that has so exercised the Auditor-General, the media and the Opposition, are shaped and massaged by the deputy minister, in effect DND’s chief financial officer, who advises the minister of national defence.

Errr….Really? I think they are shaped by them at the direction or with the approval of the Minister of Defence. I agree that the Minister and Cabinet probably are not up to speed on the latest in airframe technology and so probably aren’t hand picking the fighter plane. But you know what they are up to speed on? Spinning budgets and political messages to sell to the public. To somehow try to deflect the blame onto the public servants feels, well, like yet another death nail for the notion of ministerial accountability.

But even Granastein’s love of the F-35 is hard to grasp. Apparently:

“we cannot see into the future, and we do not know what challenges we might face. Who foresaw Canadian fighters participating in Kosovo a dozen years ago? Who anticipated the Libyan campaign?”

I’m not sure I want to live and die on those examples. I mean in Libya alone our CF-18’s were joined by F-16s, Rafale fighters, Mirage 2000s and Mirage 2000Ds, Tornados, Eurofighter Typhoons, and JAS 39C Gripen (are you bored yet?). Apparently there were at least 7 other choices that would have worked out okay for the mission. The Kosovo mission had an even wider assortment of planes. Apparently, this isn’t a choice of getting it “just right” more like, “there are a lot of options that will work.”

But looking into the future there are some solid and strong predictions we can make:

1) Granastein himself argued in 2010 that performing sovereignty patrols in the arctic is one of the reasons we need to buy new planes. Here is a known future scenario. So frankly I’m surprised he’s bullish on the F-35s since the F-35’s will not be able to operate in the arctic for at least 5 years and may not for even longer. Given that, in that same article, Granastein swallowed the now revealed to be bogus total cost of owernship figures provided by the Department of National Defence hook, line and sinke, you think he might be more skeptical about other facts. Apparently not.

2) We can’t predict the future. I agree. But I’m going to make a prediction anyway. If Canada fights an enemy with any of the sophistication that would require us to have the F-35 (say, a China in 25 years) I predict that an F-35 will get shot down by a pilotless drone in that conflict.

What makes drones so interesting is that because they don’t have to have pilots they can be smaller, faster and more maneuverable. Indeed in the 1970s UAVs were able to outmaneuver the best US pilots of the day. Moreover, the world of aviation may change very quickly in the coming years. Everyone will tell you a drone can’t beat a piloted plane. This is almost likely true today (although a pilot-less drone almost shot down a Mig in 2002 in Iraq).

But may have two things going for them. First, if drones become cheaper to build and operate, and you don’t have to worry about losing the expensive pilot, you may be able to make up for competency with numbers. Imagining an F-35 defeating a single drone – such as the US Navy’s experimental X-47B – is easy. What about defeating a swarm of 5 of them that are working seamlessly together?

Second, much like nature, survival frequently favours those who can reproduce frequently. The F-35 is expected to last Canada 30-35 years. Yes there will be upgrades and changes, but that is a slow evolutionary pace. In that time, I suspect we’ll see somewhere between 5 (and likely a lot more) generations of drones. And why not? There are no pilots to retrain, just new lessons from the previous generation of drones to draw from, and new technological and geo-political realities to adapt to.

I’m not even beginning to argue that air-to-air combat capable drones are available today, but it isn’t unlikely that they could be available in 5-10 years. Of course, many air forces hate talking about this because, well, drones mean no more pilots and air forces are composed of… well… pilots. But it does suggest that Canada could buy a fighter that is much cheaper, would still enable us to participate in missions like Kosovo and Libya, without locking us into a 30-35 year commitment at the very moment the military aerospace industry is entering what is possibly the most disruptive period in its history.

It would seem that, at the very least, since we’ve been mislead about pretty much everything involved in this project, asking these questions now feels like fair game.

(Oh, and as an aside, as we decide to pay somewhere between $26-44 Billion for fighter planes, our government cut the entire $5 million year budget of the National Aboriginal Health Organization which over research and programs, in areas like suicide prevention, tobacco cessation, housing and midwifery. While today Canada ranks 6th in the world in the UN’s Quality of Life index, it was calculated that in 2007 Canada’s first nation’s population, had they been ranked as a separate group, would have ranked 63rd. Right above healthy countries like Belarus, Russia and Libya. Well at least now we’ll have less data about the problem, which means we won’t know to worry about it.)