The NSA leaks crashed the global open government party. Don’t get me wrong – I do not think this undermines the “whole idea of open government” as someone recently claimed to me. But the open government community cannot sit on the sidelines. It forces us to think hard about two related issues that we have avoided talking about up until now.

These are:

(1) The privacy implications of open government: I feel the community has  underemphasized privacy concerns. For example, I can’t think of any open government event I’ve been to that has put a serious spotlight on the issue. Now, thanks to Snowden, we can no longer avoid it.

This applies to the privacy implications of open data, where we’ve seen the beginnings of an interesting debate just recently. See the work of the Web Foundation’s Open Data Research Network and the Open Knowledge Foundation which – rightly – suggest we should develop a set of privacy principles for open data.

This also applies to the broader open government movement. For example, campaigners pushing for greater transparency around financial flows (e.g. revenues from extractive industries, budget expenditures, tax transparency) might benefit from explaining how these data/information releases will adequately take account of privacy concerns where relevant. A privacy ‘check list’ might help to sharpen discussions about what information should be disclosed and how? (Incidentally, such a check list could also help funders when assessing grant applications.)

There is a tension here: we need to bring people together, work it over and see where we can join forces. This is another great opportunity to bust silos and join up the movement.

(2) The transparency of government surveillance efforts: for a long time, a small group of committed transparency advocates have been agitating on the issue. Back in 2011 (pre Snowden), the Opening Government publication included a section on ‘national security transparency and accountability’, the Center for Democracy and Technology and other groups (esp. in the US) have long campaigned on the issue. But the issue is not part of the mainstream open government community. It should be.

Here is an opportunity for the open government community to reach out to the groups that have developed expertise in the area (e.g. Privacy International), share what they have learned and see how we might work together. Open government groups and civil liberties campaigners sit on the same side of the fence in this case: this could be a powerful alliance. Groups campaigning for the transparency of government surveillance efforts are part of the global transparency movement. We should embrace that.

I’ve been puzzling for a while how the open data community can help the many great groups that have been fighting for transparency of key money flows for the past decade and more. I think one answer may be that open data helps us go beyond simply making information available. If done well, it can help us make it accessible and relevant to people, which has been the holy grail for transparency advocates for a long time.

The transparency community has focused too much on just getting information out there (making information available). But what’s the point of having information available if it’s not accessible? What’s the use of public reports that are only nominally ‘public’ because they languish in filing cabinets or ‘PDF deserts’ hidden within an obscure website?

If we can get this information more accessible, we can then work to increase participation and help people use it. This for me is what open data people are talking about when they talk about open formats. Machine readability and open formats matter because they are tools to increase access. I’ve seen too many techies talk about ‘open formats’ and activists’ eyes glaze over. But I think we’re both talking about the same thing we hold dear: improving access to vital data for all.

Likewise, it’s the connections between the datasets that are powerful and interesting. You may not care so much to know where most people under 15 years old live in your country, but if you’re told that those that live close to a nuclear waste disposal site happen to have the highest cancer rates, then it becomes seriously relevant. Same as above, techies often talk about technical data standards and get quizzical/skeptical – at best – looks in exchange. But technical data standards are the fuel that allows policy wonks to compare datasets, which creates relevant data. Connecting the dots makes it policy relevant – without data, you can’t make policy.

[availability of data] => [accessibility of data] => [comparability of data]

[availability of data] => [open formats] => [data standards]

Follow the Money groups do amazing work: extractives’ transparency advocates campaigning for vital releases of information on oil, gas, mining revenues into the hundreds of millions of dollars. Groups looking at curbing illicit flows of funds out of desperately poor countries via shell companies and phantom firms. Activists who scrutinize budgets, everything from big ticket national budget allocations, all the way down to very local issues like your local school spending on basic reading materials. And many more.

Together, these groups share one big thing in common – they are all seeking to follow the money. In other words, they are all trying to understand how money either gets in to government coffers, or how it fails to get there, and then how and whether it is spent for the good of the many, rather than the few lining their pockets.

To succeed, we all need data that’s not only public (e.g. public registries of beneficial ownership) but also accessible (in open formats) and comparable to other money flows.

Let’s work together to make it happen.

This is republished from my article in the Telegraph:

Trust in government has rarely been at a lower ebb. Citizens in developed and developing countries alike feel increasingly disconnected from the political process and their political leaders. They complain of having too little influence over decisions, too little access to government information and too little control over their own data. In such an environment, suspicion and anger can erupt as we have seen across the world, and most recently in Istanbul’s Taksim square.

At the same time, governments are operating in very challenging circumstances. They have to meet rising expectations from their citizens with, thanks to the impact of the global financial crisis, often severely reduced revenues. They also face a whole range of pressures such as a demographic change which will make bridging this gap ever more difficult. There has never been a greater need for open and honest dialogue and for as wide a possible involvement in arriving at the right innovative solutions.

There is no single answer to these concerns. But it is clear that opening up government data must be a major element of the answer.  Open data has enormous potential to drive economic growth and spread prosperity. It improves accountability, strengthens governance, builds trust and drives innovation in both the private sector and the delivery of key public services.

There are already many examples from around the world that these benefits are already being delivered. In the UK, Mastodon C, a start-up incubated by the Open Data Institute, used open data on prescriptions by GPs to show that the NHS could have saved over £200 million by prescribing generic drugs instead of their more expensive patented equivalents.

In India, the technology platform I Paid A Bribe enables citizens to publicly log whenever they have been shaken down for a bribe. In Mexico, Compara Tu Escuela (Check Your School) empowers parents by providing them directly with information on school performance.

We all benefit as citizens and consumers, as economies and societies, if we get this right.  It is why the expected decision by the G8 countries to adopt an Open Data Charter at the G8 summit in Lough Erne is so important.

This would be an historic move and a genuine breakthrough for the growing movement to open up government and private sector data. It should mean that all G8 members agree to publish as much information as possible in a form, which is useable and accessible to all.

While there should be, rightly, exceptions to protect personal information, the default position should change from non-publication to publication of government data in a way which is timely, comprehensive, clear and does not impose costs on its users.

Such a move would fit in well with the wider agenda of the G8 this year, which is about reforming trade and bringing greater fairness and transparency to international taxation. Countries have understood they all benefit if there is much less secrecy about who pays what tax and where.

Agreeing the Charter, however, is only a first step if countries and citizens are to benefit from the decision to open up and democratise information. First, of course, we need to see governments meet not just the letter but also the spirit of the charter. Opening up data should cut across all G8 priorities, including tax, transparency on extractive industries, aid and land deals. Information that is released as open data and is fully searchable will enable governments and civil society to crack down on tax evasion.

This will include governments having the courage to realise that opening up will lead to calls for reform. They have a great deal to gain from encouraging greater engagement and participation in policy debates and decisions. They will also need to balance citizens’ concerns with privacy and the need for transparency of government data.

Secondly, we also need companies to follow the G8’s lead. They must consider how they can make as much of their data as possible open to the public. Thanks to recent European Commission directives and the US Dodd-Frank Act, extractive companies are required to be increasingly transparent on the revenues they pay to national government. All companies are sitting on a goldmine of information which is not commercially sensitive but which can help provide the missing piece of the jigsaw.

The more actors which open up data the greater the benefits will be. In just 20 months, the Open Government Partnership, which the UK currently co-chairs, has grown into a global movement comprising 59 members. They too must consider how they can make as much of their data as possible open to the public.

Finally, we need to see actual impact on people’s lives as a result of all this information. Overcoming barriers, formal and informal, to publishing data is only one half of the battle.  We need to help research institutions, civil society organisations, the private sector and individuals work together to find solutions to local, national and global problems thanks to this data.

Technology platforms have already allowed us to harness the collective power and wisdom of thousands to improve policies and public services. Open data can help us drive the next stage of this revolution, and enable millions to have their voices heard and save precious government funds.

I worry that civil society advocates working on Open Government Partnership are making a tactical mistake.

There has been a lot of activity – rightly – around which OGP countries should be ‘in or out’. There were discussions in the past year around South Africa’s media bill (the so called ‘secrecy bill’) and whether it might impact the country’s OGP eligibility. Most recently the discussion has centered on Russia’s decision to ‘postpone’ its entry into OGP. Many had informally questioned whether Russia should have been eligible in the first place.

Whilst important, such a strong focus on eligibility misunderstands the nature of the Partnership. The Open Government Partnership is not a ‘good performers’ club’. If it was, it would entail setting a high bar for entry and focusing civil society attention on getting new countries in to meet the entry standard and monitoring those that fall behind with a view to expelling them. OGP is different. It purposefully sets a low bar for entry and then seeks to encourage countries in a ‘race to the top’ by rewarding excellence and penalizing backsliding or inaction. Here is where as a community we could do a lot more to ensure OGP succeeds in these precious formative years.

There are currently three basic rules of the game for OGP – we should refine and strengthen these and I would suggest to also add a ‘relevance check’.

(1) Civil society participation: this is the defining factor of the Open Government Partnership. Civil society sit on the OGP steering committee, are represented at co-chair level, are involved in the drafting, co-creation and implementation of OGP national action plans. Yet the OGP guidance on participation of civil society (which the Independent Reporting Mechanism uses as the standard to measure countries against) is far too broad, weak and focuses only on the drafting of a country’s initial action plan.There are just five basic steps that focus on ‘consultation’: we could and should do better. There is so much more we could be doing here, from using platforms like OpenIdeo to co-create action plans, to setting up innovative civil society/government/private sector collaborations to make them happen. More on this in a future post.

(2) The OGP ‘stretch’: no country action plan should merely coast on past successes, or bottle prior commitments under the pretense of new. The only reason to be part of OGP is  to ‘stretch’, to innovate and try something new, different, possibly uncomfortable at first. This idea was present from the very genesis of the Partnership – countries should make ‘stretch commitments’ in their action plans that take them beyond their comfort zone into new territory. It is then that OGP really starts to makes sense: countries then need support from their peers to make it happen, the networking mechanism matches idea/innovator/implementer etc. We need to be much much clearer on defining what ‘stretch’ means. More on this shortly.

(3) A ‘relevance check’: even if we get the above two right, there will always be countries where the finished product – the country action plan – may be completely off the mark and perhaps not even have much to do with open government at all (e.g. ‘faster marriages for pregnant women’, ‘cleaner beaches’, ‘tweets about drug traffickers‘) ! This could happen for a number of reasons (we failed to connect with open government reformers, civil society was not engaged etc.). At present, there is no safeguard: an action plan is finalised and is put into the system, no questions asked. We need a better relevance check.

(4) The Independent Reporting Mechanism (IRM): this is where the heart of the action has taken place so far. OGP watchers will have a view on this, but regardless of how well we do on the IRM and how much it incentivises government and civil society to implement better open government commitments, it will always be post-hoc. By definition, the IRM comes after the action plans have been designed, implemented etc.  So we need to worry about the totality of the process described above as well as engage with the reporting mechanism.

This won’t be enough – these are necessary but not sufficient rules. But if we can at least get these right, we will I hope have helped towards building and iterating towards an even better Open Government Partnership that delivers meaningful change.

Comments are valued, please let me know your views.

Am sure no-one missed the White House’s executive order on Open Data but have found a quick list of relevant links to be helpful:

From the White House

http://www.whitehouse.gov/the-press-office/2013/05/09/executive-order-making-open-and-machine-readable-new-default-government-

From Sunlight Foundation

http://sunlightfoundation.com/blog/2013/05/09/open-data-executive-order-shows-path-forward/

From openthegovernment.org

http://www.openthegovernment.org/node/3975

From Personal Democracy Forum

http://techpresident.com/news/23860/president-obama-sets-government-default-open

The Guardian on comparing US and UK approaches to open data

http://www.guardian.co.uk/news/datablog/2013/may/10/us-open-data-uk-comparison

The actual memorandum is here

Click to access m-13-13.pdf

Todd Park’s blog on the White House site

http://www.whitehouse.gov/blog/2013/05/09/landmark-steps-liberate-open-data

The data.gov blog that gives helpful background on related innovations going on

http://www.data.gov/blog/under-hood-open-data-engine

And the You Tube version! is here

Really great blog by David Eaves on open data http://eaves.ca/2013/04/25/the-value-of-open-data-dont-measure-growth-measure-destruction/

“Look at CKAN and Socrata. Most people believe these are open data portal solutions. That is a mistake. These are data management companies that happen to have simply made “sharing (or “open”) a core design feature.”

Another good article on open data: “open data only exists in relation to a broader information system that gives it meaning: open data as a thing-in-itself does not exist in the real world.”

What does the U.S. taxpayer get for that $50 billion? great post on IATI by Publish What You Fund US Advisor Rodney Bent http://sunlightfoundation.com/blog/2013/04/12/opengov-voices-bring-that-lantern-over-here-why-budget-transparency-matters/ …

The investees of Global Integrity’s TESTING 123 program are now announced. Nice idea in that they have few investees (5) and small grants ($10,000) so should be able to dig deep and provide real support.

http://innovation.globalintegrity.org/

Check out the Center for Global Development’s parallel Fermanagh Declaration, highlighting what success at the G8 could look like

Click to access FermanaghDeclaration.pdf

Interesting from TechPresident on NYC BigApps “re-inventing” the Hackathon

http://techpresident.com/news/23800/nyc-bigapps-refines-civic-hackathon