The digital revolution holds the potential to use mere “data” to improve the budget and the economy, and hence our businesses and our lives. But you have to wonder whether our politicians are up to the challenge.
In a speech last week, the Australian Statistician, boss of the Australian Bureau of Statistics, David Kalisch, said the new statistical frontier is “data integration” – you take two or more separate sets of statistics and put them together in ways that reveal new information. Things you didn’t know about how bits of the world work.
This is just exploring the huge, still largely untapped potential of computers to manipulate a lot of figures and produce useful information about what’s going on in this field or that. But it also involves new statistical techniques for combining data in ways that make sense and don’t mislead.
(This, BTW, raises a bugbear of mine. Digitisation, which allows us to measure any number of aspects of a company’s performance cheaply and easily, has given rise to the enthusiasm for “metrics”. But bosses who allow their metrics to be chosen and presented by people who know a lot about IT but nothing about the science of statistics, or who draw conclusions from those metrics without any knowledge of stats, are asking to be led up the garden path. They never know when the metric is answering a different question to what they imagine.)
Kalisch says data integration is already delivering new insights, such as improved estimates of Indigenous life expectancy, understanding outcomes for successive cohorts of migrants, and the importance of small to medium enterprises for job creation (not as outstanding as the propaganda would lead you to expect).
There’s much more of that kind of thing we could do. But Kalisch points also to the considerable untapped potential to use data integration to assess the performance of government policies and programs, and thus to target budget funding to programs assessed as more likely to be effective.
Kalisch says “Australia does not have a strong tradition of rigorously evaluating outcomes of government programs and policies”. That’s putting it politely. The Americans do (because Congress insists on it) and so do many other countries – even those backward and poverty-stricken Kiwis do.
Why don’t we? Because too many ministers and department heads fear the embarrassment if rigorous assessment showed a program was a waste of money, as many would. And also because Treasury and Finance don’t bother pushing it – perhaps because program evaluation costs money upfront, and only saves money down the track.
But that’s only one reason we risk failing to exploit all the benefits of big data analysis. The biggest is the very real probability bully-boy politicians and over-zealous agency heads try to ram through data aggregation schemes over the worries of people concerned about breaches of their privacy.
Consider the hash they’re making of My Health Record where, among other things, the instigators are relying more on slick ads than honest explanation. Consider the long running attempt by the masterful Alan Tudge, the department and the Centrelink PR man to deny there was any problem with robodebt, until the full extent of the fiasco – and the hurt it caused many innocent victims – could no longer be concealed.
Then consider the way Tudge used the shield of Parliament to reveal very private information about a woman who'd had the temerity to criticise him. And he escaped uncensured.
Such episodes, and many years of spin doctor-led politicians playing the true-but-misleading game, have hugely reduced the public’s trust in politicians and their happy assurances that nothing could possibly go wrong.
We stand on the cusp of reaping huge benefits from data analysis, or stuffing it up so badly the electorate punishes any government that touches it.
Part of this is the risk that government penny-pinching doesn’t give the data gatherers enough funding to install adequate privacy safeguards, or enough resources to respond honestly and adequately to the public’s questioning.
But that’s just part of a bigger money question: data integration isn’t particularly dear relative to the benefits of greater understanding, better public policy and more effective government spending it offers, but that doesn’t mean the pollies have the sense to cough up.
Operational funding of our bureau of statistics has been cut by 30 per cent in real terms over the past decade, by governments of both colours.
An independent benchmarking exercise in 2016 found that our bureau’s funding was about half the funding provided to Statistics Canada for roughly equivalent work. Even New Zealand’s official statistician got more than ours did. Smart thinking.