How the digital world is getting better at measuring us up

These days we hear incessantly about “data”. The media is full of reports of new data about this or that, and there’s a new and growing occupation of data analysts and even data scientists. So, what is data, where does it come from, what are people doing with it, and why should I care?

Google “data” and you find it’s “facts and statistics collected together for reference or analysis”. The advent of computers has allowed businesses and governments to record, calculate, play with and store huge amounts of data.

Businesses have data about what goods and services they’re making, buying and selling, importing or exporting, and paying their workers, going back for 30 or 40 years.

Our banks have data about what we earn and what we spend it on, especially when we use a credit or debit card – or our phone – to pay for something.

Much of this data is required to be supplied to government agencies. If you ever go onto the Australia Taxation Office’s website to do your annual tax return, it will offer to “pre-fill” your return with stuff it already knows about your income from wages, bank interest and dividends.

Try it sometime. You’ll be amazed by how much the taxman knows and how accurate his data are.

Another dimension of the “information revolution” is how advances in international telecommunications – including via satellites – have allowed us to be in touch with people and institutions around the world in real-time via email and the web – news, entertainment, social media, whatever.

Last month, the Australian Statistician – aka the boss of the Australian Bureau of Statistics – Dr David Gruen, gave a speech outlining some of the ways these huge banks of “big data” about the economic activities of the nation’s businesses, workers, consumers and governments can be used to improve the way we measure the economy in all its aspects: employment, inflation, gross domestic product and the rest.

We’re getting more information and more accurate information, and we’re getting it much sooner than we used to. But we’re still in the early days of exploiting this opportunity to be better informed about what’s happening in the economy and to have better information to guide the government’s decisions about its policies to improve the economy’s performance.

Gruen starts by describing the Tax Office’s “single-touch” payroll system, software that automatically receives information about employees’ payments every time an employer runs its payroll program.

Not all employers have the software, but those who do account for more than 10 million of our 14 million employees.

Gruen says the arrival of the pandemic in early 2020 made access to this “rich vein of near real-time information” an urgent priority. The taxman pulled out the stops, and the stats bureau began receiving these data in early April 2020.

With a virus spreading through the land and governments ordering lockdowns and border closures, they couldn’t afford to wait a month or more to find out what was happening in the economy. Thus, the whole project of using big data to help measure the economy received an enormous kick along – here and in all the other rich economies.

So, in addition to the longstanding monthly sample survey of the labour force, we now have a new publication: Weekly Payroll Jobs and Wages Australia. These data allowed the econocrats—and the rest of us—to chart the dramatic collapse in jobs across the economy over the three weeks from mid-March 2020.

They show employment in the accommodation and food services industry falling by more than a quarter in just three weeks. Employment in the arts and recreation services industry fell by almost 20 per cent. By contrast, falls in utilities and education and training were minor.

The monthly labour force survey has a sample size of about 50,000 people, compared with the payroll program’s 10 million-plus people, meaning it provides information on far more dimensions of the workforce than the old way does.

So, the bureau’s access to payroll data taught it new ways of doing things. And the pandemic increased econocrats’ appetite for more info about the economy that was available in real-time.

With household consumption – consumer spending – accounting for about half of gross domestic product, improving the timeliness and detail of the data was a great idea.

So, in February 2022, the bureau released the first monthly household spending indicator using (note this) aggregated and de-identified data on credit and debit card transactions supplied by the major banks. This indicator provides two-thirds coverage of household consumption, compared with the less than one-third coverage provided by the usual survey of retail trade.

The bureau has also begun publishing a monthly consumer price index in addition to the usual quarterly index. This is possible because big data – in the form of data from scanners at checkout counters and data scraped from the websites of supermarket chains – is much cheaper to gather than the old way.

The bureau has also started integrating different but related sets of big data from several sources, so analysts can study the behaviour of individual consumers or businesses. It has developed two large integrated data assets.

The one for individuals links families and households with data sets on income and taxation, social support, education, health, migrants and disability.

The one for businesses links them with a host of surveys of aspects of business activity, income and taxation, overseas trade, intellectual property and insolvency.

The purpose is to allow analysts from government departments, universities or think tanks to shed light on policy problems from multiple dimensions.

For instance, one study showed that people over 65 who’d had their third COVID vaccination within the previous three months were 93 per cent less likely to die from the virus than an unvaccinated person.

But that’s just the tiniest example of what we’ll be able to find out.

Read more >>

Speech in the Great Hall of Sydney University

I’m too old to suffer from impostor syndrome, but the thought has occurred to me that, had the University of Sydney’s officials taken a look at my academic transcript at Newcastle University, and seen how much trouble I had persuading that uni to give me a pass degree, we’d be holding this gathering down at Ralph’s cafe in the women’s gym.

The truth is that I had a lot of trouble passing a subject called economics, which I couldn’t make any sense of – perhaps because it didn’t interest me greatly. I failed a subject called international economics but, since it was the last subject I had to go, my lecturer was prevailed upon to give me a conceded pass.

So I have to tell you I’m a bit bemused by a university, of all institutions, making such a fuss about me and my job. I’ve spent much of my time urging the people I’ve helped to hire and train as economic journalists not to write like an academic. Keep it simple, I’d say. Don’t try to impress people with big words. Try to be understood, not to mystify. Now, obviously, that’s not the right advice to be giving an academic.

In my job, I’m paid to have an opinion on everything. And I’m paid to give free advice to everyone, from the prime minister down. And I’m now so much older than my boss I’m allowed to give him – and sometimes her – free advice. She or he, of course, is paid to pretend she greatly values that advice.

So while I’m here in this hallowed hall of learning, let me give the academics two bits of free advice. Some years ago, the federal government’s chief scientist paid good money to get one of those now-infamous four firms of accountants-turned-consultants to fudge up a dollar figure for the value of science to the economy. 

One of my proteges, filling in for me while I was on holidays, Gareth Hutchens, these days a columnist at the ABC, wrote a piece saying the chief scientist had to be kidding. Anyone who wasn’t smart enough to know that our material prosperity was built on technological advance, and that technological advance rested on a bed of pure science, wasn’t someone who’d be impressed by any magic number. Gareth was right, of course.

The point is, academics should never yield to intimidation by those who can see no further than immediate income. Academics must never be ashamed to proclaim their belief in the value of knowledge for its own sake. Knowledge doesn’t have to have a monetary value to be of value. Humans are an inquisitive species. We’d like to know whether the universe is expanding for no better reason than that we’d like to know. And thanks to the material prosperity science has brought us, we can afford to pay some scientist to find out for us.

My second bit of free advice is that universities should never be ashamed of their preoccupation with theory rather than practice. Every profession needs its theory. We develop theories to help us make some order, some meaning, out of the seeming chaos we see around us.

If you look at what I write about the economy, I think you’ll find I write about economic theory a lot more than other economic commentators do. Why? Because I think theory is important. Academic economists will complain that I’m often very critical of economic theory. Why? Because I think theory is important. The great sin in academic economics is to stop seeking the truth because you think you’ve already found it.

I want to talk now about the little-discussed paradox of the commercial news media. On one hand, most news outlets are in the business of selling their news to make a profit, just like all businesses. On the other, the commercial news media play a vital role in our democracy, informing citizens about the actions of governments and holding governments to account. We rarely think about this paradox, but the truth is, it was ever thus. We had newspapers before we had democracy.

Today we talk about public-interest journalism, but I like to think of it as the commercial media’s “higher purpose”. Making enough profit to keep our shareholders happy is the obvious part, but we must keep our eyes focused on the more important part, our self-appointed duty to ensure our readers are kept fully informed about all the things our governments are doing – and not doing

There’s an old saying in journalism: news is anything somebody somewhere doesn’t want you to know about. Governments have a lot of things they do want the public to know about what they’re doing. And their spin doctors are always trying to induce the news media to help them get the good news out to the voters

Governments have a near monopoly on news about their own doings. When they want something known, they can just put out a press release. Or, maybe a better idea would be for me to leak it to you exclusively – provided you give it a lot of prominence, and provided you run it uncritically. Why would the media agree to such a restriction on their freedom to fully inform their readers? Because if I play along today, you might give me another leak tomorrow. And that will make me look a lot more successful than my competitors.

Small problem: what about the reader? Is this the way to keep them fully informed and ensure they’re never misinformed? What if I’m so busy trying to be the best at extracting from the government news the government wants our readers to know about that I neglect my duty to dig out all the news the government doesn’t want our readers to know about?

Now, let me be clear. In saying this critical stuff, I don’t want you thinking I’m having a go at my own masthead. I’m giving my free advice to all mastheads. The mastheads formerly known as Fairfax aren’t perfect. No one knows that better than I do. But there are other outlets that have strayed a lot further from perfection than we have. Naturally, I won’t name those other Australian news outlets.

The digital revolution has hugely changed the news media. Once I’m retired, I’ll give in to the thought that it was all much better in my day. But while my day is still the present day, I can see the things that are better than they were.

These days, the mastheads our envious competitors like to dismiss as “the Nine newspapers” devote more resources to investigative journalism than we ever have. Maybe because of the digital world we now inhabit, generating your own news makes more commercial sense. What I’d add is that we need to make all our ordinary news more investigative. More questioning of all the messages some interest group or another wants us to pass on to our readers.

Another thing that’s better than it used to be – one close to my heart – is much greater emphasis on explanatory journalism. The internet has hugely increased the blizzard of news that we must fight our way through each day. Our readers don’t need more news, they need more help figuring out what on earth it all means. This, of course, is a big part of what I’ve always seen as my role.

With the advent of the internet, social media, the greater scope for the spread of misinformation and disinformation, and now AI, it’s easy see all this as a huge threat to what’s now called the MSM – the mainstream media, of which the SMH is a prime example. We live in a media world where people are finding it harder and harder to know whose news to believe. Who to trust.

What I want to say is that, for the mainstream media, and the quality end of the MSM, all the extra doubt and uncertainty about who to believe is playing into our hands. We are still more trusted by our customers than other, less reputable sources of information. Provided we retain our readers’ trust, work to regain what trust we have lost, and make retaining the trust of our readers our highest priority, I think we’ll survive – maybe even prosper – in a world teeming with misinformation.

We must never knowingly mislead our readers. We must see quickly correcting any errors we’ve made inadvertently, not as an admission of failure, but as badge of honour. Proof that we can be trusted. It means no more “I’m not sure it’s true, but it’s a great yarn and the punters will love it” stories. No more dubious stories published to oblige a powerful source – usually a government – and keep it supplying us with exclusives. It means telling our readers what they need to know, not what they want to hear. It means being a good read the hard way, not the easy way.

I’m confident that, if we get the trust right, enough money will follow. I’m hoping to stay around doing my bit for a few years yet.

This is an edited version of the speech Ross Gittins gave at the event honouring his 50th anniversary at The Sydney Morning Herald. Held in the Great Hall of the University of Sydney and staged in partnership with the Faculty of Arts and Social Sciences.

Read more >>

If you’re virtuous, don’t be afraid to signal it to the world

I’m troubled by the fashion of accusing others of “virtue signalling”. This world could use more virtue and less vice. And if people want others to see their virtue, well, there are worse sins.

Usually, it’s an accusation hurled at those on the other side of the political fence as a way of impugning their motives. They’re not genuinely virtuous, they just want people to think they are when they’re not.

They want to be seen as better than we are. They want me to feel guilty for not being as good as them, but I’m not buying that. I may be motivated by self-interest in the government policies I advocate, but so are they – they’re just pretending otherwise.

You can rationalise such a response by using the assumption of the neo-classical economic model that economic agents (you and me) are always and only motivated by self-interest. Altruism doesn’t exist. When I help someone, I’m doing so only because it makes me feel good.

In truth, social psychology has found plenty of evidence for the existence of altruism. It’s associated with another truth: homo sapiens’ success as a species is owed as much to co-operation as to competition.

I remember how shocked I was years ago to hear a top Treasury official refer with contempt to the Australian Council of Social Service – the peak body representing welfare organisations, including the Salvos – as “the compassion industry”.

First time I’d heard that word used as a term of derision. It reminded me of a song we sang when I was a Salvo: “Except I am moved with compassion, how dwellest Thy Spirit in me?”.

The Treasury man’s claim was that the ACOSS people didn’t really care about the poor and needy, they’d just found a way to make their living by representing the interests of poor. They were no more than another lobby group with their hand out.

As social animals, humans form themselves into tribes – groups. We have a compulsion to divide the world into good guys and bad guys. Naturally, my group are the goodies but, unfortunately, your group are the baddies.

Each of us sees ourselves as good, but some others as bad. I’m genuinely virtuous, whereas you’re just pretending to be.

In truth, none of us is all good or all bad. All of us are good in some respects and bad in others. And psychologists tell us we’re all often guilty of hypocrisy – applying high standards in judging others’ behaviour while making excuses for our own.

Equally, much of what we do we do for mixed motives. Try this test (one I usually fail): when you’re giving money to charity, how do you answer when asked if you’d like your donation to remain anonymous?

It’s possible some of us do virtuous acts – or make statements in support of virtuous policies – without any genuine interest in the wellbeing of others. It’s possible, but I doubt it’s very common.

What’s much more likely is mixed motives: we’re genuine in our professed concern about others, but equally genuine in our desire to be seen by others as having such a concern. That’s not really hypocritical, just being human.

Because we’ve evolved as group animals, all of us care deeply about what others think of us. We want to be accepted by the other members of the group. And we fear being excluded from the group.

Like teenagers, we’re desperate to fit in. The more we look and act like the others, the more comfortable we feel.

(This points to a further weakness in the neo-classical model: its assumption that each of us is a rugged individualist who makes decisions – about what movie to see or what clothes to buy – totally without reference to what those around us are doing.)

Turns out humans are signalling animals. We’re always using what we do, what we say, the way we dress, to signal our virtues to others – including our conformity to the group’s norms of acceptable behaviour.

The economy abounds with people and businesses sending signals. The first three economists to realise this won the Nobel prize for their genius.

We resort to sending signals because neither we nor others have enough hard information about the people we deal with and who deal with us. The main message we send is: you can trust me to deal with you honestly.

In today’s economy we’re suffering from a loss of trust, caused by a lack of virtuous behaviour, which has damaged reputations. We need economic behaviour to be a lot more virtuous. As that virtue is signalled, others will join in and the group norm of acceptable behaviour will be restored.

Read more >>

We're on the way to peak everything

Some economists worry the world economy isn't growing fast enough. It's slowing down and reaching the point of "secular stagnation".

On a very different wavelength, however, environmentalists worry that if the world economy keeps growing the way it is, it won't be long before we run out of the natural resources on which that growth depends. Whoops.

But if all that's a bit heavy for the holiday season, here's something lighter. Remember all that crazy talk a few years back about the paperless office? What a joke.

Then there was peak oil. Whatever happened to that looming disaster?

If any of those possibilities piques your interest, I have news - courtesy of an essay by Professor John Quiggin, of the University of Queensland.

Quiggin thinks the paperless office is on the way, especially because the world has already reached "peak paper".

Despite continuing economic growth, peak paper was reached in 2013. "Global paper production and consumption reached its maximum, flattened out, and is now falling," he says.

Until relatively recently, the growth and spread of information was directly linked to the growth in paper, books and newspapers.

The closely related information revolution and digital revolution have broken that link. Businesses and governments don't print reports, they just put them on their website. We read e-books and online newspapers.

Banks and businesses want to stop sending us statements and bills through the post. If we hold out too long, they impose a fee for continued paper statements.

As for peak oil, Quiggin says that, in terms of oil consumption per person, the world reached it in 1979.

"In the developed countries, the decline in oil consumption per person has outpaced population growth, with the result that total consumption is declining. The average person in a developed country now uses less oil than her parents did 40 years ago," he says.

Why has this remarkable change attracted so little notice? Partly because much of the reduction in energy use has taken the virtually invisible form of improvements in energy efficiency. Both industrial processes and household appliances use far less energy than they used to.

But also because, until fairly recently, the main substitutes for oil have been other fossil fuels, such as coal and gas. Only in the past 10 years have renewable energy sources, especially wind and solar, begun to play a significant role, he says.

Peak coal has already arrived in the developed world. Coal consumption has fallen substantially in the US and Europe, and is set to fall further.

Until recently, the decline in fossil-fuel use in the developed world has been more than offset by rapid growth in the developing countries.

But even China - the planet's largest coal consumer by far - has changed course. Beginning with Beijing, it has begun closing down all the coal-fired power stations near major cities.

"In fact, China reached peak coal in 2013, at the same time as it reached peak paper," Quiggin says.

As for peak steel, it's different. Steel lasts a long time and can be recycled almost endlessly, but demand for it is finite.

In developed countries, the stock of steel reached about eight tonnes a person decades ago, he says, and has remained stable or slowly declining since then.

"With the stock of steel on a gently sloping plateau, the need for more can be met almost entirely by recycling scrap, rather than by burning coal to smelt iron ore in blast furnaces.

"The result has been described as a 'circular economy'. When this arrives, peak steel will have been reached."

All this has happened while economic growth has continued and living standards have risen.

Economists have been saying for years, particularly in the developed world, that growth is becoming "weightless". The part of the economy that's growing isn't goods - things you can drop on your toe - but services: people doing things for people, whether fixing their health, teaching them nuclear physics or waiting on their table.

With an ever greater proportion of gross domestic product - the quantity of goods and services produced in a period - accounted for by services, economic growth becomes ever less dependent on the increased use of natural resources.

Over the long term, growth in real GDP comes less from the use of more raw materials, human labour and man-made machines and structures and more from improved "productivity" - greater efficiency with which those inputs are transformed into outputs of goods and services.

What drives productivity improvement? Advances in technology and accretion of human capital. That is, the growth and spread of knowledge and information.

But an information-driven economy is very different from the one we've become used to since the industrial revolution, one driven by the use of natural resources to produce goods plus a few conventional services.

Natural resources are finite. If you want to use my coal or paper you must pay me (they're "excludable"). Any coal or paper you use can't be used by someone else (they're "rivalrous").

This makes economic growth relatively easy to measure. But knowledge and information are opposite to natural resources: they're often freely available (non-excludable) and my knowing something doesn't stop you knowing it, too (non-rivalrous).

What's the difference between a taxi and Uber? Information. What's the difference between renting a hotel room or self-catering accommodation and Airbnb? Information.

A knowledge and information-driven economy is one whose continuing growth makes less demands on the natural environment than many scientists and environmentalists imagine. That's particularly true as we move to renewable energy.

But a knowledge and information-driven economy is harder to measure, especially using the metrics (GDP) we developed to measure a raw materials and goods-based economy.

We're now in a world where GDP is going one way and raw-materials use is starting to go the other way.

That's why Quiggin doubts that world economic growth is grinding to a halt.

Read more >>

How signalling helps make the economy work

Why do so many people go on to university after finishing school? Why do some uni graduates get a job, but then go back to uni for further qualifications?

Why do sensible people dress up for a job interview – or wear a suit and tie if they're in court charged with an offence?

For that matter, why do people engage in conspicuous consumption – buy flash clothes or cars or houses, or send their kids to flash private schools?

Why do so many businesses put so much money and effort into protecting and projecting their brands?

Short answer to all those questions: because they're trying to signal something. What? Usually, their superior quality – although in the case of conspicuous consumption they're signalling their superior social status.

Signalling is something you don't read about in economics 101 textbooks, even though it occurs in all real-world markets.

That's because the simple neo-classical model makes the unrealistic assumption of "perfect knowledge" – buyers and sellers know all they need to know about all goods and services – not just the range of prices on offer but also the characteristics of the goods offered by various sellers, including their quality.

For many years, progress in economic theory has involved relaxing the various assumptions of "perfect competition" to see what we can learn from more realistic assumptions – which, by the very nature of theory and models, will still be a fairly simplified version of reality. (If a model was as complex as the real world, it would tell us nothing about what causes what in that world.)

Since the early 1970s, economic theorists have been studying "imperfect knowledge" (which in econospeak means "far from perfect", not "almost perfect"), recognising that there's much relevant information people don't know and that information is often costly to collect (in money or time).

As well, information is often "asymmetric", in that the people selling something, usually being professionals, know a lot more about it than buyers, usually amateurs, do.

In 2001 three American academic economists – Michael Spence, George Akerlof and Joseph Stiglitz – shared the Nobel prize in economics for their seminal contributions to the relatively new field of "information economics".

Akerlof (who's married to a certain central bank chairwoman) got his gong for a paper he wrote in 1970 called The Market for Lemons, aka used cars. Spence got the gong for a paper he wrote in 1973 about signalling in the job market.

So let's start again: why do people delay their income earning to get educational qualifications?

If you say it's because they want to gain knowledge and expertise in some field to make their labour more valuable – to increase their "human capital" – and help them get a better-paid job, you're not wrong and Spence wouldn't disagree with you.

But he focuses on a different, less obvious motivation. Employers are looking for intelligent workers and are willing to pay more for their services. But when you're hiring workers, it's hard to know how smart they really are. As economists say, it's an "unobservable characteristic".

So how do workers who know they're smart demonstrate that to potential employers? By using their educational qualifications to signal the fact. Employers are impressed by qualifications because they know they're not easy to obtain – they're costly, in a sense.

Of course, people who aren't so smart can gain qualifications if they try hard enough. But genuinely smart people don't have to try as hard, so they can gain higher, better qualifications than the less-smart can, and employers know this.

You're in line for a Nobel prize when you open up a new field and then other, more junior academics come along behind you to elaborate and expand on your discovery, eventually making it look pretty primitive.

By now thousands of academic papers have been written about signalling in various markets. It's become part of the study of "industrial organisation" (industry economics, as we used to say) but is also a branch of game theory.

Theorists have looked at cases of people sending signals implying they possess qualities that they don't and cases where signals are distorted by "noise" (say, you struck it lucky in the exams). And whereas in simple theory markets only ever have one equilibrium point – where everything is in balance – with signalling there are multiple equilibria.

One signalling theorist is Dr Sander Heinsalu, a bright young Estonian now in the Research School of Economics at the Australian National University.

In a recent paper he develops a "repeated noisy signalling model", quoting examples such as a politician giving speeches intended to make him appear competent, a firm buying positive product reviews, and a male deer growing antlers every mating season.

He finds that, if the cost and the benefit of signalling are constant across periods, the degree of signalling effort falls over time. This fits with the way conspicuous consumption falls with age.

In another paper Heinsalu says the conclusion of most signalling papers is that people for whom gaining more of the valued characteristic would be costly don't exert as much signalling effort as those for whom it is less costly.

But in his own paper he demonstrates that in some circumstances it can be the other way round.

With corruption, politicians face minor temptations and big ones. A pollie who is "too clean" may be avoiding minor misdeeds so he can survive long enough to engage in major graft when the opportunity arises, whereas another planning to avoid graft may not worry about small misdemeanours.

The guilty may deny accusations more strenuously than the innocent do because the innocent know they'll have less trouble proving it later.

As Shakespeare said, "the lady doth protest too much, methinks".

But if you want more proof than a quote from the bard, read the paper on his website. Hope your maths is up to it.

Read more >>