The Guardian article Digital dystopia: how algorithms punish the poor addresses the impact of technology on government, and specifically on the apparatus of the welfare state. I’m going to comment on what I think is driving these changes, and on the article as a whole.
All around the world, from small-town Illinois in the US to Rochdale in England, from Perth, Australia, to Dumka in northern India, a revolution is under way in how governments treat the poor. Only mathematicians and computer scientists fully understand the sea change, powered as it is by artificial intelligence (AI), predictive algorithms, risk modeling and biometrics. But if you are one of the millions of vulnerable people at the receiving end of the radical reshaping of welfare benefits, you know it is real and that its consequences can be serious – even deadly.
It is certainly true that AI and cloud computing are driving changes in many industries as well as in the government. In industry this change is disruptive. These changes, like nearly all changes, are being driven by cost reduction. If you want to compete with Amazon in retail, you will be competing against a technology powerhouse that uses state of the art AI and hosts the worlds largest computing platform, Amazon Web Services (AWS). Companies that don’t adopt new technologies can’t compete head to head.
The Guardian has spent the past three months investigating how billions are being poured into AI innovations that are explosively recasting how low-income people interact with the state. Together, our reporters in the US, Britain, India and Australia have explored what amounts to the birth of the digital welfare state.
What is the digital welfare state? More importantly, what is bringing it about?
Their dispatches reveal how unemployment benefits, child support, housing and food subsidies and much more are being scrambled online. Vast sums are being spent by governments across the industrialized and developing worlds on automating poverty and in the process, turning the needs of vulnerable citizens into numbers, replacing the judgment of human caseworkers with the cold, bloodless decision-making of machines.
Here then is the definition: “vast sums are being spent on automating” the welfare apparatus to replace “human case workers” with machines. Why are governments doing this, given that, at least in the short term, benefits are being “scrambled” by the new systems?
Listen to governments, and you will hear big promises about how new technologies will transform poverty as a noble and benign enterprise. They will speed up benefits payments, increase efficiency and transparency, reduce waste, save money for taxpayers, eradicate human fallibility and prejudice, and ensure that limited resources reach those most in need. But so often, those pledges have fallen flat.
Do you ever believe the government when they say they will transform anything for the better? I don’t. I believe the truth is that governments are going digital to “increase efficiency…, reduce waste, and save money”. I don’t believe that governments give a crap about transparency. Are you surprised when government pledges fall flat? I’m not.
At a time when austerity dominates the political landscape, millions have had their benefits slashed or stopped by computer programs that operate in ways that few seem able to control or even comprehend. Mistakes have become endemic, with no obvious route for the victims of the errors to seek redress.
Austerity does not dominate politics. The US is running record deficits, and Canada’s Liberal’s have been on a four year spending spree and are promising even more spending if we reelect them. The fact is, you can’t continue to spend beyond your means indefinitely. Saving costs by eliminating waste and replacing expensive government employees with machines has become a necessity as debts have grown to the point where governments are no longer able to borrow money at favourable rates.
This week, the automation of poverty will be brought on to the world stage. Philip Alston, a human rights lawyer who acts as the UN’s watchdog on extreme poverty, will present to the UN general assembly in New York a groundbreaking report that sounds the alarm about the human rights implications of the rush to digitalize social protection.
The benefits of a society are not a human right. If the society lacks resources, it cannot simply print money. Even the US, which for decades has controlled the de facto world currency, can’t keep up quantitative easing (AKA printing money) forever. If the value of the US dollar reduced by the US government, other countries won’t want to hold it and, eventually, they will stop using it as the common currency of trade.
In Illinois, the Guardian has found that state and federal governments have joined forces to demand that welfare recipients repay “overpayments” stretching back in some cases 30 years. This system of “zombie debt”, weaponized through technology, is invoking fear and hardship among society’s most vulnerable. As one recipient described it: “You owe what you have eaten.”
Governments are incompetent. In my opinion, they should focus on fixing their systems to prevent overpaying in future. Recouping money from welfare recipients is unlikely to be successful, and will probably waste even more money.
The [UK] government is rushing forward with its digital mission despite the pain already being inflicted on millions of low-income Britons by the country’s “digital by default” agenda. Claimants spoke of the hunger, filth, fear and panic that they are enduring.
More government incompetence.
In Australia, where the Guardian has reported extensively on robodebt, the scheme that has been accused of wrongly clawing back historic debts through a flawed algorithm, we now disclose that the government has opened a new digital front: using automation to suspend millions of welfare payments. Recipients are finding their money cut off without notice.
More government incompetence.
The most disturbing story comes from Dumka in India. Here, we learn of the horrifying human impact that has befallen families as a result of Aadhaar, a 12-digit unique identification number that the Indian government has issued to all residents in the world’s largest biometric experiment. Motka Manjhi paid the ultimate price when the computer glitched and his thumbprint – his key into Aadhaar – went unrecognised. His subsistence rations were stopped, he was forced to skip meals and he grew thin. On 22 May, he collapsed outside his home and died. His family is convinced it was starvation.
More government incompetence.
The Guardian investigations illuminate the shared features of these new systems, whether in developing or developed countries, east or west. The most glaring similarity is that all this is happening at lightning speed, with hi-tech approaches sweeping through social services, work and pensions, disability and health, often with minimal public debate or accountability.
I think this speaks to the desperate straights governments are in with years of overspending on bloated, inefficient programs. They must come up with ways of saving money or be forced to initiate real austerity measures like the ones the EU forced upon Greece. Since doing so virtually insures being voted out, automation offers cost savings that will let the current system continue to operate, at least for a while.
Within that revolution, the human element of the welfare state is being diluted. Instead of talking to a caseworker who personally assesses your needs, you now are channeled online where predictive analytics will assign you a future risk score and an algorithm decide your fate.
Since governments are not motivated by profit, they are likely to try to make their algorithms as fare as possible. On the flip side, governments are less competent, so the systems are likely to have more flaws and be far more difficult to change. In the US, systems that can be sold to the US government have to be certified by the FedRAMP (Federal Risk and Authorization Management Program), a complex process that adds cost to government systems.
In the new world, inequality and discrimination can be entrenched. What happens if you are one of the five million adults in the UK without regular access to the internet and with little or no computer literacy? What if the algorithm merely bakes in existing distortions of race and class, making the gulf between rich and poor, white and black, college-educated and manual worker, even more pronounced?
Welfare already does this.
There is also a chilling Kafkaesque quality that spans the globe. As Manjhi so tragically discovered, mistakes are made. Machines glitch. If there is no one within reach who sees you as a person and not as a 12-digit number to be processed, the results can be fatal. The computer says “No payments”. Now what do you do?
There is a rule in software development: you can have any two of cheap, fast, and good, but not all three. Since the reason the government is implementing these systems is cost savings, they will want them cheap. As stated above, they are rushing them into production. Therefore, expect their quality to be poor.