New Zealand’s Ministry of Social Development (MSD) is currently facing a firestorm of criticism. Critics have begun to criticize its increasing use of Automated Decision-Making (ADM) systems in the welfare industry. While the Ministry aims to simplify welfare processes and reduce public spending, advocates worry that insufficient safeguards may replicate the issues seen in Australia’s infamous Robodebt scandal. Depending on how ADM is applied, it may affect thousands of beneficiaries at a time. This begs important questions related to transparency, accountability and the safety of our most vulnerable citizens.
First, the MSD argues that ADM will improve the welfare delivery system by making it more efficient and speeding up the delivery of assistance. The federal government’s own commitments would see 50,000 people moved off Jobseeker payments. They plan to do this by getting them into jobs by 2030. This past spring, the Ministry adopted Ontario’s first new ADM standards. They focused particularly on notice and transparency and provided people the opportunity to appeal decisions made by an algorithm. These standards are now out for review in consultation with New Zealand’s privacy commissioner.
The Dark Shadow of Robodebt
Australia’s Robodebt scheme should be a warning to the architects of New Zealand’s ongoing welfare reforms. The former conservative federal government introduced Robodebt in 2016. This program was meant to automate more debt recovery processes and saved taxpayers thousands of dollars each year by the billions. It became the most expensive public policy scandal in Australian history. This manmade crisis hurt a diverse array of everyday people.
Advocates in New Zealand are understandably concerned about not wanting to go down the same path Australia has just taken. They raise alarms that the very same automated systems would make such miscalculations, putting vulnerable populations at risk. Professor Nicholas Agar highlights the unpredictable nature of AI technologies:
“Often, when you’re using a technology that you don’t fully understand, and even those who make it [AI] don’t fully understand it, stuff can go wrong and that hurts people.”
These sentiments are quite representative of the broader conversation regarding the efficiency and morality of using ADM in welfare.
Safeguards and Standards Under Review
In answer to that increasing public outcry, the MSD has introduced a new series of standards to regulate the use of ADM. These are not insurmountable provisions. There should be provisions for transparency and for beneficiaries to appeal automated decisions. Melissa Gill from MSD on behalf of MSD, the decision to deny any benefit must be made by a member of the appropriate staff. Automation won’t be the driving force behind this process.
“Any decision to decline a benefit will always be made by a staff member,” – Melissa Gill.
She goes on to explain, “And there are no plans to amend the situation.” This statement aims to reassure beneficiaries that human oversight remains a central component of decision-making processes within the welfare system.
The standards require robust transparency and validation requirements for creating and deploying algorithms, or complex formulas, that power ADM. Gill argues that these requirements improve accountability. They are the ones who make sure privacy, human rights, and ethical considerations take precedence before rolling out algorithms.
“MSD’s ADM Standard includes additional requirements for the use of complex algorithms,” – Melissa Gill.
Critics counter that these standards lead to impressive appearances without real substance. Here, though, they prove unable to prevent the kind of disastrous outcome experienced by so many dependent on the welfare state. Ana Ika from Maryland-National Capital Park & Planning Commission cautioned that increasing layers of complexity might add to the burdens already created.
“Introducing additional layers on top of the complexities that are already there in the welfare system will just push people more into hardship,” – Ana Ika.
The Path Forward
New Zealand’s welfare system is leaning into greater automation. This decision advances the government’s broader agenda of reducing public expenditure. The Ministry’s new strategy aims to reduce red tape for striking a balance, pushing recipients to take greater initiative to seek job opportunities. Social Development and Employment Minister Louise Upton stated:
“It’s important that beneficiaries who can work are taking reasonable steps to re-enter the workforce or remain work-ready, and that consequences exist for those who don’t.”
It would fulfill the government’s stated desire to radically reform welfare through ADM. We’ve learned through continued consultations with privacy advocates and other stakeholders that we must consider the effects on society’s most vulnerable people.
New Zealand is busily grappling with the thorny issues posed by automated decision-making. It needs to weigh these remarkable efficiency gains against ethical considerations and human rights. The re-evaluation of ADM standards presents stakeholders a valuable opportunity to raise their objections. Together, we can help build a welfare system that is fair, accountable and supportive of all beneficiaries.
Leave a Reply