Senior civil servants told a committee of MPs that an AI being developed by the Department for Work and Pensions (DWP) to detect fraud and error is still in the learning stages of its development and is exhibiting bias against some claimants. Part of a project to target more cases of benefit over and underpayments, the systems won’t be ready for deployment for a number of years, MPs have been told.
DWP officials were quizzed by parliament’s Public Accounts Committee (PAC) following the publication earlier this year of a National Audit Office (NAO) report looking at the system’s development. As reported by Tech Monitor, the NAO revealed that the DWP was investing £70m over the next two years in advanced analytics to deal with rising levels of fraud and error. The department believes the investment will generate savings to the taxpayer of around £1.6bn by 2031.
DWP was represented by Bozena Hillyer, director for counter fraud compliance and debt; Peter Schofield CB, permanent secretary; and Catherine Vaughan, finance director general. Richard Hawthorn, director, of operational excellence, customer service group represented HMRC.
The session delved into why fraud and error – which covers over- and underpayments, deliberate or otherwise, of welfare – had increased within the benefits system. MPs raised questions and concerns over underpayments of pension credit, a rise in Universal Credit ‘fraud’ and underpayments of personal independence payments (PIP) due to changes in circumstances and paused automated health assessments.
As part of the session, the DWP explained the progress it had made with its targeted case review project and its AI system, which will be used to identify cases of fraud using details from historical cases. However, concerns of AI bias were raised by the committee on whether the AI would unconsciously choose cases that affected protected characteristics. DWP admitted that bias existed within the AI, but that it was still in the early stages of learning.
MPs question whether DWP requires AI for fraud detection
In the early part of the session, MPs asked Neil Couling, director general of change and resilience at the DWP, about the effectiveness of the AI project for fraud detection. He is the senior officer responsible for the project
“The targeted case review is having an effect today [but] it won’t be at a volume big enough to be really picked up in the sampling, unless you were really lucky in the cases you sampled,” Couling said. “We take 3,000-odd universal credit cases through the sampling. That will take two years before it starts to have a measurable effect.”
MP Jonathan Djanogly asked whether the investment in the project was value for money given it was pulling up the same percentages of incorrect claims (30%) as random sampling.
“Obviously my question is: why the project? Why do you need to spend all this money if the result is the same as if you kept it random?” he asked Couling.
The DWP official said that AI could get a “higher hit rate”. Couling also pointed to pressure from the government for his team to speed up the development of the AI: “I am resisting the urges of lots of people around me to go faster and snatch at success here, because what I do not want it to do is to turn into a process,” he said.
Couling said it was important the department was given time and resources to ensure the system worked on launch. “It depends on me keeping the quality there, and we are still learning,” he explained. “We are still developing the service, and I will not have finished that until we are ready to go to market.”
AI bias exists within DWP project
The NAO noted in its report that the DWP has been using machine learning to flag potentially fraudulent claims for Universal Credit advance payments since 2021. It has since been piloted in other parts of the welfare system including for people living together, the self-employed, and for housing claims.
But MPs raised fears that the use of historical claimant data and fraud referrals by the AI could lead to biased results. Couling said he shared those concerns: “I do not want to preside over a system that does that,” he said, going on to explain that DWP was doing two things to mitigate the risk.
“The first is that we always put a human being at the point of decision-making about a claim, and if we are using any of this information, we put to the decision makers false positives, as well as cases that we think are flagged for attention,” he said. “The second thing we are doing is that we are taking this carefully… with small numbers at the start so that you can observe what is going on. You can then run analyses against this to check for bias.”
When asked how the DWP was teaching the system the dangers of discrimination and misinterpreting information, Couling said that he couldn’t disclose the models being used, but that the department was taking steps to rectify the issues with the AI.
Data watchdog the Information Commissioner’s Office has been liaising with the DWP on the development of the AI system, and the department plans to give MPs an annual report on the progress of the project.
Read more: DSIT aims to boost digital skills by joining STEM futures project
