After executive roundtable events in Frankfurt, Copenhagen and Stockholm, Tech Monitor and AMD returned to London in mid-September. The topic remained the same – the power and potential of artificial intelligence (AI) – but the make-up of the group was slightly different. Where the previous discussions featured predominantly IT and business leaders working in the private sector, this session was exclusively public sector – senior representatives from some of Whitehall’s largest departments alongside their peers in local government and across agencies.
As the discussion progressed it became clear that the challenges and opportunities of AI, and generative AI (GenAI) in particular, often cut across local and central government in a similar way, and mirror the agenda of the private sector, too. Sometimes important nuances emerged, however.
Outputs and outcomes: making the case for AI
One example of the latter emerged when the discussion turned to GenAI use cases. As at previous events across Europe, delegates proved keen to understand what others are doing – to borrow ideas and to validate their own.
Broadly, GenAI makes sense when it can “do it better” and/or “do it faster” than existing processes. In simple productivity terms, when organisations get more output for their input then that’s a straightforward win for technology deployment. There is an extra dimension when it comes to the public sector, however. That dimension is outcome. To quote one local government attendee: “How do we improve residents’ outcomes using AI (and ideally do it on the cheap)?”
That question illustrates this important point of differentiation. Namely, can you take the public with you? Projects aimed at cost-cutting alone are likely to lose public trust. The same applies to projects perceived to invade private spaces, regardless of how productive they may be. In other words, output matters but outcomes matter more. For an intriguing example consider the use of IoT sensors to detect mould and damp within homes. The public good may be plain to see but if residents are concerned for their privacy, then the outcome is compromised. More than one attendee pointed out that this doesn’t mean abandoning the use of technology but it does mean better explaining the benefits.
GenAI: to build or to buy?
One attendee identified “two species” of GenAI protagonists inside government. One is looking to build bespoke, open-source tools. The other is seeking to leverage GenAI add-ons to existing off-the-shelf software. On the latter, there is a project underway to deliver SaaS-based AI functionality for backend HR and finance processes. The aspiration is for the two applications to be widely adopted across Whitehall. The only challenge is ensuring departments regularly update their software to take advantage of the latest tools. This may sound like a relatively simple task but – as one attendee acknowledged – out-of-date software proliferates large parts of the government machine. The missed update runs counter to the spirit of continuous improvement.
Meanwhile, one example of bespoke development is Redbox, a purpose-built co-pilot designed to summarise the mountains of paper ministers typically find in their physical red boxes. The project is ongoing but signs are positive and it is now being rolled out for wider civil service use. One of the benefits, it appears, of using a bespoke dataset on which to train language models is that it tends to hallucinate far less than commercial GenAI chatbots.
Cloud versus on-premise
When the conversation turned to infrastructure plans to support the compute power required for AI projects, most attendees said they were leaning towards cloud. Why? For many, it came down to a simple cost equation. “We just can’t afford to build our own data centres, and then to manage and resource them,” said one attendee.
Cloud doesn’t suit every workload – latency, security, and sovereignty concerns may push some to seek a privately hosted solution, part of a hybrid cloud operating model. For others managing data-intensive applications, the cost of exiting data (egress) can prove prohibitive. But for cash-strapped local authorities especially, the capital expense of AI-ready on-premise infrastructure is equally, if not more, prohibitive. This is where a hybrid solution might prove persuasive.
AI meets digital modernisation
The economics of AI, the group was told, rests on two things: compute power and energy consumption. To manage both, public sector organisations first need to consider modernisation before embarking on fully-fledged AI projects. By modernising network, application, and data infrastructure – and by shrinking the real estate in the process – those that are delivering AI on-premise can reduce energy use, better manage compute power, lower cost of ownership, and make a more persuasive return on investment argument.
‘Powering AI’s potential ’ – a Tech Monitor roundtable discussion in association with AMD – took place at The Pem, London on 19 September 2024.
