Last July, the US National Eating Disorder Association (Neda) learned an unfortunate lesson about the consequences of jumping headfirst into the generative AI revolution. Its goals were lofty and ambitious. Seeing the potential of chatbots to dispense advice about eating disorders on a much wider scale than its small army of call centre workers, Neda launched ‘Tessa,’ an AI-powered assistant that it promised would help users ‘build resilience and self-awareness by introducing coping skills at your convenience.’
Instead, ‘Tessa’ was taken offline at the end of May after she started spouting unsafe weight loss advice — even reportedly recommending that one user pursue a calorie deficit of between 500 to 1,000 calories. By that point, Neda had also fired its entire team of call centre staff. ‘If I had accessed this chatbot when I was in the throes of my eating disorder, I would NOT have gotten help for my ED,’ wrote activist Sharon Maxwell on Instagram. ‘If I had not gotten help, I would not still be alive today.’
This debacle serves as a cautionary tale for the innumerable companies fighting to weave AI products and services into their corporate workflows. The advantages in doing so are seemingly clear: AI, after all, promises exciting opportunities to cut costs and boost efficiency. That, in turn, is increasing the pressure on chief technology officers (CTOs) to deliver the goods, and ensure that their firms are first in line to harness ChatGPT and its ilk. Some have — but many more are struggling to reconcile the inflated expectations surrounding generative AI with the reality of its implementation throughout traditional corporate hierarchies.
Dealing with AI Hype as a CTO
AI’s outsize role in popular discourse is putting a lot of pressure on CTOs’ judgments, explains Anjali Arora, who manages a team of CTOs at Perforce Software. “AI has kind of taken the world by storm, but we also know it is causing chaos,” she says, with a laugh. Staff, technical or otherwise, all want to use AI tools like ChatGPT. It’s a leader’s job to ensure that they use it thoughtfully and carefully, explains Arora – which means matching a specific problem to a solution and not simply implementing AI and LLMs for marketing purposes.
Hywel Carver agrees. This is a moment that calls for sober judgement from CTOs, says the founder of tech coaching startup Skiller Whale. “You’re starting to see people making very bad decisions based on hype,” he says, citing cases like Neda’s ill-fated launch of ‘Tessa.’
Above all, says Carver, CTOs need to be offering a clear-headed vision of the limitations of these new technologies. That’s always been a good rule of thumb for these guardians of corporate tech, he adds, recalling the days when CTOs were busy bringing board members back down to earth about ‘gamification.’ In situations like that, says Carver, it should fall to the CTO to step in and say, “Here’s what that actually means. Here are the limitations of that idea.”
But it’s not just about questioning boardroom pressures to implement new technologies. If thought leaders aren’t extolling the transformative potential of generative AI, they’re talking about existential, society-ending risks arising from ChatGPT and its ilk. Now, CTOs like Christian Floerkemeier of Scandit, a barcode scanning software company, are spending much more of their time talking about how the machine learning woven into their products isn’t going to upend their clients’ business model overnight. When Scandit was founded, AI “wasn’t part of our marketing,” says Floerkemeier. “We just tried to solve the problem and that was our way of solving this problem.” Now, however, it’s gone from “a super niche term” to a heated debate.
What, then, do CTOs need to do to keep up with all this upheaval? For one thing, says Arora, they need to expand their skillsets beyond just technical expertise. They’ll need to develop strong leadership abilities, robust people-management skills, and effective critical thinking — otherwise they might not be able to tell an overhyped fad from a groundbreaking business opportunity in the first place. Communication will also be key, explains Kunal Purohit, chief digital services officer at Indian IT services firm Tech Mahindra. CTOs need to be capable of outlining complex ideas to business leaders, says Purohit, as well as make a strong case for comprehensive planning before implementing any new technologies.
They’ll also need to “foster a culture of continuous learning and upskilling” if they have any hope of keeping up with the immense rate of change, adds Tech Mahindra’s CTO. Carver agrees. Generative AI is an “unusually fast-progressing” landscape, he says, and it can seem as though dozens of new tools and applications are being released each week. “I have no time to keep up with that,” says Carver, “and I suspect most people don’t.”
CTOs can’t be expected to know every fresh detail, he says, but it’s still crucial for them to get to grips with underlying principles of new technologies and the general landscape of the AI field. You need to know, for example, where these tools can excel and where they fall flat on their face. That’ll stand you in good stead when it comes to implementing them into your own product, explains Carver.
Cybersecurity on the brain
Cybersecurity is another ever-present concern for CTOs, says David Warshavski, vice-president at London-based cyber services firm Sygnia.“CTOs are facing increasing pressure to adopt AI technologies in the workplace and the demand from the board down to employees can often mean such AI services could be implemented before the CTO and IT team has had the chance to conduct a thorough risk assessment.”
“Companies are rushing to embed AI tools — and especially LLMs — within their products, creating a new attack surface that provides threat actors with new ways to breach the perimeter, leak customer data, and potentially launch destructive attacks,” says Warshavski. CTOs will need to educate their corporate leadership and set clear expectations for AI adoption — especially when it comes to sharing sensitive data. Without these clear guidelines, he says, AI could enhance and accelerate aggressive cyberattacks, thereby endangering even the most mature and resilient organisations.
AI might have added a heap of new security concerns but CTOs, at least, need not fear for their jobs in a post-AI world. “I don’t think we’re going to be replaced any time soon,” says Carver. “I think leadership roles are fundamentally about communication and understanding and empathy, and all of those are what I think of as essentially human skills.” Moreover, while AI tends to show a lot of promise when it comes to perception, contextual judgement still seems to elude its capabilities.
Arora is even more confident about the human future of her chosen career-path. Generative AI might soon become everyone’s favourite office assistant, she argues, but it’ll still need expert oversight and clear leadership. “The best people will be the ones who will know how to use AI to assist them,” says Arora. Given that, CTOs might become all the more important — opening up more, not fewer, jobs.
Read more: Your AI coding assistant is a hot mess
