OpenAI is reportedly looking to develop its own semiconductors to power its large language models and AI tools. Such a move would reduce the artificial intelligence lab’s reliance on costly GPUs from Nvidia, which are currently in high demand around the world as tech vendors and businesses rush to implement AI systems.
The company is currently exploring the possibility of developing chips in-house, according to a report from Reuters which cites sources familiar with the plans. Though no final decision has been made, it has apparently been evaluating a potential acquisition target to accelerate this process.
Since the launch of ChatGPT last year, OpenAI has become one of the most talked-about tech companies in the world. The success of the chatbot kicked off an AI revolution spanning every industry and helped OpenAI secure billions of dollars of investment from Microsoft. News that it is considering a move into the chip market comes a week after it was reported to be working with former Apple design chief Jony Ive to develop a device that could become the “iPhone of AI”, backed by a $1bn investment from SoftBank owner Masayoshi Son.
The Nvidia conundrum – why OpenAI needs its own chips
Like most AI companies, OpenAI uses hardware produced by Nvidia for training and maintaining its models. Nvidia’s flagship AI GPUs, the A100 and its successor, the recently released H100, account for 80% of the AI chip market, and this market share could get even higher according to some analysts.
AI companies need thousands of these chips to run their systems, with the AI supercomputer built by Microsoft for OpenAI powered by 10,000 A100s, which come at a cost of $10,000 a time. The H100 is even more expensive, at some $30,000 per chip, and with Nvidia only expecting to ship 550,000 this year, supply problems are likely to abound. Indeed, OpenAI CEO Sam Altman has openly complained that a lack of GPU availability is hindering his company’s progress.
With this in mind it is no surprise that OpenAI is looking to take production in-house to “escape its dependence on Nvidia struggling to meet demand”, says Mike Orme, who covers the semiconductor market for GlobalData. But, he says, the supply problem does not stem from Nvidia itself, but has occurred because “TSMC, which makes Nvidia’s chips, has run into a severe back-end chip packaging capacity problem, which is likely to take up to 18 months to solve.”
TSMC, the Taiwanese company that dominates chip manufacturing, is having issues with its chip-on-wafer-on-substrate packaging process (packaging, in its most simple terms, is the way components are arranged and connected in a chip), which is constraining supply. Speaking to Asia Nikkei last month, TSMC chairman Mark Liu said the company could only meet 80% of customer demand as a result of the issue, which is expected to persist into 2025.
Can OpenAI take its software success into hardware?
Designing a chip in-house could be a tall order for OpenAI which has so far been solely focused on software, rather than hardware.
One company that has been on this journey is Sambanova, which offers a full-stack AI solution and last month launched its new SN40L AI semiconductor. Alex White, general manager for the EMEA region at Sambanova, says the shortage of GPUs is “undoubtedly hurting OpenAI’s scaling ambitions”, so it would be no surprise to see the company turn to its own semiconductors.
But White says that Altman and his team – as well as OpenAI’s clients – must be prepared for the long haul to bring such a plan to fruition. “There’s a clear advantage to owning the whole stack from hardware to software – including the models that run on top,” he says. “But designing and manufacturing chips doesn’t happen overnight, it requires huge levels of expertise and resources that are in increasingly short supply.”
White adds: “It took OpenAI over five years to develop GPT-4, which may be too long to wait for customers, I wouldn’t be surprised if hardware took a similar amount of time.”
GlobalData’s Orme says OpenAI could build out its hardware team by poaching talent from companies like Sambanova – as well as engineers graduating from leading universities – and “in short, doing in speciality chips what it did in the AI software arena to create OpenAI in the first place.”
He believes that making an acquisition is unlikely to come cheap. “To acquire, directly, one of the leading start-ups as a shortcut would cost a pretty penny, as [AI chipmaker] Tenstorrent is valued at $4bn and Cerebras at $1bn,” Orme says.
Will OpenAI join the Big Tech custom silicon trend?
If OpenAI does make its own chips, it will become the latest major tech company to take processor production in-house. Apple has been using its own chips, the A1 and M1 lines, for recent iterations of the iPhone and its Macbook laptops, while Amazon has developed the Graviton range, which it deploys in the data centres of its cloud platform, AWS.
In the AI space, Google has managed to avoid being dependent on Nvidia with a dedicated range of chips, which it calls tensor processing units. It says it uses TPUs for 90% of its AI workloads, and earlier this year it claimed that an AI supercomputer running on a string of TPUs was able to outperform a similar machine powered by Nvidia A100s, though it is likely the H100’s performance would trump such a set-up.
Microsoft is said to be working on its own range of AI processors, codenamed Athena. The Information reported last week that the first chip in the range could debut at MSFT’s Ignite 2023 developer conference next month.
Read more: Have we reached peak generative AI?
