WORLDTECHGoogle unveils TorchTPU to challenge Nvidia's grip on AI compute
Google has unveiled TorchTPU, a specialised build of the open-source PyTorch framework intended to erode the advantage Nvidia's graphics processing units hold in AI computing. According to SiliconANGLE, the release targets a key reason Nvidia has remained so dominant.
What TorchTPU gives developers
TorchTPU is an alternative take on PyTorch that lets developers more readily assemble models and applications able to run on Google's Tensor Processing Units, chips designed for AI workloads. Applications and models already built can be moved onto TPUs through a handful of configuration changes, leaving the underlying code untouched.
Why Nvidia's position is hard to shift
Nvidia's GPUs remain dominant largely because the bulk of AI applications have been written with them specifically in mind. PyTorch, a widely used open-source toolkit for building, training and running AI models and neural networks, sits at the centre of that software base. Some accounts cited by the report say more than 85% of AI applications are built with PyTorch, a framework Meta Platforms created in 2016 with Nvidia's chips in mind.
Nvidia's hold persisted after Meta open-sourced PyTorch in 2022 and handed its stewardship to the nonprofit Linux Foundation. TorchTPU is Google's attempt to loosen that reliance.
Amazon and Google pursue parallel routes
Google is not the only company moving against the arrangement. Earlier this week Amazon Web Services announced its own PyTorch variant, TorchNeuron, which similarly simplifies running AI applications on Amazon's Trainium chips. Amazon also said it has contributed code to the original PyTorch project that decouples several key features from Nvidia's GPUs.
Google did not follow that path, but the report suggests it may have gained ground in other ways. The company said it worked closely with Meta, Hugging Face, and the startups Inferact Inc. and RadixArk Inc., developers of vLLM and SGLang, to make TorchTPU features and code libraries compatible with their developer tools.
Open questions
- How much of the AI software base will actually shift from Nvidia GPUs to TPUs?
- How will TorchNeuron and TorchTPU compare for developers in performance terms?