Interview After some teething pains, TensorWave CEO Darrick Horton is confident that AMD’s Instinct accelerators are ready to take on large-scale AI training.
“There were definitely some bumps in the road with the first generation product,” Horton, whose rent-a-GPU outfit was among the first to throw its weight behind the Nvidia alternative, told El Reg. “It’s very publicly documented that AMD training performance was not ideal in 2024.”
Announced in late 2023, AMD’s MI300X promised 30 percent higher performance than Nvidia’s H100 while delivering more than twice the memory capacity and 60 percent higher bandwidth. In theory, the chips should have easily outpaced Nvidia’s Hopper GPUs.
The chip’s faster and more capacious (192GB) supply of HBM3 presented obvious advantages with regard to inference, enabling larger models to run on fewer GPUs than the equivalent H100 box. However, when it came to training, Horton tells us performance wasn’t quite there at the onset.
The issue, he says, wasn’t so much AMD’s hardware as it was the software.
“The difficulty in creating a training stack is orders of magnitude larger than creating an inference stack,” Horton said. “Their hardware is really good out of the box … but they were a little behind on software when they started.”
Because of this, TensorWave initially focused its attention on inference, with a bare-metal offering and later a managed offering. But over the past 12 to 16 months, Horton says, AMD’s software stack has improved to the point where the outfit is now looking to expand its offerings to large-scale AI training.
While much of the hype around GenAI has already shifted from training to inference over the past year, training remains relevant, he said. “There will always be training that’s happening, especially as we uncover new methods and foundational shifts in how we architect AI, it resets the clock.”
“We expect several more of those foundational shifts to happen in the next few years,’ he added.
This week, the GPU bit-barn operator snapped up a $100 million Series A funding round backed by Magnetar, AMD Ventures, Prosperity7, Maverick Silicon, and Nexus Venture Partners.
Some of that cash will go toward building out TensorWave’s team, but we’re told the bulk of it will be used to finance an AI training cluster containing 8,192 of AMD’s flagship MI325X GPUs, totaling roughly 42 exaFLOPS of sparse FP8 compute.
Unveiled last fall, AMD’s MI325X is essentially a bandwidth-boosted version of the original MI300X. The chip boasts the same floating point performance of its predecessor, but swaps the 192GB of HBM3 memory modules for 256GB of faster HBM3e memory, good for 6TB/s of bandwidth.
Staying cool
In order to extract the full performance of the 1,000-watt accelerators, TensorWave has opted for direct liquid cooling right out of the gate. If you’re not familiar, this technology replaces large, heavy heat sinks with small cold plates through which warm or chilled coolant is pumped.
“This generation, technically, you can do air cooling with. There are some other providers that have chosen to do air cooling, but it’s unwise and it’s short sighted,” Horton said. “Next-generation if you do air cooling you’re sacrificing performance. The generation after that, it probably won’t even be an option.”
This shift is already happening to some extent with Nvidia’s latest generation of accelerators. When the GPU giant still offers air-cooled designs, its most powerful NVL72 systems require liquid cooling to tame their 120-kilowatt power envelopes.
AMD doesn’t yet offer a comparable rack-scale system, though its Instinct MI400-series accelerators, expected to debut in 2026, are aimed at that space.
These rack-scale systems typically pack large numbers of GPUs stitched together with high-speed interconnects to act like a single accelerator—using Nvidia’s proprietary NVLink or, eventually, the open-standard UALink backed by AMD, Intel, and others. This presents a couple of advantages, particularly when it comes to running or training very large models, like Meta’s upcoming Llama 4 Behemoth, which reportedly features nearly 2 trillion parameters, including 288 billion active ones.
“There are some customers that really, really have to have NVL72 and it’s hard to win those customers right now,” he said. “But most customers don’t have a specific need for that. And when you consider the cost, AMD is still winning on TCO.”
Horton is looking forward to leveling the playing field when AMD’s rack-scale systems hit the market. TensorWave already has several more AI clusters for the not-too-distant future. Presumably, at least some of these will feature AMD’s next-gen MI355X accelerators, expected to make their debut next month.
TensorWave’s overarching strategy, meanwhile, remains unchanged from last year. Up to this point, much of its compute capacity has been amassed through VC funding rounds. Going forward, the biz aims to replicate the playbook used by CoreWeave and others, and use its GPU stockpiles as collateral for a large round of debt financing. ®
