Jim Keller, a legendary chip designer and presently CEO of Tenstorrent, an American AI chip design startup, is trying to design chips which can be extra environment friendly than Nvidia GPU, thereby decreasing the price of AI functions and aiming to seize a portion of Nvidia’s market share.
As AI expands into smartphones, electrical automobiles, and cloud providers, many firms are in search of cheaper options. Many small companies are unwilling to pay US$20,000 for Nvidia’s high-end GPUs. With the intention to goal these market segments that Nvidia hasn’t reached presently, Jim Keller is attempting to design chips which can be extra reasonably priced and environment friendly than Nvidia.
Tenstorrent is making ready to launch its second-generation multi-purpose AI chip by the top of 2024. In keeping with the corporate, in some areas, this AI chip surpasses Nvidia’s AI GPUs by way of vitality and processing effectivity. The truth is, in comparison with Nvidia’s DGX sequence of AI servers, Tenstorrent’s Galaxy system is thrice extra environment friendly and 33% cheaper.
Key to Decreasing Energy Consumption and Value: Abandoning HBM
Excessive Bandwidth Reminiscence (HBM) is a well-liked superior reminiscence chip able to transferring massive quantities of information shortly. It’s a essential element of generative AI chips, enjoying a big position in Nvidia’s product success. Nonetheless, HBM can also be one of many culprits for prime energy consumption and costly AI chips. Typically, for every process processed, the GPU has to ship information to reminiscence, requiring the high-speed information switch capabilities of HBM.
Tenstorrent, nonetheless, has specifically designed its chip to cut back the variety of information transfers drastically and doesn’t use HBM. Every Tenstorrent chip has lots of of cores, every with a small CPU, which might independently decide which information must be prioritized and which pointless duties may be deserted, thereby enhancing general effectivity. Keller believes this new strategy can permit Tenstorrent chips to exchange GPUs and HBM in some AI analysis areas. Not solely that, the corporate can even attempt to enhance the cost-effectiveness of its merchandise.
Since every core is comparatively impartial, the Tenstorrent chip may be tailored for a wider vary of functions by stacking extra or fewer collectively. For instance, a small quantity could be adequate for a smartphone or wearable gadget, whereas 100 might be mixed to be used in AI information facilities.
Keller admits that it’d take years to disrupt the present large-scale HBM business. He predicts that extra rising firms will enter the AI market that Nvidia presently can’t serve, moderately than a single firm changing Nvidia.
Associated article: AI Chip Scarcity Continues, However There’s a Glimmer of Hope
