The Main Bottleneck For AI Is A Shortage Of The Powerful Chips

The overwhelming demand for AI has also revealed the global supply chain’s limits for powerful chips necessary to construct and deploy AI models.

According to industry analysts, the ongoing chip crisis has impacted firms large and small, including some of the AI industry’s major platforms, and may not ease much for at least a year or more.

The most recent indication of a potentially prolonged AI chip shortage came in Microsoft‘s annual report. For the first time, the research cites the availability of graphics processing units (GPUs) as a potential risk factor for investors.

GPUs are a vital form of hardware that aids in the execution of the numerous calculations required for training and deploying artificial intelligence systems.

“We continue to identify and evaluate opportunities to expand our data center locations and increase our server capacity to meet the evolving needs of our customers, particularly given the growing demand for AI services,” Microsoft said in a statement. “Our data centers are dependent on the availability of permitted and buildable land, predictable energy, networking supplies, and servers,  including graphics processing units (‘GPUs’) and other components.”

Microsoft’s reference to GPUs emphasizes how computing capacity is a significant limitation for AI. The problem immediately affects enterprises that develop AI tools and products, as well as corporations and end users that expect to employ technology for their own goals.

In May, OpenAI CEO Sam Altman testified before the US Senate, claiming that the company’s chatbot tool was struggling to keep up with the volume of inquiries.

“Because we’re running low on GPUs, the fewer people who use the tool, the better,” Altman explained. Later, an OpenAI spokeswoman informed CNN that the business is committed to providing enough capacity for users.

The issue may seem similar to pandemic-era shortages of popular consumer equipment, which resulted in gaming fanatics paying significantly inflated rates for game consoles and PC graphics cards. Manufacturing delays, a labor shortage, difficulties in worldwide shipping, and constant competitive demand from cryptocurrency miners all contributed to the scarcity of GPUs at the time, spawning a cottage industry of deal-tracking technology to help ordinary consumers find what they needed.

Increased demand for advanced work

According to industry analysts, the current shortfall is very different in sort. Instead of disrupting supplies of consumer-focused GPUs, the continuous shortfall reflects the rapid, booming demand for extremely high-end GPUs intended for sophisticated activities such as AI model training and use.

The production of those GPUs has reached capacity, but the surge in demand has exceeded the few sources of supply.

According to Raj Joshi, a senior vice president at Moody’s Investors Service who studies the chip sector, there is a “huge sucking sound” emanating from enterprises symbolizing the unrivaled need for AI.

Bottom Line – From The Bottleneck To The Bottleneck

Complicating matters, GPU manufacturers cannot gain enough of critical input from their own suppliers, according to Sid Sheth, founder, and CEO of AI company d-Matrix. The silicon interposer technology, which is required for the completion of GPUs, operates by coupling isolated computational units with high-bandwidth memory chips.

The Biden administration has prioritized boosting US chip manufacturing capacity; the CHIPS Act, passed last year, is expected to provide billions of dollars in support for the domestic chip industry and chip research and development. However, those efforts are aimed at a broad range of chip technologies rather than primarily increasing GPU production.

As more production comes online, the chip shortfall is likely to lessen.