Nvidia CEO Huang: AI Inference 'Kicking Into Gear' As T4 Sales Soar

'This year we started to see the growth of inference to the point where we sold more T4 units for inference than we sold V100s, which we sell for training, and both were record highs,' Nvidia CEO Jensen Huang says of the chipmaker's sales for its data center GPUs.

Nvidia CEO Jensen Huang said deep learning inference "is really kicking into gear" as the chipmaker's T4 GPUs surpassed its Tesla V100 in sales for the first time, a year after the inference chip launched.

"This year we started to see the growth of inference to the point where we sold more T4 units for inference than we sold V100s, which we sell for training, and both were record highs," Huang said Thursday on Nvidia's third-quarter earnings call for its 2020 fiscal year.

[Related: Ian Buck On 5 Big Bets Nvidia Is Making In 2020]

The Santa Clara, Calif.-based chipmaker's third-quarter earnings showed a continued recovery from a decline in sales earlier this year. Revenue for the quarter was $3 billion, a five percent decrease from the same period last year but a 17 percent increase from the previous quarter. Meanwhile, the company said while it could close its $6.9 billion acquisition of high-speed interconnect vendor Mellanox by the end of the year, but it's more likely to happen in early 2020.

Nvidia's stock price was down more than 1.5 percent in after-hours trading as it beat Wall Street's expectations for revenue by $90 million and net earnings, which were $1.78 a share, by 20 cents. The company expected fourth-quarter revenue to reach around $2.95 billion.

Contributing to the sequential increase was higher demand from hyperscale cloud service providers in Nvidia's data center business, which grew 11 percent to $726 million from the previous quarter. However, lower DGX sales and a different mix of products led to an 8 percent year-over-year decline.

Huang said one of the major factors impacting the year-over-year decline was a major customer deal for its DGX deep learning system, which is built for training AI models. Excluding that deal, he said, sales for the V100 and T4 GPUs "are doing great," especially since T4 launched in September of last year.

"That gives you a feeling for it […] Inference is really kicking into gear," Huang said.

Driving the need for inference workloads are conversational artificial intelligence, recommendation engines and edge computing, according to the CEO.

"Deep learning inference is understandably going to be one of the largest computer industry opportunities," Huang said. "The computational intensity is so high, and for the very first time, aside from computer graphics, this mode of software is not really practical without accelerators."

Meanwhile, the chipmaker's gaming GPU revenue was $1.66 billion. That marks a 26 percent increase from the previous quarter, thanks to growth from the company's GeForce desktop and notebook GPUs. However, revenue was down 6 percent year-over-year, representing a drop in sales for GeForce desktop GPUs after the chipmaker's new Turing-based RTX ray tracing graphics cards launched in fall 2018.

According to Nvidia CFO Collete Kress, RTX cards now represent two-thirds of Nvidia's desktop GPU sales as nearly every major game developer has pledged to incorporate ray tracing capabilities in their games.

"I would say that at this point it's fairly clear that ray tracing is the future, and RTX is a home run," Huang said.