//php echo do_shortcode(‘[responsivevoice_button voice=”US English Male” buttontext=”Listen to Post”]’) ?>
—SAN JOSE, CALIF. The lights dimmed and 17,562 folks drew breath. A sea of smartphones held above heads lit up throughout the house of San Jose’s ice hockey staff, the Sharks.
An actual-time, AI-generated digital sculpture started to fill the towering show behind the stage—a continuously swirling corpus of color-changing bubbles, suggesting barely perceptible landscapes, animals and pure varieties, like a Rorschach take a look at in vibrant technicolor.
Then the pre-game fanfare, which, as traditional, started with promotional footage of how Nvidia is altering the world. Mixed with the ambiance within the stadium, clips of cutting-edge healthcare and local weather change analysis on the large display screen grew to become surprisingly emotive.
Then he takes the stage—trademark alligator-embossed black leather-based motorbike jacket contrasting with white hair—to a barrage of smartphone flashes.
By Dylan Liu, Geehy Semiconductor 03.21.2024
By Lancelot Hu 03.18.2024
By EE Instances Taiwan 03.18.2024
“I hope you notice this isn’t a rock live performance—it is a developer convention,” he mentioned, to a roar of laughter and applause from the gang.
Welcome to Nvidia CEO Jensen Huang’s GTC Keynote, also referred to as one of many largest AI hype automobiles within the trade.
Over two hours, Huang laid out quite a few improvements and breakthroughs Nvidia made or enabled within the final yr. He talked about how generative AI has infiltrated virtually each trade—together with chip design—however emphasised the months required to coach the most important trillion-parameter fashions.
“What we want is greater GPUs,” he mentioned.
No kidding.
Then the second the stadium—and the world—had been ready for, as Huang unveiled Nvidia’s new GPU platform: Blackwell. The centerpiece, B200, is Nvidia’s first GPU superchip—two reticle-sized die on a brand new customized course of node that may supply 2.5× the FLOPS of the present state-of-the-art AI coaching chip, H100. The 2 die have a ten TB/s hyperlink and are absolutely cache-coherent, so the dies behave like one huge GPU. The result’s as shut as it’s doable to get to breaking the reticle restrict at present.
Whereas Blackwell is little doubt a triumph of engineering, much more vital features have been realized on the system stage. For generative AI, the place communication is the bottleneck, Nvidia’s change chips, NICs, and DPUs with 72 Blackwell GPUs and 36 Grace CPUs in a single rack can enhance efficiency by 30× and power-performance by 25× for generative AI, versus Hopper.
“This method is form of insane,” Huang mentioned.
Insane appears about proper—a single rack offers 1.4 exaFLOPS of AI compute (FP4/sparse), 13.5 TB of HBM and 30 TB of quick reminiscence. Liquid cooling ejects 2 liters per second of sizzling water, which Huang instructed may “energy a Jacuzzi.” In reality, this machine will doubtless energy nearly the entire world’s generative AI analysis, coaching, and deployments going ahead, and a very good portion of scientific computing, too.
Huang highlighted dozens of different Nvidia improvements over the 2 hours, together with AI software program, Omniverse and robotics, with no much less zeal, even saying that the “ChatGPT second for robotics is across the nook,” however the true focus, as at all times, was on generative AI.
Demand for coaching and inference of generative AI is exploding and Nvidia holds the lead virtually unchallenged by way of {hardware} deployments at present. With the launch of Blackwell, each competitor product that was claiming to maintain the tempo with Hopper has been blown out of the water. B200, as the brand new H100, has reset the usual by which all different AI {hardware} is measured, and that normal simply jumped 30× on the system stage.
In different phrases: the gloves are off.