Jensen Huang on NVIDIA's Extreme Co-Design, AI Scaling Laws, and Manifesting the Future of Computing

This podcast episode features Jensen Huang, CEO of NVIDIA, discussing the company's strategic evolution and its pivotal role in the AI revolution. Huang details NVIDIA's shift from a focus on individual chip design to 'extreme co-design' at the rack and data center scale. This comprehensive approach integrates GPUs, CPUs, memory, networking, storage, power, cooling, and software, driven by the necessity to overcome the limitations of distributed computing and Amdahl's Law. The core challenge is to achieve super-linear speedup in AI workloads that no longer fit within a single computer, requiring optimization across the entire technology stack.

Huang elaborates on NVIDIA's journey from an accelerator company to a full-fledged accelerated computing platform. A key strategic decision highlighted is the bold move to integrate CUDA, their computing architecture, onto GeForce consumer GPUs, despite the significant financial cost and risk to the company's gross profits and market capitalization. This decision was predicated on the belief that a large 'install base' is paramount for the success of any computing architecture, even over architectural elegance. By making CUDA accessible to millions of PC users, NVIDIA cultivated a developer ecosystem that eventually became the foundation for the deep learning revolution.

The conversation also delves into Jensen Huang's unique leadership philosophy, which emphasizes constant communication and 'shaping belief systems' within the company and across the industry. Rather than issuing sudden mandates, Huang gradually introduces ideas and insights, allowing employees, board members, and partners to internalize the vision over time. This approach ensures widespread buy-in for major strategic shifts, such as the acquisition of Mellanox or the company's 'all-in' commitment to deep learning, making these decisions seem obvious when finally announced. This proactive manifestation of the future extends to NVIDIA's external engagement, where GTC keynotes serve to align the industry with NVIDIA's long-term vision.

Finally, Huang discusses the four 'scaling laws' of AI: pre-training, post-training, test time (inference), and agentic scaling. He challenges the notion that inference is 'easy,' arguing it's a complex process of 'thinking' and reasoning, making it intensely compute-intensive. He predicts a future where AI training is limited by compute rather than data, with synthetic data playing an increasingly dominant role. The emerging 'agentic scaling law' suggests that AI will multiply its intelligence by spawning sub-agents, creating a continuous loop of data generation, memorization, refinement, and enhancement, ultimately driven by compute. NVIDIA's ability to anticipate these architectural shifts years in advance, through internal research, industry collaboration, and flexible architectures like CUDA, is critical to its continued success.

"The reason why extreme co-design is necessary is because the problem no longer fits inside one computer to be accelerated by one GPU."

"The goal of a company is to be the company is to be the machinery, the mechanism, the system that produces the output."

"The better computing company we become, the worse we became as a specialist. The more of a specialist, the less capacity we have to do overall computing."

"A computing platform is all about developers. And developers don't come to a computing platform just because, you know, it could perform something interesting. They come to a computing platform because the install base is large."

"Install base defines an architecture. Not... Everything else is secondary, okay?"

"I always say that NVIDIA is the house that GeForce built, because it was GeForce that took CUDA out to everybody."

"At some point, there's a reasoning system that, that convinces me, so clearly this outcome will happen. That this will happen. And so I believe it in my mind, and when I believe it in my mind, you know, you know how it is. You manifest a future and that future is so convincing, there's no way it won't happen."

"Inference is thinking, and I think thinking is hard. Thinking is way harder than reading."

"The next scaling law is the agentic scaling law. It's kind of like multiplying AI."

"Intelligence is gonna scale by one thing, and that's compute."

"You got to listen and learn from everybody. And have a... And then the last part is to have an architecture that's flexible, that can adapt and move with the wind."