Days ago I wanted to make a comparison of AI computing improvements from architecture to show off to someone and have let Gemini do it for me already, but after today’s news, I want to post something else first.
1) The accelerating computing of AI is all about parallel computing.
The very core of matrix calculation of AI is exactly the product-sum between 2 vectors X and Y, in which X=(x_1, x_2, x_3, …) and Y=(y_1, y_2, y_3, …), and the product-sum of X and Y is XY=x_1*y_1+x_2*y_2+x_3*y_3+….
The main part of AI computing is doing product-sum between one X and many Ys at once. So you need to accelerate compute through not only to multiply many x_n*y_n of one product-sum parallelly at same time but also to do many product-sums of pairs between one X and many Ys.
This is exactly where GPU is better than CPU on matrix compute. A GPU have way more cores to do way more product-sums and other computes at once than a CPU, and a GPU with its dedicated memory is a sub-computer focusing on and optimized for parallel computing only.
2) Jensen is Jobs and Cook of Nvidia
When Openai realized that scale up in different dimensions (weights, training data, etc) can effectively improve AI model’s intelligence level, they found that the only machine that can help them make scale up is Nvidia computer with Cuda. For except Jensen no others were willing to invest so much to develop a chip and computer focusing on parallel computing with full operators (like exp(), sin(), cos()) in parallel pipeline too through Cuda.
Just like Jobs for Apple, Jensen’s vision and insist on Cuda gave Nvidia today’s massive advantage on AI hardware, which also enabled Openai to scale up to ignite this AI revolution.
And more than that, Nvidia’s present dominance in AI hardware actually comes much more from its dominance on supply chain.
Just like Cook for Apple, Jensen’s aggressive management and vision built Nvidia a dominace on today’s AI hardware supply chain and market.
Be First to Comment