Reminder: there are no video outputs on these chatbot data center processors driving up the prices of graphics cards.
So they can’t even sell as used GPUs to crash the consumer GPU price market when the AI bubble pops.
This is a reminder that businesses aren’t “money focused calculation machines that optimize for the maximum possible profit.” They don’t worry about every little dollar, they just print money and use it to control you.
Raising prices for you is the goal, not a byproduct of some other smarter plan.
Some people don’t need the rest of this post, and it’s very long, so I’ll put it in a comment.
I’m confused — GPUs main function is to be able to do lot’s of calculations in parallel, vs a CPU which does one thing at a time (simplistically).
GPUs aren’t only used solely for video, it’s just that graphics are an excellent use case for this type of processing.
So I don’t think AI companies are buying GPUs for video output and more because they can process lots of training calculations in parallel. Like how bitcoin miners use GPUs even though there’s no video involved in that
To be more specific here, GPUs are really, really good at linear algebra. They multiply matrices and vectors as single operations. CPUs can often do some SIMD operations, but not nearly as well or as many.
Video games do a lot of LA in order to render scenes. At the bare minimum, each model vertex is being multiplied by matrices to convert from world space to screen space, clip space, NDC, etc which are calculated based on the properties of your camera and projection type.
ML also does a lot of LA. Neural nets, for example. are literally a sequence of matrix multiplications. A very simple neural net works by taking a vector representing an input (or matrix for multiple inputs), multiplies that by a matrix representing a node’s weights, then passes the result to an activation function. Then does that a bunch more times.
Both functions want GPUs, but both need different things from it. AI wants GPUs with huge amounts of memory (for these huge models) which are optimized for data center usage (using cooling designed for racks). Games want GPUs that don’t need to have terabytes of VRAM, but which should be fast at calculating, fast at transferring data between CPU and GPU, and capable of running many shader programs in parallel (so that you can render more pixels at a time, for example).