Obviously, we see new GenAI model announcements every day, but as an old chip guy, I've been intrigued by the supremacy of Nvidia's GPU chips as the GenAI compute platform of choice. I've pondered elsewhere Nvidia's repeated success. Their first boom came from building GPUs for computer graphics (the G in GPU) then booming a second time as the best platform for mining BitCoin and now a booming a third time (market cap $1.7T 🤯 ) in GenAI.
But the GenAI technology ecosystem seems to be evolving much faster than normal so there's also some amazing chip evolution going on. We watch this evolution carefully at Ryght as we it can have a huge impact on our software architecture.
So, this week saw a potentially disruptive chip innovation: Startup Groq announced a completely novel architecture for GenAI. It's radically different the GPU-approach and magically manages to be both more general purpose and more special purpose at the same time. A great visual is the attached two chip photos - Groq is comprised of numerous small blocks repeated many times and Nvidia a much smaller number of much larger and more complex blocks. Their trick seems to be to use a sophisticated compiler (if you have to ask...) to remap complex asynchronous LLM data flows into simple (synchronous) data flows that match their simple architecture. The results seems to be astounding - their chips built in a very basic 14nm chip fab process outperform (though lots of debate about this!) Nvidia's chips are built in a 4nm (12x better) process.
This reminds me of something very similar that happened long ago with more traditional computer chips - CPUs. Intel ruled the world with x86 Complex Instruction Set Computers (CISC) and along the Reduced Instruction Set Computers (RISC) like ARM. These days it seems ARM may have won, but it took decades. And once again, compilers were a big part of the magic.
Anyway, I'm not going to go into any detail here but it's wildly complicated but fascinating and suggest read the paper if you're a computer architecture buff - https://lnkd.in/dD9pCasX.
About Ryght
Ryght is a privately held healthcare technology company based in Anaheim, California that is developing the next generation of safe and secure generative artificial intelligence (GenAI) solutions for the biopharma industry. The Ryght platform leverages and optimizes multiple large language models (LLMs) and vector databases to ingest real-time data streams and make actionable knowledge directly available to biopharma discovery, clinical, and commercial teams. The platform enables healthcare professionals to rapidly leverage the power of GenAI within compliance of data security standards required by the industry.
Follow the company on 