I've been a big evangelist for AI hardware, and have made 3 investments in the space. I think it the chips that are coming to market over the next few years will have huge long term impacts on AI applications, and build some really big winners in the hardware space. But the most interesting thing I read recently about AI chips is this article that Graphcore is going straight to production at the cutting edge 3nm process size.
When semiconductors are manufactured, the smallest pieces you can etch on a chip has been constantly declining. The 7nm process came online in 2017, and then 5nm started last year I think. 3nm is currently pushing the limits of semiconductor manufacturing. One of the benefits of AI chips is that they work fundamentally differently than much of the current digital semiconductor chips and so, a lot of AI chips are starting with older fabrication technology like 10nm, or possibly even larger. The performance gains from the new AI model architecture more than enough to offset the potential gains from using a smaller fabrication technology. And cutting edge fabrication technology can be challenging and buggy. So that's why I am left scratching my head about why Graphcore is jumping to 3nm.
In my mind, the best reason I can think of is that Graphcore, being "older" by AI architecture standards, might not be enough of a performance boost just architecturally, and thus has to compete with other AI chips by adopting cutting edge fabrication techniques to compensate. The other thing that makes me believe this is that many chips are releasing their MLPerf data, and Graphcore has not. So it is easy to weave together a story that says the reason they didn't release MLPerf data, and the reason they need to jump to 3nm are the same - underperformance by the core architecture relative to other AI chip leaders.
I have no inside information about Graphcore, so there are other explanations too. Maybe they feel like MLPerf is an unfair or irrelevant measuring stick. Maybe there are some business development or other benefits to jumping to 3nm that make it worth the risk. But absent evidence of that, the best story I have is that Graphcore risks underperforming without 3nm.
The AI chip explosion is fascinating to watch and it is certainly a very competitive and well funded industry, but the winners are going to win big.
Thanks for reading.