A Look at the Race to Win the AI Chip Sector
DECEMBER 14TH, 2018
While the concept of artificial intelligence (AI) has been around for decades, it was the technological innovations of the past several years that brought the concept to reality. The combination of big data, faster processors, and more sophisticated algorithms moved AI from the drawing board into the mainstream, with companies big and small assessing the best way to use the technology to gain a competitive edge.
NVIDIA, with its pioneering graphics processing unit (GPU), has reaped the rewards of the recent emphasis on AI more so than any other company. NVIDIA was instrumental in providing the processors necessary to achieve the AI advances that are taking the world by storm. Researchers found that the GPU's ability to handle computationally intensive tasks provided the processing necessary to advance AI systems. The company's shear dominance of the field set off an arms race among technology companies to find the next generation of AI processor. Those that succeed could be the next big winner in a field that's only just getting started.
Companies have taken different approaches in the race to find a better solution than the GPU. Google just introduced the third generation of its TPU, which has eight times the computing power of its second-generation predecessor. At this point, the TPU is not available for sale, so it doesn't represent a direct threat, though it could slow the pace at which Google buys NVIDIA GPUs for its data centers.
In addition to Google, a flood of new processors are being developed at big tech co's. Intel got into the market with its purchase of startup Nervana Systems for $350m in 2016.
Microsoft is preparing an AI chip for its HoloLens VR/AR headset, and there’s potential for use in other devices.
Amazon is reportedly working on an AI chip for its Alexa home assistant.
Apple is working on an AI processor called the Neural Engine that will power Siri and FaceID.
ARM Holdings recently introduced two new processors, the ARM Machine Learning (ML) Processor and ARM Object Detection (OD) Processor. Both specialize in image recognition.
IBM is developing specific AI processor, and the company also licensed NVLink from Nvidia for high-speed data throughput specific to AI and ML.
Even non-traditional tech companies like Tesla want in on this area, with CEO Elon Musk acknowledging last year that former AMD and Apple chip engineer Jim Keller would be building hardware for the car company.
When it comes to the big tech co's designing AI chips, it seems like those companies are making custom silicon for their own use and will likely never bring them to market. Such entities have the billions in revenue that can be plowed into R&D of custom chips without the need for immediate and obvious return on investment.
There are also a plethora of startups working on the next generation of AI chips:
Graphcore – Raised $110m to-date (led by Sequoia). They are aiming for 200 teraflop per second performance. They’ve managed to get 600MB memory on-chip, but are primarily focused on improving compute performance on chip.
Wave Computing – Raised $117m (led by Samsung). Wave Computing raised a lot of money, but have only achieved 100 teraflop per second performance. Their architecture is not conducive to continued performance improvement and is very energy inefficient.
Groq – Raised $10m to date (led by Social Capital). Groq is the team that built the Google deep learning chip (TPU). They’re aiming for 4-5x increase over NVIDIA GPU performance, which puts them at 80-100 teraflop per second performance. Groq has similar bottleneck performance issues as the Google TPU, and similar to Wave Computing.
Mythic – Raised $55m to date (led by Softbank). Mythic uses an analog flash array. This requires power and area intensive ADCs and DACs, and does not solve the data movement problem (in fact it makes it worse). There are also limiting questions around their chip’s endurance and scaleability.
Syntiant – Raised $5m to date (led by Intel). Syntiant also uses an analog flash array. This requires power and area intensive ADCs and DACs, and does not solve the data movement problem (in fact it makes it worse). There are also limiting questions around their chip’s endurance and scaleability.
SambaNova – Raised $56m to date (led by Google Ventures). SambaNova uses a software defined approach to optimize deep learning chip performance.
Cerebras Systems – Raised $112m to date (led by Benchmark). Using an old approach to architecture that has historically proven dificult.
Brightstone has an investment in an AI cipmaker called Vathys. Its architecture is a clean rethink of the processor, unconstrained by legacy limitations. Vathys was born and bred for deep learning. Vathys aims to achieve one petaflop per second performance on a single chip. Current deep learning processors (NVIDIA GPU) achieve 20 teraflops per second, and ‘next gen’ processors (those listed above) achieve 100-200 teraflops per second. Vathys is 50x faster than currently available GPUs and 5-10x faster than ‘next gen’ processors.
Vathys takes a radically different approach to architecture. The company realized that data movement, not compute, is the performance bottleneck. All of the current and ‘next gen’ deep learning processors are focused on improving compute performance and ignoring 90% of the problem (data movement). Vathys solves this bottleneck by moving a significant amount of memory on-chip. This removes the need for data transfer on/off chip, a significant performance bottleneck. Elimination of on/off chip data movement also provides significant energy efficiencies over competing deep learning chips. Vathys is able to move 8GB memory on chip, while the best performing competitors only have 30-300MB on chip.
Like every other technology, this emerging AI chip industry likely won’t sustain its current plethora of competitors. Many internal-use chips that don’t come to market become pet projects for senior technologists, and a change of regime often means adopting the industry standard instead. Today’s army of AI chip startups will also diminish- many of those companies will simply hope to carve out a niche that might entice a big player to acquire them. A few will achieve real performance breakthroughs, and go on to generate significant value for their investors.
One day, this may means that the AI chip field will look a lot like those old chip fields—the x86, Nvidia GPU, ARM-worlds. But for now, this AI chip race has just gotten off the starting line, and its many entrants intend to keep running. At Brightstone, we believe the Vathys team is well suited to win this race.
