Conventional processing architectures don’t quite cut it when it comes to bringing AI to the edge. Typical von Neumann architectures and AI don’t mix well, since large amounts of data movement in and out of memory, along with high clock speeds, are not conducive for low-power, high-performance AI processing. Many companies have turned to in-memory computation as the solution, but problems still remain.
The movement of data from memory to the processor can consume 200 times more energy than the computation itself. This poses a serious challenge for AI hardware. Image from Feng Shi et al.
AI acceleration company Mythic has taken a novel approach to these problems: replacing digital computation with analog computation. We interviewed Mythic’s CEO, Mike Henry, to get details on their newest product, the M1108, and how they’re using analog computation to bring AI to the edge.
Analog Performance With a Digital-Like Footprint
Analog computers may sound antiquated, but it turns out that they offer a lot of benefits over traditional digital computing. Most notably, analog computing has the potential to offer huge power and performance savings over digital computers. However, analog computation has mostly been limited by the fact that analog memory elements are historically very large, eventually limiting speed and scalability.
The M1108. Image used courtesy of Mythic
Looking to harness the benefits of analog computing while side-stepping the memory concerns, Mythic combined embedded flash memory with analog computation.
“Now we have the best of all worlds,” CEO Mike Henry comments. “We got the power and performance of analog compute that’s been known for decades. And we have the storage and compute density from the flash memory. That is the core innovation that Mythic has been building over many years.”
Combining Analog Compute With Flash Memory
Mythic is calling its newest product, the M1108, the industry’s first AI analog matrix processor (AMP).
As mentioned, the system works by combining true analog compute-in-memory with the high memory density of flash, delivering 8-bits of on-chip memory.
The chip utilizes Mythic AMP, which is a dataflow architecture consisting of an array of 112 “tiles.” Each tile contains an analog compute engine, a digital SIMD vector engine, a 32-bit RISC-V nano-processor, an NoC router, and local SRAM. No external DRAM is required.
Mythic’s tiled architecture. Image used courtesy of Mythic
“Traditionally, raw vector matrix computation is handled in the analog domain. Everything else is handled in the digital domain,” Henry explains. “We’ve blended the two things together, which is something that we’ve been perfecting over eight years.”
The tiled system also provides Mythic with the ability to scale up or down very easily. Tim Vehling, SVP of Product & Business Development at Mythic, says, “Scaling up is easy for us. If we want to run bigger models or more models, we can add more chips on the PCI express card. If we want to make a smaller version, we could do a 4 x 4 array of tiles or a 3 x 3 array of tiles.”
He adds, “There are ways to do quite a few derivatives off this product, either by adding chips in an upward fashion or downscaling the number of tiles to hit smaller price points and sizes.”
Mythic’s M1108: AI Analog Compute in Action
Merging analog and digital domains in this way has yielded a product that offers data center performance at significantly less power, smaller size, and lower cost.
Built on 40nm CMOS process technology, the chip supports INT4, INT8, and INT16 operations while also being able to run single or multiple complex DNNs entirely on-chip—specifically, with a capacity of up to 113M weights.
Mythic M1108 vs. NVIDIA Xavier AGX. Image used courtesy of Mythic
Running ResNet-50 at its highest framerate, the M1108 has shown a peak of 35 TOPS, 870 fps, and only 4 W of power consumption. Compared to NVIDIA’s Xavier AGX, the M1108 demonstrated better performance, less power consumption, smaller area, and cheaper price.
Analog Compute for Edge AI is Here
While analog compute may seem outdated to many, Mythic claims to have revived it and tapped into its full potential with the new analog matrix processor. Combining analog compute with flash memory, the device demonstrates some impressive specs, too—even compared to data center powerhouses like NVIDIA.
With high performance, low power, low area, and a highly-scalable architecture, Mythic’s new M1108 seems poised to meet the performance and price challenges of high-end edge AI applications, like surveillance cameras and vision systems. On the M1108, Henry optimistically reported, “This is how we live up to the promise of AI.”
Vehling adds, “This is probably the first time engineers can actually can tap into and utilize the power of analog compute. It’s here, it’s arrived, and it works great.”
