A new system called MCUNet allows the design of tiny neural networks on IoT devices, even with limited memory and processing power. According to a paper by MIT scientists published on the preprint server Arxiv, the technology could bring new capabilities to smart devices while saving energy and improving data security. The research “is one of those brilliant ideas that seems obvious when you hear it,” John Suit, advising CTO at robotics company KODA, said in an email interview. “It’s an elegant approach to the problem. This research is so significant because eventually, they will allow for the real-time optimization of neural networks for any device where the resources can be known to the algorithm.”
Big Calculations on Small Devices
IoT devices usually run on computer chips with no operating system, making it difficult to run pattern-recognition tasks like deep learning. For more intense analysis, IoT-collected data is often processed in the cloud, though it’s vulnerable to hacking. There’s a lot that neural networks could do to enhance the growing number of IoT devices, but size has been a problem. “In order to move the networks down into the device themselves, which has proven difficult, you would need to find a way to optimize the search space for a variety of microcontrollers,” Suit explained. “A standard or generic system wouldn’t work because of the resource tolerances on IoT devices. Think very low power, very small processors in terms of processing power.” That’s where the work by the MIT researchers comes in. “How do we deploy neural nets directly on these tiny devices?” the study’s lead author, Ji Lin, a Ph.D. student in MIT’s Department of Electrical Engineering and Computer Science, said in a news release. “It’s a new research area that’s getting very hot. Companies like Google and ARM are all working in this direction.”
TinyEngine to the Rescue
The MIT group designed two components necessary for the operation of neural networks on microcontrollers. One part is TinyEngine, which is similar to an operating system, but strips code down to its essentials. Another is TinyNAS, a neural architecture search algorithm. “We have a lot of microcontrollers that come with different power capacities and different memory sizes,” Lin said. “So we developed the algorithm [TinyNAS] to optimize the search space for different microcontrollers. The customized nature of TinyNAS means it can generate compact neural networks with the best possible performance for a given microcontroller—with no unnecessary parameters. Then we deliver the final, efficient model to the microcontroller.” Lin’s work could translate to make smarter, more agile medical devices. “What this really shows is that power doesn’t have to be tied to size, and in hospitals, where everything moves quickly in tight spaces, that can literally mean the difference between life and death,” Kevin Goodwin, CEO of EchoNous, a company that makes AI-assisted medical devices, said in an email interview. Goodwin said his team spent years building and training a neural network that could then be used to map heart structures in a real-time ultrasound scan—all in a handheld device called KOSMOS that weighs under two pounds. “Now doctors can move easily from room to room getting diagnostic-quality scans with AI guidance,” he added. “They don’t have to send patients elsewhere for those scans or lose critical time disinfecting cart-based machinery.” MCUNet is an exciting look at a world where small gadgets could be smarter than ever. As the number of IoT devices grows rapidly, we’ll be looking for everything from smart appliances to medical devices to have their own neural networks.
