New chip design enhances precision in memory

Researchers from the University of Southern California (USC) have developed a new type of chip with an enhanced memory for edge AI (AI in portable devices). For decades, while the size of the neural networks needed for AI and data science applications doubled every 3.5 months, the hardware capability needed to process them doubled only every 3.5 years. According to Joshua Yang, Professor of Electrical and Computer Engineering at USC, hardware presents a more and more severe problem for which few have patience.

Governments, industry and academia are trying to address this hardware challenge. Some continue to work on hardware solutions with silicon chips, while others are experimenting with new types of materials and devices. Yang’s work focuses on exploiting and combining the advantages of the new materials and traditional silicon technology that could support heavy AI and data science computation.

The research paper published in Nature focuses on the understanding of fundamental physics that leads to a drastic increase in memory capacity needed for AI hardware. The team led by Yang, with researchers from USC, MIT and the University of Massachusetts, developed a protocol for devices to reduce ‘noise’ and demonstrated the practicality of using this protocol in integrated chips. This demonstration was made at TetraMem, a startup company co-founded by Yang and his co-authors to commercialise AI acceleration technology.

According to Yang, this new memory chip has the highest information density per device (11 bits) among all types of known memory technologies thus far. Such small but powerful devices could help bring incredible power to a range of electronic devices. The chips are not just for memory but also for the processes. And millions of them in a small chip, working in parallel to rapidly run AI tasks, could only require a small battery to power it.

The chips that Yang and his colleagues are creating combine silicon with metal oxide memristors in order to create powerful but low-energy intensive chips. The technique focuses on using the positions of atoms to represent information rather than the number of electrons (which is the current technique involved in computations on chips). The positions of the atoms offer a compact and stable way to store more information in an analog, instead of digital fashion. Moreover, the information can be processed where it is stored instead of being sent to one of the few dedicated ‘processors’, eliminating the ‘von Neumann bottleneck’ existing in current computing systems. In this way, Yang said, computing for AI is “more energy efficient with a higher throughput”.

Yang said that electrons which are manipulated in traditional chips are “light”. This lightness makes them prone to moving around and being more volatile. Instead of storing memory through electrons, Yang’s team is storing memory in full atoms. Normally, Yang said, when one turns off a computer, the information memory is gone — but if you need that memory to run a new computation and your computer needs the information all over again, you have lost time and energy. This new method, focusing on activating atoms rather than electrons, does not require battery power to maintain stored information. Similar scenarios happen in AI computations, where a stable memory capable of high information density is crucial. Yang said this technology may enable powerful AI capability in edge devices, such as Google Glasses, which he said previously suffered from a frequent recharging issue.

By converting chips to rely on atoms instead of electrons, chips become smaller. Yang said that with this new method, there is more computing capacity at a smaller scale. And this method, he said, could offer “many more levels of memory to help increase information density”.

Image credit: iStock.com/Mykola Pokhodzhay