AOS Memory Boosts GPU Performance and Reduces Power Consumption

The relentless pursuit of increased computational performance in modern graphics processing units (GPUs) increasingly encounters limitations imposed not by processing capability, but by memory architecture. Current GPU designs rely heavily on static random-access memory (SRAM) for rapid data access, yet the physical scaling of SRAM is proving increasingly difficult, hindering further density improvements and contributing to rising power consumption. Researchers are now investigating alternative memory technologies to overcome these constraints, specifically exploring the integration of amorphous oxide semiconductors (AOS) to create persistent, high-bandwidth memory solutions. A team from the School of Electrical and Computer Engineering at the Georgia Institute of Technology, comprising Faaiq Waqar, Ming-Yen Lee, Seongwon Yoon, Seongkwang Lim, and Shimeng Yu, detail their investigation in the article, “CMOS+X: Stacking Persistent Embedded Memories based on Oxide Transistors upon GPGPU Platforms”. Their work focuses on the potential of combining conventional complementary metal-oxide-semiconductor (CMOS) technology with AOS-based memory cells to enhance GPU performance and energy efficiency.

Contemporary graphics processing units (GPGPUs) increasingly face performance limitations due to bottlenecks in register file speed and last-level cache bandwidth, restricting the efficient delivery of data to the multiple processing elements within a single instruction stream, known as single-instruction multiple-data (SIMD) lanes. Consequently, research focuses on novel memory technologies and architectural optimisations to circumvent the stagnation of traditional static random-access memory (SRAM) scaling. This work explores integrating amorphous oxide semiconductor (AOS) transistors with capacitive, persistent memory configurations, specifically assessing the viability of utilising AOS-based gain cells in multi-ported, high-bandwidth banked memories to alleviate critical bottlenecks in modern GPU architectures and enhance both memory performance and energy efficiency.

The research team conducted a detailed analysis of the density and energy trade-offs associated with back-end-of-line (BEOL) integrated memories, employing monolithic three-dimensional (M3D) integration techniques to construct multiplexed arrays that maximise bandwidth and minimise latency. Careful consideration was given to macro-level design constraints and the impact of process variations, ensuring the robustness and long-term reliability of the technology. BEOL integration refers to building memory structures after the primary logic transistors have been fabricated, leveraging existing interconnect layers. M3D integration stacks multiple layers of silicon vertically, increasing density and reducing interconnect lengths.

The resulting AOS-based memory system demonstrates a compelling combination of performance, power efficiency, and cost-effectiveness, positioning it as a viable alternative to established memory technologies such as high bandwidth memory (HBM) and graphics double data rate 6 (GDDR6). HBM and GDDR6 are high-performance memory standards currently used in high-end GPUs. Researchers identified areas for further optimisation, including exploring novel materials and device architectures.

A novel memory controller architecture was developed to effectively exploit the high bandwidth and low latency of the AOS-based memory system. This controller implements a data placement strategy that minimises memory access latency and maximises data locality, alongside a cache coherence protocol that ensures data consistency across multiple cores and memory banks. Data locality refers to keeping frequently accessed data close to the processing units, reducing access times.

A thorough analysis of the power consumption characteristics of the AOS-based memory system revealed key sources of dissipation, prompting the implementation of optimisation techniques to minimise energy usage. The system demonstrates a significant reduction in standby power compared to traditional SRAM-based memories, contributing to improved overall energy efficiency.

The potential of AOS-based memory for emerging applications, including artificial intelligence, machine learning, and high-performance computing, was actively explored. The high bandwidth and low latency of the system significantly accelerates these applications, enabling new levels of performance and efficiency. Researchers also investigated the use of AOS-based memory for in-memory computing, where computations are performed directly within the memory itself, reducing data movement and energy consumption.

The design and implementation details of the AOS-based memory system are meticulously documented, providing a comprehensive guide for future researchers and engineers. The design files and simulation models have been released publicly, fostering collaboration and accelerating innovation in the field of memory technology.

The study concludes that AOS-based memory represents a promising technology for addressing the growing memory demands of modern GPUs and other high-performance computing systems. The demonstrated improvements in performance, power efficiency, and cost-effectiveness warrant further research and development to fully realise its potential and pave the way for a new generation of high-performance computing systems.