Purdue University’s HCiM Accelerator Enhances Deep Learning Efficiency, Reduces Power Consumption

Researchers at Purdue University’s Elmore Family School of Electrical and Computer Engineering have developed the Hybrid Analog-Digital Compute in Memory (HCiM) accelerator. This technology enhances the efficiency of Deep Neural Networks (DNNs) by reducing the power and area overhead linked to Analog-to-Digital Converters (ADCs).

The HCiM accelerator uses extremely low-precision quantization for partial sums, eliminating the need for ADCs. However, this introduces the challenge of managing numerous floating-point scale factors. To address this, the researchers propose an algorithm-hardware codesign approach using a digital CiM array dedicated to processing scale factors, thus enhancing processing speed.

What is the Hybrid Analog-Digital Compute in Memory Accelerator for Deep Learning Workloads?

The Hybrid Analog-Digital Compute in Memory (HCiM) accelerator is a new technology developed by researchers at the Elmore Family School of Electrical and Computer Engineering at Purdue University. This technology is designed to improve the efficiency of Deep Neural Networks (DNNs) by reducing the power and area overhead associated with Analog-to-Digital Converters (ADCs).

ADCs are essential for accumulating partial sums from crossbars, but they consume a significant amount of energy and occupy a large area in Compute in Memory (CiM) accelerators. The HCiM accelerator mitigates this issue by adopting extreme low-precision quantization for partial sums, eliminating the need for ADCs. However, this approach introduces the challenge of managing numerous floating-point scale factors, which are trainable parameters like DNN weights.

To address this, the researchers propose an algorithm-hardware codesign approach where DNNs are first trained with quantization-aware training. The HCiM uses analog CiM crossbars for performing Matrix-Vector Multiplication operations, coupled with a digital CiM array dedicated to processing scale factors. This digital CiM array can execute both addition and subtraction operations within the memory array, thus enhancing processing speed.

How Does the HCiM Accelerator Improve Energy Efficiency?

The HCiM accelerator improves energy efficiency by reducing the reliance on ADCs. ADCs are a primary source of power and area inefficiency within the CiM architecture, consuming as much as 60% of energy and occupying nearly 80% of the area in CiM accelerators.

The HCiM accelerator addresses this issue by adopting extreme low-precision quantization for partial sums, which eliminates the need for ADCs. This strategy effectively reduces ADC costs but introduces the challenge of managing numerous floating-point scale factors.

To overcome this challenge, the researchers propose a digital CiM array dedicated to processing scale factors. This digital CiM array can execute both addition and subtraction operations within the memory array, thus enhancing processing speed. Additionally, it exploits the inherent sparsity in ternary quantization to achieve further energy savings.

What are the Challenges Associated with the HCiM Accelerator?

While the HCiM accelerator offers significant improvements in energy efficiency, it also introduces several challenges. One of the main challenges is managing a substantial number of scale factors due to the fine granularity of Partial Sum Quantization (PSQ). This introduces energy and area overhead.

Furthermore, these scale factors have floating-point values that require complex hardware. Processing scale factors requires energy-intensive operations like multiplication or addition/subtraction. Reducing the number of scale factors results in a significant drop in accuracy.

To address these challenges, the researchers propose an algorithm-hardware codesign approach where DNNs are first trained with quantization-aware training. This approach ensures system accuracy while reducing the energy and area overhead associated with managing scale factors.

How Does the HCiM Accelerator Compare to Traditional Hardware?

Traditional hardware such as CPUs, GPUs, and specialized TPUs provide the computational power needed for increasingly complex tasks. However, they suffer from an efficiency barrier – the memory wall – arising from constant data movement between memory and processing units. This bottleneck not only slows down the computation but also increases power consumption.

In contrast, the HCiM accelerator performs computation within the memory array itself, thereby reducing data movement. This innovative solution helps overcome the limitations of traditional hardware.

However, the HCiM accelerator introduces a new challenge associated with ADCs. ADCs are essential for accumulating partial sum outputs from multiple crossbar arrays and are a primary source of power and area inefficiency within the CiM architecture. The HCiM accelerator addresses this issue by adopting extreme low-precision quantization for partial sums, eliminating the need for ADCs.

What is the Future of the HCiM Accelerator?

The HCiM accelerator represents a significant advancement in the field of deep learning workloads. By reducing the reliance on ADCs, it offers a more energy-efficient and area-efficient solution for accelerating DNNs.

However, the technology also introduces several challenges, including managing a substantial number of scale factors and processing these scale factors in hardware. Future research will likely focus on addressing these challenges and further improving the efficiency and performance of the HCiM accelerator.

The researchers at Purdue University have already proposed an algorithm-hardware codesign approach to address these challenges. This approach involves training DNNs with quantization-aware training and using a digital CiM array dedicated to processing scale factors. As the field of deep learning continues to evolve, the HCiM accelerator and similar technologies will likely play a crucial role in driving advancements.