Optimizing Data Availability and Utilization in Deep Learning Accelerator SoCs

Deep learning accelerators are pivotal in accelerating computation-intensive tasks in modern AI applications. Optimizing the utilization of system resources, including shared cache, on-chip SRAM, and data movement mechanisms, is crucial for achieving superior performance and energy efficiency. In this study, we propose an efficient system architecture specifically tailored for deep learning workloads. Our architecture enables to reconfigure the last level cache as a scratchpad with prefetch capability, which eliminates cache misses and thereby offers resource efficiency, improved performance, and energy efficiency. By implementing a strategy to overlap accelerator execution with data movement, we achieved remarkable results, including a 14× speedup and %5 reduction in energy consumption for the ResNet50 benchmark when compared to the base system configuration. These findings demonstrate the substantial benefits of incorporating prefetch support and scratchpad reconfiguration in the last level cache, leading to enhanced performance and energy efficiency in real-world deep learning accelerator applications.