Optimizing Data Availability and Utilization in Deep Learning Accelerator SoCs

Cagla Irmak Rumelili Koksal*, Nihat Mert Cicek, Ayse Yilmazer Metin, Berna Ors

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Deep learning accelerators are pivotal in accelerating computation-intensive tasks in modern AI applications. Optimizing the utilization of system resources, including shared cache, on-chip SRAM, and data movement mechanisms, is crucial for achieving superior performance and energy efficiency. In this study, we propose an efficient system architecture specifically tailored for deep learning workloads. Our architecture enables to reconfigure the last level cache as a scratchpad with prefetch capability, which eliminates cache misses and thereby offers resource efficiency, improved performance, and energy efficiency. By implementing a strategy to overlap accelerator execution with data movement, we achieved remarkable results, including a 14× speedup and %5 reduction in energy consumption for the ResNet50 benchmark when compared to the base system configuration. These findings demonstrate the substantial benefits of incorporating prefetch support and scratchpad reconfiguration in the last level cache, leading to enhanced performance and energy efficiency in real-world deep learning accelerator applications.

Original languageEnglish
Title of host publicationICECS 2023 - 2023 30th IEEE International Conference on Electronics, Circuits and Systems
Subtitle of host publicationTechnosapiens for Saving Humanity
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798350326499
DOIs
Publication statusPublished - 2023
Event30th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2023 - Istanbul, Turkey
Duration: 4 Dec 20237 Dec 2023

Publication series

NameICECS 2023 - 2023 30th IEEE International Conference on Electronics, Circuits and Systems: Technosapiens for Saving Humanity

Conference

Conference30th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2023
Country/TerritoryTurkey
CityIstanbul
Period4/12/237/12/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

  • accelerator utilization
  • cache configuration
  • cache design
  • cache hierarchy
  • data availability
  • deep learning accelerators
  • energy efficiency
  • latency
  • prefetching mechanism

Fingerprint

Dive into the research topics of 'Optimizing Data Availability and Utilization in Deep Learning Accelerator SoCs'. Together they form a unique fingerprint.

Cite this