Detail publikačního výsledku

Energy Complexity of Convolutional Neural Networks

ŠÍMA, J.; VIDNEROVÁ, P.; MRÁZEK, V.

Originální název

Energy Complexity of Convolutional Neural Networks

Anglický název

Energy Complexity of Convolutional Neural Networks

Druh

Článek WoS

Originální abstrakt

The energy efficiency of hardware implementations of convolutional neural networks (CNNs) is critical to their widespread deployment in low-power mobile devices. Recently, a number of methods have been proposed for providing energy-optimal mappings of CNNs onto diverse hardware accelerators. Their estimated energy consumption is related to specific implementation details and hardware parameters, which does not allow for machine-independent exploration of CNN energy measures. In this letter, we introduce a simplified theoretical energy complexity model for CNNs, based on only a two-level memory hierarchy that captures asymptotically all important sources of energy consumption for different CNN hardware implementations. In this model, we derive a simple energy lower bound and calculate the energy complexity of evaluating a CNN layer for two common data flows, providing corresponding upper bounds. According to statistical tests, the theoretical energy upper and lower bounds we present fit asymptotically very well with the real energy consumption of CNN implementations on the Simba and Eyeriss hardware platforms, estimated by the Timeloop/Accelergy program, which validates the proposed energy complexity model for CNNs.

Anglický abstrakt

The energy efficiency of hardware implementations of convolutional neural networks (CNNs) is critical to their widespread deployment in low-power mobile devices. Recently, a number of methods have been proposed for providing energy-optimal mappings of CNNs onto diverse hardware accelerators. Their estimated energy consumption is related to specific implementation details and hardware parameters, which does not allow for machine-independent exploration of CNN energy measures. In this letter, we introduce a simplified theoretical energy complexity model for CNNs, based on only a two-level memory hierarchy that captures asymptotically all important sources of energy consumption for different CNN hardware implementations. In this model, we derive a simple energy lower bound and calculate the energy complexity of evaluating a CNN layer for two common data flows, providing corresponding upper bounds. According to statistical tests, the theoretical energy upper and lower bounds we present fit asymptotically very well with the real energy consumption of CNN implementations on the Simba and Eyeriss hardware platforms, estimated by the Timeloop/Accelergy program, which validates the proposed energy complexity model for CNNs.

Klíčová slova

energy complexity, upper bound, lower bound, convolutional neural networks

Klíčová slova v angličtině

energy complexity, upper bound, lower bound, convolutional neural networks

Autoři

ŠÍMA, J.; VIDNEROVÁ, P.; MRÁZEK, V.

Rok RIV

2025

Vydáno

19.07.2024

Nakladatel

MIT PRESS

Místo

CAMBRIDGE

ISSN

0899-7667

Periodikum

NEURAL COMPUTATION

Svazek

36

Číslo

8

Stát

Spojené státy americké

Strany od

1601

Strany do

1625

Strany počet

25

URL

https://direct.mit.edu/neco/article-abstract/36/8/1601/121120/Energy-Complexity-of-Convolutional-Neural-Networks?redirectedFrom=fulltext

BibTex

@article{BUT189311,
  author="ŠÍMA, J. and VIDNEROVÁ, P. and MRÁZEK, V.",
  title="Energy Complexity of Convolutional Neural Networks",
  journal="NEURAL COMPUTATION",
  year="2024",
  volume="36",
  number="8",
  pages="1601--1625",
  doi="10.1162/neco\{_}a\{_}01676",
  issn="0899-7667",
  url="https://direct.mit.edu/neco/article-abstract/36/8/1601/121120/Energy-Complexity-of-Convolutional-Neural-Networks?redirectedFrom=fulltext"
}