Stony Brook University researchers are developing tools to predict how much energy machine-learning models consume and help developers find ways to reduce that consumption without sacrificing accuracy. The two-year project, "Optimizing Machine Learning Models Using Accurate Energy Prediction," is supported by a $1M award through the Defense Advanced Research Projects Agency's (DARPA) Mapping Machine Learning to Physics (ML2P) program.

The Stony Brook team involved in this project includes Department of Computer Science faculty members Aruna Balasubramanian as PI, with Niranjan Balasubramanian and Anshul Gandhi as co-PIs, and Xiaorui Wang as a PI from Ohio State University.

As AI models become more powerful, the energy required to train and run them presents a growing challenge. For services that continually process requests, energy consumption adds to operating costs. On phones, sensors and autonomous systems, it can limit how long a device can function before needing to recharge.

Developers looking to reduce that consumption face a problem: existing prediction methods often provide only broad estimates of a model's energy use. Those estimates can miss how different parts of a model interact with one another, with optimizations and with the hardware used to run them.

ML2P's program aims to deliver technologies that could enable strategic use of limited power resources. The open sourced research outputs from the program is expected to enable energy-aware edge applications in a broad range of domains, including defense industrial domains.

"Many existing approaches that predict energy are too coarse-grained," said Aruna Balasubramanian. "They are very helpful for getting a broad estimate, but they are not precise enough when you need to decide things like which model to deploy or which hardware to run it on."

The team's approach breaks a machine-learning model into its basic components and represents its execution as a structured graph. The graph captures how individual components consume energy and how their contributions combine to produce the model's overall energy use. The researchers aim to use this detailed accounting both to improve predictions and to pinpoint where energy is being spent.

By the end of the project, the team plans to develop tools that help researchers make machine learning systems more energy efficient without sacrificing the level of accuracy their applications require. The tools could help users determine which models and optimization techniques work best for a particular task and hardware platform, including resource-constrained devices such as drones, while showing the tradeoffs between energy use and performance.

"This project would help pinpoint where energy is being spent inside a model, so developers can clearly see bottlenecks and fix them," said Aruna Balasubramanian. "Together, this would make it possible to design and deploy energy aware machine learning models that balance power and performance."