Boise State researchers aim to make AI faster, more energy efficient

Principal Investigator Omiya Hassan pictured with her co-principal investigators Purab Sutradhar and Karthik Srinivasan. Photo by Torin Alm

Every time you ask a chatbot a question or generate an AI image, electricity moves through a maze of wires between memory and processor. That trip, not the math itself, is what's driving up the energy cost of artificial intelligence. As AI models become bigger, this bottleneck gets worse, increasing electricity use and putting more pressure on the power grid.

Omiya Hassan, assistant professor in the College of Engineering's Department of Electrical and Computer Engineering, or ECE, received nearly $500,000 from the National Science Foundation to help close the gap between memory and processing.

The three-year project, titled "3D Integrated Parallel Fabrics enabling Layered Opto-electronic Processors for Near-memory AI Computing," or 3DPFLOPS, will teaching AI chips to "think" in both electricity and light. It combines light-based computing, 3D-stacked chip architecture, and smart workload scheduling into a single design aimed at cutting the energy AI systems burn by just moving data back and forth.

If 3DPFLOPS hits its goals, Hassan said the effects could reach far beyond the lab. Faster AI response times, lower energy costs per query, and a smaller environmental footprint for the data centers behind the popular AI tools are just a scratch on the surface of its impact. The same near-memory efficiency gains could also let phones, wearables and other devices run more AI features while saving battery life and reducing dependence on distant, power-hungry servers.

AI's query bottleneck

Most computers today, Hassan explained, still rely on a decades-old blueprint called von Neumann architecture, which keeps memory and processing separate.

For everyday users, the costly journey from memory to processing translates into higher subscription costs, slower response times, new data centers popping up near communities, and a mountainous strain on regional power grids.

The power of light

"Electronics are ideal for the precise calculations needed during AI training and complex computations, while photonics [the science and technology of generating, controlling and detecting light] excel at moving vast amounts of data quickly during inference," Hassan said. "This ability to switch between the two based on the task is what makes this approach so innovative and exciting."

Omiya Hassan, Ph.D. leads the Low-Power Circuits and Embedded Systems Laboratory. Photo by Torin Alm.

Light-based computing, like photonics, moves data as light rather than electrical current. Multiple data streams can even travel together as different "colors" of light, a technique called wavelength-division multiplexing.

Hassan's team is pairing the technique with a 3D-stacked chip design that places memory and computing layers vertically, like the floors of a building. This shortens the distance data has to travel in an approach known as "near-memory computing."

This system is what sets 3DPFLOPS apart from the commercial efforts already underway at companies like NVIDIA, Intel and Micron, which Hassan noted are exploring photonics mainly as a data interconnect rather than a first-class part of the architecture.

"AI model sizes and the power they require have grown much faster than what traditional chip scaling can handle, while recent advances in photonic components and 3D chip-stacking manufacturing have become mature enough to be practically integrated into AI accelerator designs, moving beyond just experimental lab prototypes," Hassan said.

Building regional benefit

Long before the AI boom, Hassan's path to 3DPFLOPS traces back to her doctoral research on power-efficient hardware for wearable biomedical devices.

"With the rapid rise of AI and the widespread construction of data centers across the country, I feel like it's now more important than ever to turn our attention to the architecture of computers themselves," Hassan had said.

These early career research faculty look to explore electronic architecture to solve AI's most pressing challenges. Photo by Torin Alm.

It was this conviction that shaped the team behind the grant. Hassan, accompanied by Assistant Professors Karthik Srinivasan and Purab Sutradhar, brought their expertise in photonics and memory architecture, respectively. Along with industry partner Luceda Photonics, a leader in photonic circuit design tools, the team is confident their award will address critical energy challenges.

The grant's reach will also support two existing Boise State student courses and fund a new multidisciplinary workshop spanning digital, analog and photonic computing, which includes hands-on experiences with industry-standard photonic design tools and mentorship from Luceda Photonics.

With Boise State students learning and conducting research alongside the electrical and computer engineering team, Hassan believes this initiative will prepare students in the community for the growing workforce needs in the region, making it more than just an academic effort.

Research reported in this publication was supported by the National Science Foundation's Division of Information and Intelligent Systems under award No. 2615713. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Science Foundation.

Share This

• Share on Facebook
• Share on X
• Share on LinkedIn
• Share through Email