AI chips are getting hotter. A microfluidics breakthrough goes straight to the silicon to cool up to three times better.

For example, most Teams calls tend to start on the hour or the half-hour. The call controller gets very busy about five minutes before to three minutes after those times and isn't very busy the rest of the time. There are two ways to handle peaks in demand - install a lot of expensive extra capacity that isn't used most of the time or run the servers harder, which is called overclocking. Because overclocking makes chips even hotter, it can't be done too much or it can damage chips.

"Whenever we have spiky workloads, we want to be able to overclock. Microfluidics would allow us to overclock without worrying about melting the chip down because it's a more efficient cooler of the chip," Kleewein said. "There are advantages in cost and reliability. And speed, because we can overclock."

How cooling fits into the bigger picture

Microfluidics is part of a bigger Microsoft initiative to advance next-generation cooling techniques and optimize every part of the cloud stack. Traditionally, datacenters have been cooled with air blown by large fans, but liquids conduct heat much more efficiently than air does.

One form of liquid cooling Microsoft has already deployed in its datacenters is cold plates. The plates sit on top of the chips, with cold liquid coming in, circulating through channels inside the plates to pick up heat from the chips below, and hot liquid going out to be cooled down.

Chips are packaged with layers of materials to help spread their heat away from hot spots and to protect them. But these materials also act like blankets, limiting the performance of cold plates by holding in heat and keeping out the cold. Future generations of chips that work well for AI are expected to be even more powerful - and to get too hot to be cooled by cold plates.

Cooling chips directly through microfluidic channels is far more efficient - not just for taking away heat but also for running the overall system. With all those layers of insulation removed and coolant directly touching the hot silicon, the coolant doesn't need to be anywhere near as cold in order to do its job. That would save energy that won't be needed to chill the coolant, while doing a better job than current cold plates. Microfluidics technology also enables higher-quality waste heat use.

Microsoft also aims to optimize datacenter operations through software and other approaches. "If microfluidic cooling can use less power to cool the datacenters, that will put less stress on energy grids to nearby communities," said Ricardo Bianchini, Microsoft technical fellow and corporate vice president for Azure specializing in compute efficiency.

Heat also puts limits on datacenter design. One benefit of a datacenter for computing is that servers are physically close together. Distance slows communication between servers - something called latency. But today's servers can be packed together only so tightly before heat becomes a problem. Microfluidics would allow datacenters to increase the density of servers. That means datacenters could potentially increase compute without requiring additional buildings.

The future of chip innovation

Microfluidics also has the potential to open the door to completely new chip architectures, such as 3D chips. Just as putting servers close together reduces latency, stacking chips reduces it even more. This kind of 3D architecture is challenging to build because of the heat it generates.

However, microfluidics brings coolant extremely close to where power is consumed so "we might flow liquid through the chip," as would be the case with 3D designs, Bianchini said. That would involve a different microfluidics design using cylindrical pins between the stacked chips, a bit like pillars in a multilevel parking garage, with fluid flowing around them.

"Anytime we can do things more efficiently and simplify this opens up the opportunity for new innovation where we could look at new chip architectures," Priest said.

Removing the limit set by heat could also allow for more chips in a datacenter rack or more cores on a chip, which would improve speed and allow for smaller but more powerful datacenters.

By demonstrating how new cooling techniques such as microfluidics can be made to work, Microsoft hopes to help pave the way for more efficient and sustainable next-generation chips across the industry, the company said.

"We want microfluidics to become something everybody does, not just something we do," Kleewein said. "The more people that adopt it the better, the faster the technology is going to develop, the better it's going to be for us, for our customers, for everybody."

Related Links: