New AI models trained on physics, not words, are driving scientific discovery

While popular AI models such as ChatGPT are trained on language or photographs, new models created by researchers from the Polymathic AI collaboration are trained using real scientific datasets. The models are already using knowledge from one field to address seemingly completely different problems in another.

While most AI models - including ChatGPT - are trained on text and images, a multidisciplinary team, including researchers from the University of Cambridge, has something different in mind: AI trained on physics.

Recently, members of the Polymathic AI collaboration presented two new AI models trained using real scientific datasets to tackle problems in astronomy and fluid-like systems.

The models - called Walrus and AION-1 -can apply the knowledge they gain from one class of physical systems to completely different problems. For instance, Walrus can tackle systems ranging from exploding stars to Wi-Fi signals to the movement of bacteria.

That cross-disciplinary skillset can accelerate scientific discovery and give researchers a leg up when faced with small samples or budgets, said Walrus lead developer Michael McCabe, a research scientist at Polymathic AI.

"Maybe you have new physics in your scenario that your field isn't used to handling, and just can't burn the time working through all the possible models that might fit your scenario," said McCabe. "Our hope is that training on these broader classes makes something that is both easier to use and has a better chance of generalising for those users, as the 'new' physics to them might be something another field has been handling for a while."

The Polymathic AI team recently announced Walrus in a preprint on arXiv.org and presented AION-1 at the NeurIPS conference in San Diego.

Walrus and AION-1 are 'foundational models,' meaning they're trained on colossal sets of training data from different research areas or experiments. That's unlike most AI models in science, which are trained with a particular subfield or problem in mind.

Rather than learning the ins and outs of a particular situation or starting from a set of fundamental equations, foundational models instead learn the basis, or foundation, of the physical processes at work. Since these physical processes are universal, the knowledge that the AI learns can be applied to various fields or problems that share the same underlying physical principles. Foundational models have a number of benefits - from speeding up computations to performing well in low-data regimes to finding physics shared across different fields.

AION-1 is trained on data from astronomical surveys that are already massive in their own right, including the Sloan Digital Sky Survey (SDSS) and Gaia, containing more than 200 million observations of stars, quasars and galaxies totalling around 100 terabytes of data. AION-1 uses images, spectra and a variety of other measurements to learn as much as it can about astronomical objects. Then, when a scientist obtains a low-resolution image of a galaxy, for example, AION-1 can extract more information about it, learned from the physics of millions of other galaxies.

Walrus' domain is fluids and fluidlike systems. Walrus uses the Well - a massive dataset compiled by the Polymathic AI team, encompassing 19 different scenarios and 63 different fields in fluid dynamics. All in all, it contains 15 terabytes of data describing parameters such as density, velocity and pressure in physical systems as wide-ranging as merging neutron stars, acoustic waves and shifting layers in Earth's atmosphere.

"I continue to be awed by the fact that a multi-disciplinary physics foundation model works at all, let alone at this level," said Polymathic AI team member Dr Miles Cranmer from Cambridge's Department of Applied Mathematics and Theoretical Physics. "This question is part of what motivated us to start Polymathic in the first place, and Walrus feels like a nice checkpoint in this direction."

"Walrus feels like a real step toward general-purpose AI for physical simulation-a single foundation model you can adapt across many scientific problems instead of re-training from scratch each time," said Dr Payel Mukhopadhyay from Cambridge's Institute of Astronomy. "And because we've open-sourced the code and data, I'm genuinely excited to see what the community builds on top of it."

AION-1 and Walrus can use physics seen in a different case and apply it to learn about something new, like our own senses. "Multiple senses together - rather than one at a time - gives you a fuller understanding of an experience," the AION-1 team explained in a blog post about the project. "Over time, your brain learns associations between how things look, taste and smell, so if one sense is unavailable, you can often infer the missing information from the others."

Then, when a scientist is performing a new experiment or observation, they have a starting point - a map of how physics behaves in other similar situations. "It's like seeing many, many humans," said Shirley Ho, Polymathic AI's principal investigator. "When you meet a new friend, because you've met so many people before now, you are able to map in your head … what this human is going to be like compared to all your friends before."

Foundational models make scientists' lives easier by streamlining data processing. Scientists will no longer have to create a new framework from scratch for every project or task; instead, they can start with an already trained AI to use as a foundation. "I think our vision for some of this foundation model is that it enables anyone to start from a really powerful embedding of the data that they're interested in … and still achieve state-of-the-art accuracy without having to build this whole pipeline from scratch," said AION-1 lead researcher Liam Parker from the University of California, Berkeley.

Their goal is to make tools that scientists can use in their day-to-day research. "We want to bring all this AI intelligence to the scientists who need it," Ho said.

Adapted from a Simons Foundation media release.

The text in this work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Images, including our videos, are Copyright ©University of Cambridge and licensors/contributors as identified. All rights reserved. We make our image and video content available in a number of ways - on our main website under its Terms and conditions, and on a range of channels including social media that permit your use and sharing of our content under their respective Terms.