Testing the limits of what’s possible (and what isn’t) with AI

The researchers, from the University of Cambridge and the University of California Santa Barbara, designed 'adversarial' mathematical systems designed to fool any AI algorithm. Like ethical hackers stress-testing the security of a network, these adversarial systems were designed to map out exactly where and why AI prediction breaks down.

Many real-world systems - like those in oceans, the human brain, or robotics - are too complex to describe neatly with equations, so researchers often learn how they behave by using machine learning. But these AI methods don't always work well, returning unreliable results or poor predictions.

Sometimes, however, providing reliable solutions is fundamentally impossible, even with infinite data. The adversarial systems developed by the researchers may help developers and users of AI systems know whether they're working on a solvable or unsolvable problem, build methods that work, and avoid wasting time, effort or AI tokens when a problem is beyond the bounds of possibility.

Their results, reported in the journal Nature Communications, could also help explain why popular AI chatbots like ChatGPT or Claude can be accurate in the short term, but can drift or hallucinate over time.

"We're probing the boundaries of what you can and can't do with AI," said lead author Dr Matthew Colbrook, from Cambridge's Department of Applied Mathematics and Theoretical Physics. "It's so important to understand what problems can't be solved with these methods, because otherwise you end up wasting a lot of time and money."

Colbrook and his co-authors used an approach called Koopman operator learning, which turns complicated nonlinear behaviour into a linear form that's easier to analyse.

"What we were doing with these 'adversaries' was trying to figure out the types of systems that are hard or impossible to predict, and the types of systems that could be adapted to return reliable results," said Colbrook.

The researchers identified two main reasons why machine learning breaks down when analysing complex systems: either the algorithm can't tell when it's seen enough data to return a reliable result, or patterns in the system can be hidden or hard to distinguish.

"In a lot of AI research, a common assumption is that if we just collect more data, learning will eventually work," said Colbrook. "But we found this is often wrong. Learning is often layered, and requires multiple steps in the right order to work."

When a system is chaotic - meaning tiny differences in starting conditions lead to wildly different trajectories, like a choose-your-own-adventure story - the Koopman operator often ends up with a continuousspread of frequencies rather than clean, distinct modes. Short-term prediction was accurate, but long-term prediction became fundamentally unreliable, because the sensitivity to initial conditions compounds over time.

The same mathematical instability that defeats prediction algorithms may also explain why AI chatbots confidently fabricate facts: small changes in a question can send the chatbot down an entirely different path, one that looks plausible word-by-word but loses its grip on reality over longer outputs.

The researchers developed a way to classify these problems based on how many steps are needed to solve them. Where the data is not sufficiently layered or in the right order, the best an algorithm can do - even with infinite data - is 50/50, essentially classifying the problem as unsolvable.

The team also produced a new, provably reliable and highly efficient algorithm with built-in error bounds: essentially giving AI researchers a way to know when they're able to trust the answer, at a fraction of the cost of most supercomputers.

The researchers tested their approach on over 40 years of Arctic sea ice data. Using their algorithm they found hidden patterns in how the ice is declining, and were able to outperform current leading AI models at a fraction of the cost, on a standard laptop.

"We're at the stage now where there have been a lot of flashy examples and success stories in AI, but it's vital that we also ask how certain the models are, and how we know whether they're certain," said Colbrook. "Otherwise, we're building on very shaky foundations."

Reference:
Matthew J. Colbrook, Igor Mezić, Alexei Stepanenko. ' Adversarial dynamical systems characterize when data-driven learning succeeds or fails .' Nature Communications (2026). DOI: 10.1038/s41467-026-74220-8