DMU research uses grey-fuzzy framework to tackle 'black box' problem in AI predictions of rock permeability

Researchers at De Montfort University Leicester (DMU) have developed a new artificial intelligence framework that not only predicts how easily fluids can move through underground rock, but also indicates the reliability of each prediction. 

Led by Dr Ahmad Lawal,  Lecturer in Data Analytics at DMU, the researchers set out to tackle the so-called black box problem - one of AI's recognised weaknesses. 

Essentially, AI systems can be good at making predictions, but are often very poor at offering users any indication of  how much confidence should be placed in those predictions, or how their conclusions were actually reached.

An oil platform in the North Sea.

An everyday example of a black box problem might be someone applying for a loan and being told that a computer has rejected the application, but not being given any meaningful explanation as to how or why the decision had been made.

Such problems can be particularly significant when AI is being used to help make costly decisions about the development and management of oil and gas reservoirs buried deep underground.

The DMU-led team tackled the problem by combining three computational methods to improve the accuracy of AI predictions and make their reliability easier to understand.

Gaussian Process Regression, known as GPR, is a machine learning method that produces a prediction together with an estimate of the uncertainty surrounding it.

Grey Relational Analysis compares patterns within data and helps identify the model settings that produce the strongest and most useful relationships.

Fuzzy logic then translates statistical uncertainty into a more understandable scale of reliability. Rather than forcing every prediction into a rigid category such as "reliable" or "unreliable", it allows for different degrees of confidence.

The team called its three-pronged approach an Integrated Grey-Fuzzy Gaussian Process Regression framework. In this instance they used it to predict rock permeability - but say that it could just as easily be used to help avoid black box-type issues in relation to the use of AI in tackling other engineering questions.

Accurate permeability estimates are important when assessing a reservoir of oil or gas and deciding if and how it should be developed. Errors can contribute to poor investment decisions, inefficient operations and poorly informed reservoir planning.

By combining the three methods, the framework produces a permeability prediction, calculates the uncertainty surrounding it and converts that uncertainty into a reliability assessment that engineers can more easily interpret.

This allows users to distinguish between predictions in which they can have relatively high confidence and those that should be treated more cautiously.

The framework was tested using two different sets of geological information: Nuclear Magnetic Resonance logs from North Sea sandstone formations and conventional well logs from sedimentary basins in central Australia.

Across both datasets, the new approach consistently performed better than three established machine learning methods: XGBoost, Least Squares Boosting and K-Nearest Neighbours.

The framework achieved reliability scores of approximately 0.91 and produced substantially narrower prediction intervals than the alternative methods.

Dr Lawal, lead author of the study, said: "Many machine learning systems can produce a prediction, but in a high-stakes setting it is equally important to understand how much confidence should be placed in that prediction.

"Our framework doesn't just give engineers a single number. It weighs how accurate a prediction is against how confident the model claims to be. A narrow, confident range only earns a high reliability score when the accuracy backs it up. And when a model is confident but wrong, the framework flags it. This helps engineers know when the evidence is strong, and when they should be more cautious."

"Decisions about reservoir development can have very significant financial and operational consequences. This approach is intended to support, rather than replace, professional judgement by giving engineers more transparent and useful information."

The researchers also examined different ways of presenting reliability through fuzzy logic.

They found that triangular membership functions created clearer divisions between low, medium and high reliability, while Gaussian functions produced more gradual transitions between the categories. 

This suggests that the way reliability is presented could be adjusted according to the needs of engineers and other decision-makers.

The research was led by Dr Lawal as part of his doctoral work at DMU.

He worked with Professor Yingjie Yang, Professor of Computational Intelligence at DMU; Dr Nathanael Baisa, Senior Lecturer in Artificial Intelligence at DMU; and Professor Hongmei He, from the University of Salford.

The researchers say the framework can be adapted for different geological settings and types of data.

Its combination of predictive performance and interpretable uncertainty assessments could make it valuable in supporting reservoir characterisation, development and management.

The framework was tested using existing geological datasets rather than through a live industrial deployment. Further research could examine its performance when incorporated into operational reservoir assessment systems.

 The peer-reviewed study, Reservoir Permeability Prediction Using Integrated Grey-Fuzzy Gaussian Process Regression: A Comprehensive Framework for Uncertainty Quantification and Interpretability, is published in Volume 177 of Engineering Applications of Artificial Intelligence.