Removing information from AI models is more difficult than it sounds

Researchers at the Universitat Rovira i Virgili have studied the effectiveness of unlearning techniques in artificial intelligence models. These strategies seek to eliminate personal, incorrect or discriminatory data from large language models such as ChatGPT, Mixtral, Bard or Copilot, among others. The analysis reveals that there is currently no method that guarantees total and irreversible erasure, other than retraining the model without the data in question, a very costly and inefficient process. This creates a conflict with the right to be forgotten, which is enshrined in European legislation and which obliges data controllers to delete people's personal data upon request. The solution to this incompatibility, they argue, is to design new ways of training models that facilitate unlearning with guarantees.

The performance of artificial intelligence (AI) models - also called large language models (LLMs) - depends on the data they are trained on. The companies that run them feed them with as much information as they can from as wide a range of sources as possible to make them more powerful and, above all, better than their competitors. These are huge models with billions of parameters and they know many, many things. "In some cases, they even know things that, for various reasons, they should not know," explains Josep Domingo, researcher at the Department of Computer Engineering and Mathematics and co-author of the research.

So what happens when an AI model has been fed with copyrighted works - could it mimic the style of a particular writer and write a sequel to the latest bestseller? And, if the model has personal information, does it know if someone has been ill, taken sick leave or just bought, say, a flat? Fortunately, we have legal mechanisms to protect all this data, such as the Spanish Intellectual Property Law or the European Union's General Data Protection Regulation (GDPR).

The GDPR regulates the processing of personal data of any natural person in the European Union and includes, among other aspects, the right to be forgotten. Therefore, when data controllers receive any request to remove personal data from their systems, they have to comply. And that includes all companies with AI models operating in Europe. However, the way these models have been configured makes deleting specific data a much more complex technical challenge than it might seem.

Against this backdrop, researchers from the CRISES Research Group have studied the ability of large language models to unlearn and the computational cost of doing so. There are mainly two approaches to removing knowledge from an AI model. The first option is the most rudimentary and involves removing all the knowledge and training the model again without the data that you want to eliminate. "This is an impractical and computationally very costly process, but it is currently the only way to guarantee one hundred per cent removal," says David Sánchez, researcher at the Department of Computer Engineering and Mathematics and co-author of the study.

The other approach to unlearning involves getting the model to forget specific information, which avoids the need to retrain the model from scratch every time some information needs to be removed. The underlying problem, according to the researchers, is that nobody fully understands how LLMs work, not even the people who have designed them. Although it is known how to train them and how to make them more efficient and accurate, there is no way of knowing in which region of the model a particular piece of information resides. Sánchez warns, however, that the aforementioned methods, although much more efficient, do not fully ensure unlearning and reminds us that the Regulation is very clear and requires absolute safeguards to be put in place.

A conflict between law and technology

The results of the study show that there is currently a conflict between the legislation and the available technology: that is, while it is possible to remove personal data from AI models, the only techniques that guarantee its removal are "frighteningly expensive". In this regard, Domingo points out that the administrators of these models will only implement guaranteed unlearning if so requested by lots of users: "If people see that these models contain their personal data and they start to request to be forgotten, the companies could have problems". LLM owners operating in Europe will need to take into account the GDPR and its right to be forgotten, and that means making unlearning affordable and cost-effective from a computational and economic point of view.

The URV researchers believe that in order to find more efficient ways of unlearning, the models must first be trained with unlearning in mind. Currently, LLMs are trained by feeding them all the data at once, but there are several alternatives, which still need to be developed. Some, for example, involve fragmenting the data and feeding it piecemeal to successive versions of the model so that, if a request to be forgotten is received, it is possible to retrieve an earlier version of the model that does not have the knowledge question. Others have to do with the structure of the system and are based on modular learning, which allows parts of the model with specific information to be extracted without affecting the rest of the information stored or the capabilities that the model can legitimately retain.

Reference: Blanco-Justicia, A., Domingo-Ferrer, J., Jebreel, N. M., Manzanares-Salor, B., & Sánchez, D. (2025). Unlearning in Large Language Models: We Are Not There Yet. IEEE Computer Society. https://doi.org/10.1109/MC.2024.3468588