AI chatbots give us a narrow slice of knowledge: Researchers warn of ‘knowledge collapse’

• News
• All news
• AI chatbots give us a ...

• Technology
• Artificial intelligence

We are exposed to a much narrower range of knowledge through AI chatbots than through a basic web search. This is documented by a new study led by the University of Copenhagen. As more and more of us turn to AI chatbots for information, researchers warn that the risk of 'knowledge collapse' increases.

"Let me just ask the chatbot." For many of us, this has become an everyday phrase. Where we used to turn to Google to find information, AI chatbots have become a common way of getting answers to everything from what to make for dinner and how to word that difficult email to the boss, to what it actually means when interest rates rise.

But the large language models underpinning AI chatbots give us a significantly narrower range of information than a conventional web search. This is the finding of a new study led by researchers at the Department of Computer Science (DIKU) at the University of Copenhagen.

"Every language model we tested provides users with more uniform information than a simple Google search across all the topics we looked at. In other words, people are to a large extent exposed to the same information over and over again. So AI chatbots are not just changing how we find knowledge, but also which knowledge we have access to," says first author Dustin Wright, a former postdoctoral researcher at DIKU who is now an assistant professor at Aalborg University.

At least 18% less diverse than Google

The researchers tested 27 different large language models on 155 topics. For each topic, they used 200 different prompt formulations based on questions from real users. This generated a dataset containing around 70 million individual claims produced by the models.

An AI chatbot is a service such as ChatGPT, Gemini or Claude that you can communicate with by typing or speaking and that provides answers in return. The chatbot uses a so-called large language model to understand questions and formulate responses.

A large language model is the underlying AI model that powers the chatbot. It is trained on very large amounts of text and learns patterns in how words and sentences relate to one another.

When you type a question into an AI chatbot, the chatbot sends it to the underlying large language model, which generates a response by predicting which words are most likely to fit your question.

The results show that even the language model producing the most diverse answers - OpenAI's GPT-5 - provides at least 18.7 per cent less varied information than Google. The topics tested by the researchers ranged from nuclear weapons, marriage, pornography, racism and genocide to more country-specific topics such as Marine Le Pen, the Falklands War and K-pop.

According to the researchers, the fact that people are increasingly using AI models as their primary gateway to information could have significant consequences:

"We risk exposing people to fewer perspectives and a narrower range of knowledge. This could create a vicious cycle in which the most popular content becomes even more dominant, while other content is increasingly overlooked," says Professor at DIKU and senior author Isabelle Augenstein, adding:

"It's similar to globalisation. Today, you can buy the same products and find the same coffee chains almost everywhere in the world. That has many advantages, but it has also reduced diversity."

Why is diversity so low?

According to the researchers, the low level of diversity in language models is partly a result of how the models basically work. They compress the vast amount of text they are trained on and learn the patterns that occur most frequently. In the process, information that deviates from the most common patterns is filtered out.

The effect could be amplified if language models are increasingly trained on text produced by other AI models - something the researchers expect to happen. In that case, models would learn from their own outputs, which are already less diverse than the human-written texts on which they were originally trained.

If this process is repeated over several generations of models, the range of information could gradually become narrower. This is what the researchers refer to as 'knowledge collapse'.

"It's a worrying thought. However, we can see that the more recent models produce slightly more diverse answers than older models, so knowledge collapse is not happening yet. But the mechanism that could trigger it in the longer term is already there. So it is something we should be aware of," says Isabelle Augenstein.

Seek out different sources

The researchers therefore also hope that people will use AI thoughtfully:

"AI chatbot summaries can of course be useful - that is why so many people use them. But it is still important to seek out different sources in order to understand the nuances and get a broader picture - especially for the younger generation growing up with AI. We must not become so dependent on the technology that we stop understanding and thinking for ourselves," says Isabelle Augenstein.

The AI industry should also pay attention to the issue, the researchers argue:

"We hope AI developers will build language models in a way that preserves the breadth of knowledge available to us. We have developed a method that they can use to measure diversity in models, which could help ensure that future language models do not become less diverse," says Dustin Wright.

Contact

Isabelle Augenstein
Professor
Department of Computer Science
University of Copenhagen
[email protected]
+45 93 56 59 19

Dustin Wright
Tenure-Track Assistant Professor
Department of Computer Science
Aalborg University
[email protected]
+45 93 50 91 78

Maria Hornbek
Communications Consultant
UCPH Communication
University of Copenhagen
[email protected]
+45 22 95 42 83

• Technology
• Artificial intelligence

More stories