Coming to a library near you: chatbots!

How can conversational agents such as OpenAI's ChatGPT, often referred to as chatbots, best be used professionally by university libraries? What skills do librarians need in order to master these tools and use them for the benefit of library users?

Université de Montréal librarians Teresa Bascik and Stéphanie Pham-Dang explore these questions in an article published in the latest issue of the quarterly French-language journal Documentation et Bibliothèques.

In particular, they look at three key functions of university libraries - reference services, user training and metadata creation - and consider the potential and limitations of chatbot technologies when used in each one.

Like 'Puss in Boots'

AI-powered chatbots are like the character Puss in Boots in Charles Perrault's 17^th century fairy tale, they say: eloquent and forward-thinking. Playing on the French title of the fable (Le Chat botté), as well as the English word "chatbot" and the word "robot," they dub chatbots "le chat (ro)botté."

They see a parallel with Perrault's cunning cat because, while conversational agents can be useful research tools-for example, they can generate bibliographic reference lists in a flash-they have a tendency to invent information that seems plausible at first glance but turns out to be false.

"For example, when asked for sources on AI ontologies, ChatGPT provided entirely fictitious bibliographic references, inventing article titles and author names such as John Smith," said Bascik. "It shows the importance of having humans verify the information generated by these tools."

In the course of their research, the authors did note a significant improvement in the quality of results between ChatGPT versions 3.5 and 4. They argue that these tools could become valuable assistants for librarians, provided they are used judiciously and under human supervision.

Tested with nursing students

For the second function of libraries, user training, Pham-Dang tested the use of ChatGPT in a session with nursing students on research data management.

"AI was used to generate a fictitious research project that served as the basis for practical exercises," she explained. "Students were asked to analyze ChatGPT's responses and find its errors and omissions - a way to develop their critical thinking and understanding of the issues involved."

How did it work out?

Using ChatGPT "made the training more interactive and engaging, but it must be borne in mind that the success of such applications depends, first and foremost, on the pedagogical skills of the librarians," Pham-Dang said. "The technology is only a tool to support the learning process."

Generating bibliographic records

Bascik and Pham-Dang also looked at the use of conversational agents for metadata creation, a crucial and time-consuming task for libraries. They tested CatalogerGPT, a GPT-4-based tool that generates bibliographic records in the MARC 21 format used by libraries, with mixed results.

"CatalogerGPT demonstrated its ability to generate syntactically correct MARC records from just an image of the book's cover and it also showed some knowledge of cataloging principles by correcting itself and explaining its choices," said Bascik.

"However, it also made significant errors. For example, it assigned a classification code for Russian literature to a Ukrainian novel, revealing potential biases in its training data. It also tended to invent information that wasn't available from the image."

So these tools could become valuable helpers in automating the cataloguing process, but only under the close supervision of information professionals who have acquired new skills in this area, the two librarians believe.

Mastering clear 'prompts'

In their article, they also discuss the importance of another new skill for librarians: mastering "prompts", i.e. learning how best to phrase questions for conversational agents.

"This skill is similar to the art of traditional documentary research, but in a chatbot context," said Pham-Dang. "You have to know how to formulate clear, precise and contextualized queries to get the best possible answers from AI."

The authors also stress the importance of developing a sharp critical sense to assess the relevance and accuracy of AI-generated information. They believe librarians need to cultivate their ability to adapt and continuously learn to keep up with the rapid evolution of these technologies.

Making librarians more efficient

Bascik and Pham-Dang are convinced that conversational agents will become everyday partners for librarians, making them more efficient in their work with no risk of actually replacing them.

With chatbots, "we can imagine cutting-edge reference services supporting research, interactive and personalized training, and scholarly papers and catalogues automatically enriched with precise metadata for optimal discoverability," they suggest in their article.

In their view, the advent of these tools represents both an opportunity and a challenge for university libraries. Librarians therefore need to tread carefully.

"How we use them will determine their impact in the realm of information and knowledge," they write.

"With a thoughtful and ethical approach, these high-tech 'Puss in Boots' could well become the librarian's best allies in their mission to disseminate knowledge-provided we securely fasten our digital boots and learn how to dance with our new partners."