Five Growing Concerns about Generative AI for Librarians and Information Professionals 

Meeting the technology where it's at

A woman with a large exclamation point, annoyed by her growing concerns with generative AI for librarians

When it comes to generative AI, enthusiasm, panic, and backlash play equally well. They generate a lot of clicks and create a market for consultants, writers, and other entrepreneurs. At LibTech Insights, we see generative AI as a major change within the information landscape but try to take a measured approach. And I think we can be honest that generative AI has not only large or theoretical problems (e.g., academic integrity, copyright questions, exploitative deepfakes) but also practical ones undercutting its basic use value. 

It’s easy to lose sight of the fact that these tools are still in their preliminary stages, and some still require a lot of development. All uses of them are essentially beta tests, and for beta tests to be helpful, they need to determine both what’s working and what isn’t. To that end, this post seeks to highlight new or growing areas of concern around generative AI. The point isn’t to delegitimate these tools or create clickbait from alarmism (though thank you for clicking, and please share this article with a friend!), but to draw attention to the problems that have been emerging as this global beta test continues to deepen and expand. 

Generative AI has intellectual limits 

Despite their diversity and proliferation, free generative AI chatbots remain, well, not that intelligent. I had a conversation with ChatGPT (3.5) about Thomas Mann’s novel The Magic Mountain (1924), and its responses were no more illuminating than the Wikipedia or SparkNotes page and demanded much more effort. Similarly, I asked ChatGPT to explain the philosophical oeuvre of Peter Sloterdijk. Though I appreciated the readability of its output—a neat summary delivered in a list format with bolded subheadings—I felt the responses were shallow. My attempts to get more substantive answers returned replies that were somehow both more specific and equally vague.  

I’m told the intellectual quality of these chatbots increases significantly in their paid versions. The trouble is that most university students won’t pay for a subscription when a free version is available. As it currently stands, instructing undergraduate students to collaborate with AI chatbots for assignments may be no more intellectually rewarding than having them complete partner work with a loquacious eighth grader. Anecdotally, some college educators have confirmed my suspicions: 

A tweet from Ben Collier reading "Taught two classes this year on using LLMs in sociological research. It's been great! Long three hour sessions going through all the applications with students, practical exercises, etc. Every one of which, we conclude, is an expensive way of doing existing methods quite badly."
via Ben Collier on X

These tools might become rapidly more intelligent and affordable in the next five years, with universities buying subscription packages for their campuses to use. But we need to be honest about where these tools are right now, or we risk doing students a disservice by overestimating the capabilities of this software. This doesn’t mean that AI has no place in the curriculum. Rather, given AI’s current limits, we need to be very specific about what that place is. We might be too hasty in treating it as a “collaborator.” 

Generative AI hasn’t solved its misinformation problem 

It was clear from its advent that generative AI would have the potential to become a huge vector for misinformation. Companies have tried adding guardrails in their products and banners admonishing users to check the chatbots’ outputs against other sources. Harvard’s Misinformation Review published an article assuring readers that “[f]ears about the impact of generative AI on misinformation are overblown” and simply part of “an old and broad family of moral panics surrounding new technologies.” Understandably, with an election year in the US, political anxieties can easily feed into technological fears, because misinformation has especially potent real-world stakes. Reports that AI chatbots are supplying “inaccurate, misleading responses” about elections, however, suggest that the misinformation problem is more than a “moral panic.” 

As stewards of information literacy and democratic institutions, librarians are right to be concerned with misinformation and its effects on public trust. I’m impressed by the librarians developing information literacy skills and framework that are attuned with the advancements in artificial intelligence. Rather than incubate an atmosphere of paranoia in our information environment, we also need to become comfortable saying which guardrails are insufficient and which are still necessary to install. Public trust is easy to break and hard to rebuild. 


🌟 Subscribe to the LibTech Insights newsletter for weekly roundups and bonus content, including: 


Generative AI uses dubious information quality 

Much has been written about hallucinations, i.e., information that LLMs fabricate to complete a request. Obviously, hallucinations, particularly in the form of fake citations, pose a hurdle for students and scholars using LLMs for research. But I want to draw attention to a related, but underappreciated, problem: information quality.  

Information professionals concerned with copyright are, naturally, troubled by the unknown datasets that LLMs use for training. Likely to stall lawsuits over mishandled copyrighted material, most LLMs do not disclose this information. For researchers, however, this “black box” is a huge problem. How do you know which sources a LLM has checked and which it hasn’t? Is the information it’s using the best or simply the most convenient? What “gaps” exist in the training data that influence its output? Do these gaps re-entrench biases? 

Perplexity has taken a step in the right direction by including citations in its responses. The citations allow users to verify the chatbot’s claims, which is good. But the sources it sometimes draws on from the open web are… not great. Certainly not the caliber that anyone teaching an information literacy session would hold up as models. As an instructor, I would balk if a student submitted a paper with GradeSaver.com on the references list. (At the very least, pick a website whose name doesn’t reek of desperation!) It’s possible that this technology, harnessed to academic databases, might prove very, very useful one day. But as it currently stands, the scholarly applications are limited.  

(That said, having students use Perplexity to answer prompts and then evaluate the quality of its sources would make a great InfoLit exercise for thinking critically about how LLMs work.) 

Generative AI exacerbates privacy concerns 

Many sites, apps, and software offer the privacy bargain: you purchase personalization with your privacy. At first glance, the bargain seems fair: if a software is going to learn that you prefer saying “next to” instead of “alongside,” then it needs data to go on. That’s just a basic fact of how digital technologies work.  

But the life of these data—what happens to them behind the scenes—is neither clear nor can be assumed to be innocuous. Many AI companies, including part of OpenAI, are for-profit, and we know that data are highly valuable to tech companies. ChatGPT’s new “memory,” for instance, potentially deepens its usefulness by remembering your writing style or the names of your children (OpenAI’s example—not mine!). But it also contributes to the company’s data profile on you—a hot commodity. ChatGPT allows users to delete certain “facts” from its memory through its settings page. But that very limited expression of user agency seems like false comfort for how much of your data remains outside of your control. 

Generative AI can’t shake off harmful stereotypes

We have covered before how racial and gender biases seep into algorithms and AI. Companies such as OpenAI and Google reassure us that they are installing guardrails to protect against the use and dissemination of biases. Still, the title of a recent report puts the problem into stark relief: “AI chatbots use racist stereotypes even after anti-racism training.” These biases can have high stakes. In an example from the article, chatbots are more likely to recommend the death penalty in cases where someone tried for first-degree murder uses African American English. Researchers have also found that gender stereotypes continue to circulate in ChatGPT’s responses. These problems place strong limits on who this technology is for and the applications it should have, because these biases can have strong repercussions.

Conclusion 

As I said, my aim in writing this post is not to stoke alarmism. Rather, I think an honest reckoning with the shortcomings of generative AI is necessary for developing and implementing AI literacy. We are told that AI is a revolution that will change everything. Maybe that’s correct, but those changes will take place unevenly and over time. We shouldn’t treat the tools as they currently stand as completed, and we also shouldn’t take their perfection as inevitable. We need to think of ourselves beyond the roles of passive consumers or adaptive survivalists in a hostile technological landscape. Librarians are, and always have been, proactive thinkers and doers and should guide these preliminary stages of AI use and development. 


🔥 Sign up for LibTech Insights (LTI) new post notifications and updates.

📅 Join us for a free webinar on library content strategy and web design.

✍️ Interested in contributing to LTI? Send an email to Deb V. at Choice with your topic idea.