by Sarah Bender
Artificial intelligence is reshaping research, teaching, and higher education. At Carnegie Mellon University Libraries, it's also reshaping how library employees work — not through isolated experiments, but through coordinated efforts to evaluate AI responsibly, build new approaches to information discovery, and prepare staff for an AI-enabled future.
Across the Libraries, faculty and staff are developing frameworks to assess emerging AI tools, building an AI reference agent that connects users with trusted Libraries information, and defining the skills employees need to evaluate and apply AI in their own work. They are also looking further ahead, using scenario planning to consider how the Libraries can prepare for an AI future that remains difficult to predict.
Together, these efforts are helping the Libraries determine not simply where AI can be used, but what employees need to understand about the technology to make informed decisions about how and when to use it.
Evaluating before adopting
When researchers began asking librarians about AI-powered research products, STEM Librarian Huajin Wang and her colleagues found that general warnings about hallucinations and unreliable citations weren't enough to advise someone about a specific tool. They needed to understand how AI tools fit into different points of the research workflow, and the specific benefits and drawbacks of using them.
The team developed the AI-Powered Tool Assessment Framework, which examines how a product retrieves information, generates responses, performs against its cited sources, and addresses usability and sustainability. Librarians used the framework to assess individual research tools and provide researchers with context about their strengths and limitations through the Libraries' AI in Research (AIR) program.
“Initially, we wanted to be able to educate ourselves,” Wang said. “AI is increasingly integrated into many of the tools the Libraries subscribes to, and one of our strengths is helping researchers evaluate how useful those tools are for their work. By sharing the framework, we wanted to give others a way to make those evaluations for themselves.”
The framework has since been used by librarians at other institutions, including The Ohio State University Health Sciences Library, where it influenced both how one librarian evaluates AI-generated research summaries and how she approaches teaching AI literacy.
While the framework gives librarians a method for evaluating technologies developed elsewhere, another Libraries project is giving staff experience shaping an AI system themselves.
Building AI with librarian expertise
STEM Librarian Haoyong Lan led the development of ScottAI, the Libraries' AI reference agent, with coding contributions from Zeyang Zhang, a master's student in CMU's Artificial Intelligence Engineering program, and input from Libraries colleagues. Launched in July 2025, ScottAI lets users ask questions conversationally and searches a growing network of Libraries sources.
Those sources include the Libraries Catalog, KiltHub, Scholars at CMU, databases and e-resources, and University Archives collections. Connecting the agent to that information, however, is only part of the work. ScottAI also needs to determine which source is appropriate for a question, recognize when an initial response misses the mark, and know when to ask for clarification.
“That's the real reference conversation librarians have with users — continuing to dig deeper, asking what they are looking for, and going back and forth until we get the most accurate information,” Lan said.
Librarians help shape instructions governing those interactions, determine which Libraries sources should be connected to the agent, and test it with questions drawn from their subject expertise. Building ScottAI is therefore giving employees experience not only using AI, but determining how an AI-enabled service should work: what information it can access, how it should handle ambiguity, and when a user needs human expertise.
That work raises a broader workforce question: What do Libraries employees need to understand about AI to do this work well?
Defining an AI-fluent workforce
The need for AI fluency extends beyond employees developing and evaluating AI-enabled services. Librarians are increasingly helping the CMU community develop those skills as well. Libraries faculty have worked with first-year writing students to examine when and how generative AI belongs in academic work, while designing workshops for AI Fluency Week on topics ranging from AI-assisted literature searching and prompt engineering to teaching, research, and coding.
As those responsibilities expand, the Libraries is also considering what AI fluency should mean for its own workforce. In June 2026, the Libraries formed an AI-Fluent Workforce Task Force to develop a roadmap for building those skills across the organization. The task force was nominated for an Andy Award this fall.
The roadmap organizes AI fluency around four areas: understanding AI, applying it, evaluating and creating with it, and using it ethically. Employees might learn to distinguish between AI and non-AI systems, decide when AI is appropriate for a task, verify generated information against authoritative sources, or identify privacy and copyright concerns before incorporating a tool into their work.
The framework doesn't assume every employee needs the same expertise. Staff members will work with supervisors to assess their skills and determine which areas of development are appropriate for their roles. Practical exercises range from testing a generative AI system for hallucinations and bias to developing AI-supported workflows, verifying AI-generated information, and considering the ethical implications of AI use.
“Our goal with the AI-Fluent task force was to demystify these emerging tools and support our staff and faculty through structured, low stress professional growth,” said Organizational Development Manager Kelly Woessner. “True fluency looks different across our libraries, whether that means refining prompts for research, using AI to speed up administrative workflows, or streamlining communications. Ultimately, it's about giving our employees the confidence and ethical grounding to decide when using AI makes sense for their work and when human expertise is the better fit.”
The roadmap is designed to evolve. Leadership will review it each semester, while an annual employee survey will help inform future revisions as the technology and its implementation at CMU evolve. That flexibility acknowledges a challenge inherent in preparing a workforce around AI: The tools employees use several years from now may look very different from those they are learning about today.
Building capacity for what comes next
At the Libraries' 2026 Summer Gathering, Helen and Henry Posner Jr. Dean of the University Libraries Keith Webster asked employees to consider that uncertainty directly.
Webster led a shortened version of the Futurescape Libraries AI Toolkit Workshop, which he developed for the Association of Research Libraries and the Coalition for Networked Information. The toolkit asks participants to consider how libraries might operate under very different AI-influenced research environments in 2035, building upon the ARL/CNI AI-Influenced Futures scenarios.
In one scenario, advances in AI and human-computer interaction have expanded access to knowledge and transformed research and collaboration. Another imagines AI developing primarily around profitable consumer applications, with relatively limited impact on research. A third explores the consequences of hasty adoption, including biased systems, misinformation, declining trust, and poorly conceived regulation. In a fourth, increasingly autonomous AI systems accelerate research while people relinquish some agency over the process.
The scenarios gave Libraries employees a way to examine questions that cut across their work: How might AI affect the scholarly record and the integrity of research data? What happens to digital literacy as AI systems take on more responsibility? How could changes in access, trust, policy, or human agency alter the role of a research library?
“The future we think most likely is usually today with the dials turned up, and forecasters have a poor record on questions like this one,” Webster explained. “If we plan only for that version, we’re well placed for one outcome and exposed to the rest. Working through several very different futures shows us which of our assumptions are fragile and which investments hold up whatever happens.”
The exercise places the Libraries' current AI work in a longer view. The assessment framework provides a method for investigating unfamiliar technologies. ScottAI gives librarians experience determining how an AI-enabled service should interact with trusted information and human expertise. The workforce framework identifies competencies employees can develop while acknowledging that those competencies will need to evolve.
None of those efforts depends on knowing which version of 2035 will arrive. Instead, they build capabilities that remain useful across different futures: evaluating unfamiliar systems, maintaining reliable information, recognizing where human judgment is necessary, and making informed decisions about where AI belongs in library work.
“We don’t need to know which future arrives,” Webster said. “In every one of them, someone has to be able to vouch for what’s true: where information came from, whether it can be trusted, and when a human needs to be involved. That has always been the library’s job. Doing it well now means staff who can evaluate systems they’ve never seen and the judgment to know where AI belongs and where it doesn’t. AI changes who, or what, is asking the questions. It doesn’t change the need for someone to answer for the answers.”