Cohan wins NSF CAREER Award to build more reliable AI for science

Large language models have already shown real promise as tools for scientific work. They help researchers sift through enormous bodies of literature, summarize prior findings, answer technical questions, and generate new hypotheses. But these systems come with significant limitations in high-stakes scientific settings. They can overlook key publications, draw on weak or irrelevant evidence, express unwarranted confidence in uncertain claims, or produce answers that sound authoritative but are not supported by current scientific evidence.

“Researchers are already using AI agents to help with scientific work: searching the literature, analyzing evidence, using tools, running experiments, and reasoning through difficult research questions,” Cohan said. “That creates enormous opportunities to accelerate science. But in scientific settings, reliability and validity are critical. If these systems give misleading answers, cite evidence that does not actually support the task, overlook important prior work, or fail in ways that are hard to detect, they can send research in wrong directions. This project is about building the foundations needed to make AI systems more trustworthy, evidence-grounded, and useful for science.”

Cohan addresses these challenges along three fronts. The first focuses on evaluation – developing new frameworks to rigorously measure how well AI models perform on scientific tasks, including their ability to retrieve relevant evidence, reason through challenging problems, and produce detailed analyses. The second develops new methods for adapting LLMs to the particular demands of scientific domains, drawing on the rich structure of scientific literature and experimental data. The third focuses on reliability and interpretability: developing methods to better understand where AI systems succeed or fail on scientific tasks, make their outputs more understandable and transparent, and generally improve their trustworthiness in research settings.

The research focuses on core LLM methodologies while having broad applications across scientific disciplines, including computer science, medicine, biology, and physics – fields where scientific knowledge is expanding rapidly and the consequences of error can be high.

A key emphasis of the project is openness and accessibility. Cohan's team will release publicly available datasets, benchmarks, software, and tutorials designed to help researchers across institutions use these technologies more effectively and responsibly.