There's no need to actually read a whole research paper when you can ask a bot to explain it to you. Scientific papers can transform into AI agents that, according to the Stanford team behind the project, should speed up the dissemination of new scientific discoveries. 

Paper2Agent, the team's new framework described in a paper published in Nature on Wednesday, converts scientific papers and their associated research outputs into agents that can discuss a paper’s findings, reproduce analyses and results, apply its methods to new data, and even collaborate with other paper agents on new research problems.

“Papers have been static documents for centuries,” James Zou, a Stanford computer scientist and biomedical data science professor and one of the paper’s authors, said in a LinkedIn post announcing P2A’s publication. “Paper2Agent turns them into active AI agents that can answer questions, apply their methods, and collaborate with other papers to make new discoveries.”

Giving a large language model access to a scientific paper is unpredictable, Zou added. What his team wanted was an agent that could act as a “virtual author” that had hands-on experience with a paper’s work, not just reading it and attempting to understand it. 

What that means in practice, as explained in the paper, is a workflow that uses a paper and its associated data, repository, and codebase to create a Model Context Protocol (MCP) server exposing the research's tools, resources, and workflows. An LLM agent can then connect to the server and use natural-language requests to autonomously run demonstrations, reproduce analyses, apply a paper's methods to new data, and the like.

“Paper2Agent agentifies the full research outputs, including manuscripts, supplementary materials, code, datasets, executable examples and analysis workflows,” the researchers explained in their writeup. 

According to the paper, the MCP server itself can be hosted remotely, but Zou explained to The Register in an email that it can also be run locally to protect sensitive information, though such info will still be sent to whichever LLM backend P2A is connected to. 

“If the user has sensitive data (e.g. protected health information) that they don't want to send to an LLM then they should exclude that data from P2A,” Zou told us, adding that P2A should be compatible with any AI coding agent, but that not all have been tested. 

AI hallucinations are an obvious concern, and the paper notes that researchers should always evaluate anything P2A presents to be sure it’s correct.

“We … view Paper2Agent as a tool for augmenting scientific discovery and improving access, reproducibility and reuse of papers, rather than as an autonomous or authoritative source of scientific conclusions,” the paper notes. 

That said, P2A still does its best to prevent such errors from cropping up. The paper explains that each tool used by a paper agent is validated against the paper’s results and figures and “locked to ensure reproducibility.” This decreases hallucination risk and minimizes randomness, but still - best to double-check its work.

Tests to see how well P2A scales appear to have gone well, with the researchers evaluating it across 136 papers in three groups, including 100 computational-biology papers. Of those 100 papers, 74 were successfully turned into agents, with the researchers attributing failures largely to incomplete codebases, missing documentation, or unresolvable environment configurations.

Still, it’s a massive leap in what could be possible when it comes to getting new scientific discoveries into the hands of more researchers. 

P2A is open source, and is available on GitHub for those who want to take a stab at it. There’s also a live version online that can explain the P2A paper in more detail and reproduce its results, and Zou tells us it can also ingest other papers to see how it works on their projects.

Zou explained to us that he hopes the open source community will help improve P2A, but notes that his team isn’t done with it either. Next up, they hope to create an online platform for paper agents to collaborate and discuss various “agentified” scientific discoveries. Let’s just hope those boffin bots behave a bit better than their counterparts. ®