Study background
As biomarker testing expands and cancer patients live longer, pathology reports have become longer and more complex. They often run to dozens of pages and include results from several institutions. That volume of information creates a heavy workload for physicians who have to assemble clinical summaries under time pressure.
A new study from Northwestern Medicine, published in JCO Clinical Cancer Informatics, tested six open-source language models on the task of summarizing cancer pathology reports and compared their output with summaries written by practicing physicians.
Methods and findings
The researchers analyzed 94 de-identified reports from patients with lung cancer. The reports included histology, immunohistochemistry, and molecular and genomic data relevant to treatment. The language models were asked to generate structured summaries, which a team of oncologists then compared with the original physician-written summaries.
The six tested models were Llama 3.0, 3.1, and 3.2 from Meta, Gemma 9B from Google, Mistral 7.2B, and DeepSeek-R1. All are open-source models that can run locally.
The result: the AI-generated summaries were consistently more complete, with a clear advantage in capturing molecular and genomic findings. DeepSeek-R1 and Llama 3.1 had the best performance.
Why this matters for pathologists
The largest gap appeared in the capture of molecular findings, especially data that can change treatment decisions. In daily practice, one missed detail in a long report can mean a missed therapeutic opportunity. This study suggests that language models can act as an additional checking layer, helping ensure that clinically relevant findings are not lost.
The practical use case does not require large infrastructure. Open-source models can run locally, which means they could be deployed inside a hospital environment without sending patient data to external cloud services. The Northwestern team is currently developing an application based on Llama 3.1 that allows physicians to upload reports and receive immediate summaries.
Important caveats
The study was limited to 94 lung cancer reports. Generalizing the results to other tumor types will require additional testing. The application is also still in the testing stage and has not entered clinical use. The researchers emphasize that the goal is to support physicians, not replace them.
Source: Toward Automating the Summarization of Cancer Pathology Reports Using Large Language Models to Improve Clinical Usability, JCO Clinical Cancer Informatics