Teaching AI to speak the language of pathology

Artificial intelligence is already helping clinicians spot patterns in diagnostic images and detect signs of disease. But many of today’s pathology AI systems are built for a single task and must be rebuilt for each new application.

A study  recently published in Nature Medicine  describes a different approach aimed at advancing research in this field. Researchers from Microsoft Research and Paige, now part of Tempus, developed PRISM2, a pathology foundation model trained on both tissue images and language based on real pathology reports. The goal was to help AI learn from the large amount of available text data and create a system that could support future pathology research and development.

In testing, the model matched or exceeded the performance of specialized cancer-detection systems on several benchmark tasks, including prostate cancer, breast cancer and breast lymph node metastasis detection. Importantly, researchers reported these results without creating a separate model for each task.

The full PRISM2 model weights are publicly available for research use on Hugging Face  here and here.

Why it matters 

Pathology sits at the center of many cancer diagnoses. Pathologists examine tissue samples and create reports that guide treatment decisions. As healthcare generates larger volumes of data, researchers have been exploring how AI can support clinicians in making these decisions and possibly even deliver new insights.

Most pathology AI systems today are designed for a single purpose, such as detecting a particular type of cancer. PRISM2 was developed as a research effort to support a broader range of tasks through a single model that can respond to prompts and questions. Researchers say that could make it easier to build and adapt future pathology tools without starting from scratch for each new application.

How it works 

The central idea behind PRISM2 is that pathology is not only a visual discipline. It is also a language-driven one.