Microsoft Research has developed an artificial intelligence (AI) model that can convert routine cancer pathology slides into detailed immune system data, a step that could make large-scale immunotherapy research more accessible. The work, done with Providence and the University of Washington, was published in “Cell” on Dec. 9.
The study focuses on a major challenge in cancer care. Immunotherapy helps the immune system recognize and attack tumors, but results vary widely among patients. Some tumors respond well, while others show little or no response. These differences are often linked to how cancer cells and immune cells interact inside the tumor, a setting known as the tumor immune microenvironment.
To study this environment, researchers use a technique called multiplex immunofluorescence, or mIF. It can show many immune-related proteins at once and reveal where cells are located within a tumor. This information can help predict whether a tumor is likely to respond to immunotherapy and guide treatment decisions. However, mIF tests are expensive and hard to scale. Running them on a single tissue sample can cost thousands of dollars, limiting their use to small studies.
Microsoft Research addressed this issue by developing GigaTIME, a multimodal AI model that can generate virtual mIF images from standard hematoxylin and eosin, or H&E, pathology slides. H&E slides are routinely used in hospitals and are widely available, making them a practical starting point for large studies.
GigaTIME was trained using a Providence dataset containing 40 million cells with paired H&E slides and mIF images across 21 protein channels. After training, the model was applied to data from 14,256 cancer patients treated across 51 hospitals and more than 1,000 clinics within the Providence health system. This process generated about 300,000 virtual mIF images spanning 24 cancer types and 306 cancer subtypes.
From this virtual population, researchers identified 1,234 statistically significant links between immune cell activity and clinical factors such as biomarkers, cancer stage, and patient survival. Many of these links matched findings from earlier studies, while others had not been reported before. The results were further tested using data from 10,200 patients in The Cancer Genome Atlas, where similar patterns were observed.
“GigaTIME is about unlocking insights that were previously out of reach,” said Carlo Bifulco, MD, chief medical officer of Providence Genomics and medical director of cancer genomics and precision oncology at the Providence Cancer Institute. “By analyzing the tumor microenvironment of thousands of patients, GigaTIME has the potential to accelerate discoveries that will shape the future of precision oncology and improve patient outcomes.”
The study is described as the first population-scale analysis of the tumor immune microenvironment based on spatial protein data. Previous research was limited by the small number of mIF samples available. By using AI to translate routine pathology slides into detailed immune data, Microsoft Research and its partners were able to study patterns across a much larger group of patients.
The researchers also found that combining information across all 21 protein markers provided clearer patient grouping by disease stage and survival than relying on single markers alone. The virtual data revealed spatial patterns and interactions between proteins that were difficult to examine using traditional methods.
To support wider research use, Microsoft Research has made the GigaTIME model publicly available through Microsoft Foundry Labs and Hugging Face, with the goal of helping researchers explore cancer immune responses using existing clinical data.