How Deep Learning Models Are Transforming Histopathology in Cancer Research

Predicting Molecular Alterations and Genetic Mutations

Results demonstrate that deep learning models are powerful tools to predict molecular alterations and genetic mutations from histopathology images. In cancer research, finding which specific genetic mutation is causing your cancer may lead to knowledge to guide you to a personalized treatment strategy, such as targeted therapy or immunotherapy. These mutations are traditionally identified through molecular assays that can be costly and time-consuming. However, histopathology images could be used to directly infer genetic information by AI models trained on histopathology images.

For instance, we have developed deep-learning models to predict MSI and dMMR in colorectal cancer, both key biomarkers in immunotherapy. Molecular analyses were eliminated by testing these models against H&E-stained slides, in which they successfully induced high accuracy to detect MSI. Similarly, deep learning algorithms have been employed to identify other known genetic mutations, BRAF and KRAS mutations, from routine histopathology images, each resulting in faster, cheaper diagnostics.

Tumor Classification and Subtype Prediction

Determining the appropriate treatment plans is based on tumor classification based on histopathological appearance. Traditionally, such classification has been based on visual inspection by pathologists who classify tumors into different subtypes based on features of their cellular content and tissue architecture. Automating this process with deep learning models has greatly increased the accuracy and consistency with which the tumors were classified.

Different types of cancers, like lung cancer, breast cancer, and colorectal cancer, have been classified using several AI-based models. As an example, deep learning models have been used to distinguish non-small cell lung cancer (NSCLC) adenocarcinoma from squamous cell carcinoma, two major subtypes with different treatment courses. These models can then analyze more subtle morphological differences that may not have been seen by the human eye and therefore have more precise and more objective classifying.

Deep learning models have also been used to classify the histopathological images of breast cancer by molecular subtypes like HER2-positive, ER-positive, and triple-negative breast cancer. Therefore, it is important to classify patients to determine the best-targeted cancer therapy, since each type of patient does not necessarily respond equally to hormonal therapy or a HER2 inhibitor.

Enhancing Prognosis Prediction and Treatment Response

Deep learning is one of the most promising applications of deep learning in cancer research for predicting patient outcomes and responses to treatment from histopathological images. Traditional prognostic factors—tumor stage and grade—give us valuable information but can’t fully capture the complexity of tumor biology. However, unlike previous methods in cancer, deep learning models can integrate information from histopathology images with genomic data to make more accurate predictions of patient survival or treatment efficacy.

Great potential for the prediction of patient prognosis from multimodal deep learning models combining histopathology images, radiology scans, and genomic sequencing has been demonstrated. Such models can also model the tumor microenvironment, and depending on which immune cells or stromal components are present, these patterns (e.g., the presence of immune cells or stromal components) are often found to be associated with disease progression and treatment response. Including this layer of information allows deep learning models to offer more actionable prognostic insights to clinicians by leveraging the patient heterogeneity that exists in these tumors to inform more specific treatment plans for each patient.

For instance, in renal cell carcinoma (RCC), deep learning models have been trained to derive five-year survival predictions from multiscale histopathological images, CT scans, and genomic data. These models discriminate significantly better than traditional clinical parameters, such as TNM stage and histopathological grading, and thereby predict patient outcomes better. Likewise, AI-based models have been developed to predict lymph node metastasis in colorectal cancer and have enabled more accurate risk stratification and decision-making regarding adjuvant therapy.

Discovering New Biomarkers and Biological Mechanisms

Besides, deep learning models are also employed to find new biomarkers and explore biological mechanisms underlying cancer progression to improve diagnostic accuracy and prognostic predictions. These models can examine large waves of histopathology images, find patterns that are linked to particular molecular modifications or clinical results, and reveal new insights into cancer biology.

However, one particularly promising use of deep learning in cancer research is in the identification of tumor-infiltrating lymphocytes (TILs), immune cells found in the tumor microenvironment. Indeed, TIL presence is associated with a favorable prognosis and is associated with a response to immunotherapy in cancers such as melanoma or breast cancer. Because histopathology images are fundamentally captured with the density and distribution of TILs, we can use deep learning models to automatically quantify the density and distribution of TILs so that we get a more standardized, objective view of immune infiltration.

Additionally, AI models have been developed to characterize the spatial distribution of cancer clones within tissues and dissect tumor heterogeneity and clonal evolution. These models can then map the spatial distribution of genetic mutations in a tumor to help researchers understand how different clones within the tumor interact with each other and respond to treatment. It would be very useful to determine which combination therapies are better at simultaneously targeting multiple clones because this information may inform how to design more effective combination therapies.

Addressing Challenges and Future Directions

Despite the enormous promise of deep learning models in histopathology, these technologies are still not ready for integration into clinical practice. The one major challenge is that we need large, annotated datasets to train AI models. Many research institutions can obtain digital histopathology ‘slides’ but they have been unable to automatically annotate these slides using an expert’s manual annotation. To combat this, more efficient annotation tools and potentially semi-supervised or unsupervised learning techniques could be developed.

A second issue is that deep learning models need to be generalizable to other patient populations and clinical settings. However, the performance of AI models can also be affected by variability in tissue preparation, staining techniques, and scanning protocols. As a result, researchers have started developing domain adaptation techniques to overcome this and combine different datasets from multiple institutions to make the deep learning models robust in real-world clinical use.

The integration of deep learning models into routine clinical workflows promises significant progress toward improving cancer diagnosis, prognosis, and treatment, looking ahead. With these models getting better and more advanced, the realm of histopathology is moving from being a subjective, labor-intensive field to a data-driven, highly efficient, morphologically precise, personalized care for cancer patients.

Conclusion

The field of histopathology is undergoing a revolution with deep learning models automating the analysis of tissue samples, improving diagnostic accuracy, and increasing our understanding of cancer biology. Potential clinical uses will range from the prediction of genetic mutations to subtype classification and enhancement of the prognosis prediction. Going forward, as technological advancements, new learning models will shape the future of cancer diagnosis and treatment in ways that will ultimately improve outcomes for patients.

References

1. Pocock, J., Graham, S., Vu, Q.D., Jahanifar, M., Deshpande, S., Hadjigeorghiou, G., Shephard, A., Saad Bashir, R.M., Bilal, M., Lu, W. and Epstein, D., 2021. Tiatoolbox: An end-to-end toolbox for advanced tissue image analytics. bioRxiv, pp.2021-12.
2. van Treeck, M., Cifci, D., Laleh, N.G., Saldanha, O.L., Loeffler, C.M., Hewitt, K.J., Muti, H.S., Echle, A., Seibel, T., Seraphin, T.P. and Trautwein, C., 2021. DeepMed: a unified, modular pipeline for end-to-end deep learning in computational pathology. BioRxiv, pp.2021-12.
3. Schulz, S., Woerl, A.C., Jungmann, F., Glasner, C., Stenzel, P., Strobl, S., Fernandez, A., Wagner, D.C., Haferkamp, A., Mildenberger, P. and Roth, W., 2021. Multimodal deep learning for prognosis prediction in renal cancer. Frontiers in oncology, 11, p.788740.
4. Krause, J., Grabsch, H.I., Kloor, M., Jendrusch, M., Echle, A., Buelow, R.D., Boor, P., Luedde, T., Brinker, T.J., Trautwein, C. and Pearson, A.T., 2021. Deep learning detects genetic alterations in cancer histology generated by adversarial networks. The Journal of pathology, 254(1), pp.70-79.
5. Sui, D., Liu, W., Chen, J., Zhao, C., Ma, X., Guo, M. and Tian, Z., 2021. A pyramid architecture‐based deep learning framework for breast cancer detection. BioMed Research International, 2021(1), p.2567202.
6. Schrammen, P.L., Ghaffari Laleh, N., Echle, A., Truhn, D., Schulz, V., Brinker, T.J., Brenner, H., Chang‐Claude, J., Alwers, E., Brobeil, A. and Kloor, M., 2022. Weakly supervised annotation‐free cancer detection and prediction of genotype in routine histopathology. The Journal of pathology, 256(1), pp.50-60.
7. Wulczyn, E., Steiner, D.F., Xu, Z., Sadhwani, A., Wang, H., Flament-Auvigne, I., Mermel, C.H., Chen, P.H.C., Liu, Y. and Stumpe, M.C., 2020. Deep learning-based survival prediction for multiple cancer types using histopathology images. PloS one, 15(6), p.e0233678.
8. Saillard, C., Schmauch, B., Laifa, O., Moarii, M., Toldo, S., Zaslavskiy, M., Pronier, E., Laurent, A., Amaddeo, G., Regnault, H. and Sommacale, D., 2020. Predicting survival after hepatocellular carcinoma resection using deep learning on histological slides. Hepatology, 72(6), pp.2000-2013.