Multimodal Foundation Models in Cancer Medicine: Integrating Imaging, Pathology, Multi-Omics, and Longitudinal Clinical Data

Abstract

Cancer remains one of the leading causes of morbidity and mortality worldwide despite remarkable advances in molecular biology, targeted therapeutics, immunotherapy, and precision medicine. The increasing availability of multimodal biomedical data—including radiological imaging, digital pathology, genomic sequencing, transcriptomics, proteomics, metabolomics, laboratory biomarkers, electronic health records, wearable technologies, and longitudinal clinical information—has created unprecedented opportunities for personalized oncology while simultaneously presenting major computational challenges. Recent developments in multimodal foundation models have introduced a new paradigm in cancer medicine by enabling unified representation learning across heterogeneous biomedical modalities. Unlike conventional artificial intelligence (AI) systems that analyze individual data types separately, multimodal foundation models integrate imaging, pathology, multiomics, clinical narratives, physiological monitoring, and longitudinal patient trajectories into comprehensive computational representations capable of supporting diagnosis, prognostic prediction, molecular characterization, therapeutic planning, toxicity assessment, clinical decision support, and survivorship management. Advances in transformer architectures, selfsupervised learning, contrastive learning, graph neural networks, multimodal large language models, retrieval-augmented generation, and generative AI have substantially accelerated development of these foundation models. These intelligent systems have demonstrated remarkable performance across radiology, computational pathology, radiogenomics, biomarker discovery, immunotherapy prediction, digital twins, clinical trial matching, and precision therapeutics. Nevertheless, important challenges remain regarding multimodal data harmonization, computational scalability, explainability, interoperability, regulatory validation, cybersecurity, ethical governance, and equitable implementation. This review provides a comprehensive overview of multimodal foundation models in cancer medicine, highlighting computational principles, current clinical applications, technological innovations, and future perspectives for next-generation precision oncology.[1]