Large Multimodal Models for Computational Oncology: Opportunities, Challenges, and Clinical Translation

Abstract

Cancer remains one of the leading causes of morbidity and mortality worldwide despite remarkable advances in molecular biology, precision medicine, targeted therapeutics, and immunotherapy. The rapid expansion of biomedical data generated from radiological imaging, digital pathology, genomic sequencing, transcriptomics, proteomics, metabolomics, electronic health records, laboratory investigations, wearable technologies, and longitudinal clinical documentation has created unprecedented opportunities for computational oncology while simultaneously presenting major analytical challenges. Large multimodal models (LMMs) have emerged as a transformative artificial intelligence (AI) paradigm capable of integrating heterogeneous biomedical information into unified patient-centered representations that support diagnosis, prognostic prediction, molecular characterization, therapeutic optimization, and clinical decision-making. Unlike conventional AI systems designed for single-modality analysis, LMMs combine imaging, pathology, multi-omics, clinical narratives, physiological monitoring, and real-world healthcare data through transformer architectures, multimodal representation learning, self-supervised learning, graph neural networks, retrieval-augmented generation, and foundation model pretraining. These intelligent systems have demonstrated remarkable capabilities across computational pathology, radiogenomics, precision therapeutics, digital twins, clinical trial matching, drug discovery, and survivorship care. Despite substantial advances, significant challenges remain regarding multimodal data harmonization, computational scalability, explainability, interoperability, regulatory validation, cybersecurity, algorithmic fairness, and ethical governance. This review provides a comprehensive overview of large multimodal models in computational oncology, emphasizing computational principles, clinical opportunities, implementation challenges, and future perspectives for translation into routine cancer care.