Table 1 Summary of deep learning methods, their relevant applications and brief technical descriptions of each DL model

Microscopy-based assessment of cancer

CNN

Ruy et al. [40]

Nir et al. [41]

Ström et al. [42]

Ehteshami Bejnordi et al. [43]

Vuong et al. [44]

El Achi and Khoury [45]

Trained CNNs on pathology images to predict grading of prostate [40,41,42], breast [43], colon cancer [44] and lymphoma [45]

CNN & explainability

Hägele et al. [46]

LRP used to assigned feature contribution for cancer grade for each pixel of WSIs

Semantic segmentation

Poojitha and Lal Sharma [47]

A semantic segmentation technique called GAN was used to segment tissue maps for prostate cancer grade prediction

Molecular subtyping

MLP

DeepCC [48]

Gene set enrichment analysis used to transform gene expression input into functional spectra

CNN

imCMS [49],

Sirinukuwattana et al. [50], Stalhammar et al. [51], Couture et al. [52]

Woerl et al. [53]

Models trained on histopathology images to classify molecular subtypes of of lung [49], colorectal [50], breast [51, 52] and bladder cancer [53]

GCNN

Rhee et al. [18]

Utilised a hybrid GCNN model to organise input gene expression profiles into STRING PPI network [16] and predict breast cancer molecular subtypes

Multimodal learning

Islam et al. [54]

Two CNN models used to predict breast cancer molecular subtypes from CNAs and gene expression;

Outputs of the last fully connected layer of each model concatenated for a final subtype prediction

Cancer of unknown primary

MLP

Jiao et al. [55]

Model trained to predict origins of 24 cancer types using somatic mutation patterns and driver genes

CNN

SCOPE [56],

CUP-AI-Dx [57]

Both studies trained models to predict different cancer types from gene expression

RNN & explainability

TOAD [58]

RNN-based model called Attention was trained on WSIs to predict metastasis and cancer origin;

Attention algorithm reveal image regions contributing most to predictions were mostly cancer cells

Prognosis prediction

MLP

Cox-nnet [59],

DeepSurv [60], RankedDeepSurv [61]

Cox regression used as the last layer of MLP models for prognosis prediction

MLP & AEs

AECOX [62]

AE used to “compress” gene expression into low-dimensional embedding vector and used as an input for Cox-regression

Explainability

PASNET [63],

Cox-PASNET [64]

A pathway layer used between the input and the hidden layers with each node representing a known pathway;

Analysis of weight differences in pathway layers reveal clinically actionable genetic traits

MesoNet [65]

Histopathology images split into tiles and scored by survival prediction contributions;

Scores used to identify top-contributing regions, reviewed by pathologists

GCNN & explainability

Chereda et al. [19]

Combine GCNN and explainability method LRP to identify biologically and therapeutically relevant genes in predicting metastasis of breast cancer

Explainability with multimodal learning

PAGE-Net [66]

CNN used to compress features from WSIs;

Cox-PASNet used to incorporate gene pathway and provide cross-modal analysis with image features extracted by CNN

PathME [67]

AEs used to compress features from four omics modalities, which are combined to predict survival;

SHAP used to assign each omics feature survival prediction contribution score

Precision Oncology

MLP

HER2RNA [68]

Transcriptomic profiles inferred from histopathology images divided into tiles;

Predictions added up for all tiles and compared with ‘ground truth’ transcriptomic profiles

CNN

Image2TMB [69]

Ensemble of three CNNs to extract features from histopathological images at different resolutions (x5, x10 and x20);

Extracted features are combined to infer TMB

Kather et al. [70]

TCGA histopathology images used to predict mutational status of key genes, molecular subtypes and gene expression of standard biomarkers

Tumour microenvironment

MLP

Scaden [71]

Ensemble of three models with different filter sizes to predict TME composition from gene expression;

Predictions from the models are averaged into a final prediction

Explainability with MLP

MethylNet [72]

MLP and AE used to ‘compress’ CpG beta values into an embedding vector for predicting TME composition;

SHAP used to assign feature contribution to each CpG site

Semantic segmentation

Saltz et al. [20]

Semantic segmentation model used on H&E images to localise spatial heterogeneity patterns of TIL and necrosis

Spatial transcriptomics

CNN

ST-Net [73]

Images split into tiles centred on spatial transcriptomics spots;

Tiles used to train a CNN to predict expression of 250 target genes

Pharmacogenomics

CNN

CDRscan [74]

Two models used to extract features from somatic mutational fingerprints and molecular profiles of drugs (cell lines);

Feature vectors combined to predict efficacy of drugs based on genomic profiles

MLP

DeepSynergy [75]

Cell line gene expression and chemical features of drugs in drug combinations used as input;

Predicts ‘synergy score’ between the drug combinations and transcriptomic profiles

GCNN

Jiang et al. [76]

Utilised graph structure to integrate protein-protein, drug-drug and drug-protein interactions to predict synergistic drug combination for specific cell lines

Multimodal learning

DeepDR [77]

Collection of ten AEs to integrate ten drug-disease networks, which predict drug-disease associations

CNN

DeepDTI [78]

Protein sequence and drug fingerprint as input to predict drug protein-binding sites