Abstract

Background: Colorectal cancer (CRC) is a leading cause of mortality in China, with metastasis significantly contributing to poor outcomes. Histopathological growth patterns (HGPs) in colorectal liver metastasis (CRLM) provide vital prognostic insights, yet the limited number of pathologists highlights the need for auxiliary diagnostic tools. Recent advancements in artificial intelligence (AI) have demonstrated potential in enhancing diagnostic precision, prompting the development of specialized AI models like COFFEE to improve the classification and management of HGPs in CRLM patients.
Methods: This study developed a transformer-based deep learning model, COFFEE, for the precise classification of colorectal cancer subtypes using whole slide images (WSIs) from 514 patients diagnosed with colorectal cancer liver metastasis. The model was pre-trained using DINO on 1,442 WSIs from the TCGA-COAD cohort, utilizing a vision transformer (ViT) architecture to extract 384-dimensional feature vectors from 256 × 256 pixel patches. The proposed model integrates a transformer-based multiple instance learning (TransMIL) framework, which effectively aggregates spatial and morphological information through multi-head self-attention and pyramid position encoding generator (PPEG) modules. This design enables efficient handling of large instance sequences within WSIs, allowing for accurate binary and four-class classification. The model was validated on 972 WSIs from a recent dataset, demonstrating its robustness and clinical applicability.
Results: A total of 431 patients were included in three cohorts: training (n=297), testing (n=104), and prospective (n=30). Desmoplastic tumors were associated with longer overall survival (OS, 53.6 vs. 31.9 months, p=0.002) and progression-free survival (PFS, 25.2 vs. 10.7 months, p<0.001) compared to non-desmoplastic tumors. The COFFEE binary classification model achieved high predictive performance with AUC values of 0.961 in the training, 0.935 in the testing, and 1.000 in the prospective cohort. The four-class model also showed strong performance, with AUCs of 0.961 and 0.966 in the training and testing cohorts, and 0.985 in the prospective cohort. AI-assisted models helped junior pathologists achieve an accuracy of 94.7% (vs. 85.9%) and reduced diagnostic time by 36%, improving both accuracy and speed.
Conclusion: This study developed the first AI model for HGP classification in colorectal cancer liver metastasis, achieving high accuracy in both binary classification and four-class classification models. The model demonstrated potential for improving diagnostic precision and guiding post-surgery treatment strategies, with AI-assisted pathologists surpassing traditional methods in a prospective randomized trial.

Keywords

Colorectal liver metastasis (CRLM), histopathological growth patterns (HGPs), artificial intelligence (AI) in diagnosis, vision transformer (ViT), desmoplastic classification

TABLE 1. Baseline characteristics of training, testing, and prospective cohorts.

HER2: Human Epidermal growth factor receptor 2; CEA: Carcinoembryonic Antigen; CA199: Carbohydrate Antigen 19-9; CA125: Cancer Antigen 125; IQR: Interquartile Range.

* 0 (Negative): No membrane positivity, 0% proportion; interpreted as negative;

1+ (Weakly Positive): Weak membrane positivity, ≤10% proportion; interpreted as negative;

2+ (Equivocal): Moderate to strong membrane positivity, 10-50% or ≥50% proportion; interpreted as equivocal, FISH testing recommended;

3+ (Positive): Strong membrane positivity, ≥50% proportion; interpreted as positive.

** Eleven patients have double gene mutations.

TABLE 2. Pathological classifications in training testing and, prospective cohorts.

TABLE 3. Clinicopathological characteristics of the training cohort based on binary pathological classification.

HER2: Human Epidermal Growth Factor Receptor 2; CEA: Carcinoembryonic Antigen; CA199: Carbohydrate Antigen 19-9; CA125: Cancer Antigen 125; IQR: Interquartile Range; OS: overall survival; PFS: Progression-Free Survival.

* 0 (Negative): No membrane positivity, 0% proportion; interpreted as negative;

1+ (Weakly Positive): Weak membrane positivity, ≤10% proportion; interpreted as negative;

2+ (Equivocal): Moderate to strong membrane positivity, 10-50% or ≥50% proportion; interpreted as equivocal, FISH testing recommended;

3+ (Positive): Strong membrane positivity, ≥50% proportion; interpreted as positive.

** Five patients have double gene mutations.

TABLE 4. Clinicopathological characteristics of the training cohort based on four-class pathological classification.

HER2: Human Epidermal Growth Factor Receptor 2; CEA: Carcinoembryonic Antigen; CA199: Carbohydrate Antigen 19-9; CA125: Cancer Antigen 125; IQR: Interquartile Range; OS: Overall Survival; PFS: Progression-Free Survival.

* 0 (Negative): No membrane positivity, 0% proportion; interpreted as negative;

1+ (Weakly Positive): Weak membrane positivity, ≤10% proportion; interpreted as negative;

2+ (Equivocal): Moderate to strong membrane positivity, 10-50% or ≥50% proportion; interpreted as equivocal, FISH testing recommended;

3+ (Positive): Strong membrane positivity, ≥50% proportion; interpreted as positive.

** Five patients have double gene mutations.

REFERENCES

1.    Rebecca L Siegel, Angela N Giaquinto, Ahmedin Jemal “Cancer statistics, 2024.” CA Cancer J Clin, vol. 74, no, 1, pp. 12-49, 2024. Erratum in: CA Cancer J Clin, vol. 74, no. 2, pp. 203, 2024. View at: Publisher Site | PubMed

2.    Ranmali Ranasinghe, Michael Mathai, Anthony Zulli “A synopsis of modern - day colorectal cancer: Where we stand.” Biochim Biophys Acta Rev Cancer, vol. 1877, no. 2, pp. 188699, 2022. View at: Publisher Site | PubMed

3.    René Adam, Aimery De Gramont, Joan Figueras, et al. “The oncosurgery approach to managing liver metastases from colorectal cancer: a multidisciplinary international consensus.” Oncologist, vol. 17, no. 10, pp. 1225-1239, 2012. View at: Publisher Site | PubMed

4.    L Viganò, B Branciforte, V Laurenti, et al. “The Histopathological Growth Pattern of Colorectal Liver Metastases Impacts Local Recurrence Risk and the Adequate Width of the Surgical Margin.” Ann Surg Oncol, vol. 29, no. 9, pp. 5515-5524, 2022. View at: Publisher Site | PubMed

5.    Pieter-Jan van Dam, Sofie Daelemans, Elizabeth Ross, et al. “Histopathological growth patterns as a candidate biomarker for immunomodulatory therapy.” Semin Cancer Biol, vol. 52, no. Pt 2, pp. 86-93, 2018. View at: Publisher Site | PubMed

6.    Florian E Buisman, Eric P van der Stok, Boris Galjart, et al. “Histopathological growth patterns as biomarker for adjuvant systemic chemotherapy in patients with resected colorectal liver metastases.” Clin Exp Metastasis, vol. 37, no. 5, pp. 593-605, 2020. View at: Publisher Site | PubMed

7.    Boris Galjart, Pieter M H Nierop, Eric P van der Stok, et al. “Angiogenic desmoplastic histopathological growth pattern as a prognostic marker of good outcome in patients with colorectal liver metastases.” Angiogenesis, vol. 22, no. 2, pp. 355-368, 2019. View at: Publisher Site | PubMed

8.    Wasswa William, Andrew Ware, Annabella Habinka Basaza-Ejiri, et al. “A pap-smear analysis tool (PAT) for detection of cervical cancer from pap-smear images.” Biomed Eng Online, vol. 18, no. 1, pp. 16, 2019. View at: Publisher Site | PubMed

9.    Ding-Qiao Wang, Long-Yu Feng, Jin-Guo Ye, et al. “Accelerating the integration of ChatGPT and other large-scale AI models into biomedical research and healthcare.” MedComm - Future Medicine, vol. 2, no. 2, pp. e43, 2023. View at: Publisher Site

10. Dimitris Bertsimas, Georgios Antonios Margonis, Suleeporn Sujichantararat, et al. “Using Artificial Intelligence to Find the Optimal Margin Width in Hepatectomy for Colorectal Cancer Liver Metastases.” JAMA Surg, vol. 157, no. 8, pp. e221819, 2022. View at: Publisher Site | PubMed

11. R Ferrari, C Mancini-Terracciano, C Voena, et al. “MR-based artificial intelligence model to assess response to therapy in locally advanced rectal cancer.” Eur J Radiol, vol. 118, pp. 1-9, 2019. View at: Publisher Site | PubMed

12. Ming Y Lu, Bowen Chen, Drew F K Williamson, et al. “A visual-language foundation model for computational pathology.” Nat Med, vol. 30, no. 3, pp. 863-874, 2024. View at: Publisher Site | PubMed

13. Ginimol Mathew, Riaz Agha, Joerg Albrecht, et al. “STROCSS 2021: Strengthening the Reporting of cohort, cross-sectional and case-control studies in Surgery.” Int J Surg, vol. 96, pp. 106165, 2021. View at: Publisher Site | PubMed

14. Ming Y Lu, Drew F K Williamson, Tiffany Y Chen, et al. “Data-efficient and weakly supervised computational pathology on whole-slide images.” Nat Biomed Eng, vol. 5, no. 6, pp. 555-570, 2021. View at: Publisher Site | PubMed

15. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, et al. “An image is worth 16x16 words: Transformers for image recognition at scale.” Computer Vision and Pattern Recognition, 2020.

16. Julien Calderaro, Narmin Ghaffari Laleh, Qinghe Zeng, et al. “Deep learning-based phenotyping reclassifies combined hepatocellular-cholangiocarcinoma.” Nat Commun, vol. 14, no. 1, pp. 8290, 2023. View at: Publisher Site | PubMed

17. Liu W, Lin Y, Liu Y, et al. “Vision Transformer for Small-Size Datasets.” Neurocomputing, vol. 473, pp. 144-155, 2022.

18. Yunfang Yu, Wenhao Ouyang, Yunxi Huang, et al. “AI-Based multimodal Multi-tasks analysis reveals tumor molecular heterogeneity, predicts preoperative lymph node metastasis and prognosis in papillary thyroid carcinoma: A retrospective study.” Int J Surg, 2024. View at: Publisher Site | PubMed

19. Mathilde Caron, Hugo Touvron, Ishan Misra, et al. “Emerging Properties in Self-Supervised Vision Transformers.” Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9650-9660, 2021.

20. Zhuchen Shao, Hao Bian, Yang Chen, et al. “TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification.” 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

21. Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, et al. “Nyströmformer: A nyström-based algorithm for approximating self-attention.” Proc AAAI Conf Artif Intell, vol. 35, no. 16, pp. 14138-14148, 2021. View at: PubMed

22. Jie-Ying Liang, Shao-Yan Xi, Qiong Shao, et al. “Histopathological growth patterns correlate with the immunoscore in colorectal cancer liver metastasis patients after hepatectomy.” Cancer Immunol Immunother, vol. 69, no. 12, pp. 2623-2634, 2020. View at: Publisher Site | PubMed

23. Diederik J Höppener, Pieter M H Nierop, Joost Hof, et al. “Enrichment of the tumour immune microenvironment in patients with desmoplastic colorectal liver metastasis.” Br J Cancer, vol. 123, no.2, pp. 196-206, 2020. View at: Publisher Site | PubMed

24. Boris Galjart, Pieter M H Nierop, Eric P van der Stok, et al. “Angiogenic desmoplastic histopathological growth pattern as a prognostic marker of good outcome in patients with colorectal liver metastases.” Angiogenesis, vol. 22, no. 2, pp. 355-368, 2019. View at: Publisher Site | PubMed

25. Pulathis N Siriwardana, Tu Vinh Luong, Jennifer Watkins, et al. “Biological and Prognostic Significance of the Morphological Types and Vascular Patterns in Colorectal Liver Metastases (CRLM): Looking Beyond the Tumor Margin.” Medicine (Baltimore), vol. 95, no. 8, pp. e2924, 2016. View at: Publisher Site | PubMed

26. Kåre Nielsen, Hans C Rolff, Rikke L Eefsen, et al. “The morphological growth patterns of colorectal liver metastases are prognostic for overall survival.” Mod Pathol, vol. 27, no. 12, pp. 1641-1648, 2014. View at: Publisher Site | PubMed

27. Vitoria Ramos Jayme, Gilton Marques Fonseca, Isaac Massaud Amim Amaral, et al. “Infiltrative Tumor Borders in Colorectal Liver Metastasis: Should We Enlarge Margin Size?” Ann Surg Oncol, vol. 28, no. 12, pp. 7636-7646, 2021. View at: Publisher Site | PubMed

28. Pieter M H Nierop, Boris Galjart, Diederik J Höppener, et al. “Salvage treatment for recurrences after first resection of colorectal liver metastases: the impact of histopathological growth patterns.” Clin Exp Metastasis, vol. 36, no. 2, pp. 109-118, 2019. View at: Publisher Site | PubMed

29. Hanna Nyström, Peter Naredi, Anette Berglund, et al. “Liver-metastatic potential of colorectal cancer is related to the stromal composition of the tumour.” Anticancer Res, vol. 32, no. 12, pp. 5183-5191, 2012. View at: PubMed

30. Chiara Cremolini, Massimo Milione, Federica Marmorino, et al. “Differential histopathologic parameters in colorectal cancer liver metastases resected after triplets plus bevacizumab or cetuximab: a pooled analysis of five prospective trials.” Br J Cancer, vol. 118, no. 7, pp. 955-965, 2018. View at: Publisher Site | PubMed

31. Zhaoyang Xu, Carlos Fernádez Moro, Danyil Kuznyecov, et al. “Tissue region growing for hispathology image segmentation.” In: Proceedings of the 2018 3rd International Conference on Biomedical Imaging, Signal Processing. Bari, Italy: Association for Computing Machinery, 2018.

32. David Tellez, Diederik Höppener, Cornelis Verhoef, et al. “Extending unsupervised neural image compression with supervised multitask learning.” Proc Mach Learn Res, vol. 121, pp. 770-783, 2020.

33. David Tellez, Geert Litjens, Jeroen van der Laak, et al. “Neural image compression for gigapixel histopathology image analysis.” IEEE Trans Pattern Anal Mach Intell, vol. 43,no. 2, pp. 567-578, 2021. View at: Publisher Site | PubMed

34. Chiara Cremolini, Massimo Milione, Federica Marmorino, et al. “Differential histopathologic parameters in colorectal cancer liver metastases resected after triplets plus bevacizumab or cetuximab: a pooled analysis of five prospective trials.” Br J Cancer, vol. 118, no. 7, pp. 955-965, 2018. View at: Publisher Site | PubMed

35. Artem Shmatko, Narmin Ghaffari Laleh, Moritz Gerstung “Artificial intelligence in histopathology: enhancing cancer research and clinical oncology.” Nat Cancer, vol. 3, no. 9, pp. 1026-1038, 2022. View at: Publisher Site | PubMed