Feature distillation multiple instance learning method based on sequence reorganized Mamba_Journal of Biomedical Engineering

Authors：

ZENG Junying ¹ , LUO Weibin ² , ZHAO Jiaxi ² , HUANG Guolin ² , ZHAO Jianwen ¹ , MAI Zhipeng ³ , YAN Weigang ³ , XIAO Yu ⁴ ,  QIN Chuanbo ¹

1. School of Electronic and Information Engineering, Wuyi University, Jiangmen, Guangdong 529020, P. R. China;
2. School of Computer and Software, Shenzhen University, Shenzhen, Guangdong 518060, P. R. China;
3. Department of Urology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Beijing 100730, P. R. China;
4. Pathology Department, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Beijing 100730, P. R. China;

Corresponding?author：

QIN Chuanbo, Email: tenround@163.com

Keywords：

Histopathology; Multiple instance learning; Feature distillation; Slice classification

DOI：

10.7507/1001-5515.202501058

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Prostate cancer is one of the most prevalent malignancies among men worldwide, and its diagnosis relies heavily on accurate analysis of whole slide imaging (WSI) in histopathology. However, manual interpretation is time-consuming and prone to inconsistent accuracy. Existing Multiple Instance Learning (MIL)-based studies can assist diagnosis but still suffer from high computational cost, insufficient exploitation of inter-instance relationships, and neglect of tissue heterogeneity. To address these challenges, this paper proposes a Feature Distillation Multiple Instance Learning method based on Sequence Reorganization Mamba (FDMIL). The proposed approach leverages the long-sequence modeling capability of SR-Mamba to capture effective inter-instance dependencies and heterogeneity. Meanwhile, a feature distillation mechanism is introduced to remove redundant representations and reduce computational overhead. Additionally, an auxiliary loss function is designed to mitigate pseudo-bag noise interference. We evaluate FDMIL on the Peking Union Medical College Hospital (PUMCH) prostate cancer WSI dataset and the public Camelyon16 dataset. Experimental results demonstrate that FDMIL achieves significant performance improvements on both datasets, reaching an AUC of 93.9%, ACC of 90.1%, and F1-score of 87.3%, outperforming existing state-of-the-art methods. These results verify the effectiveness and clinical applicability of FDMIL in both institutional and public scenarios.

1.	He Y, Xu W, Xiao Y T, et al. Targeting signaling pathways in prostate cancer: mechanisms and clinical trials. Signal Transduct Target Ther, 2022, 7(1): 198.
2.	Bray F, Ferlay J, Soerjomataram I, et al. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin, 2018, 68(6): 394-424.
3.	James S L, Abate D, Abate K H, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet, 2018, 392(10159): 1789-1858.
4.	Foreman K J, Marquez N, Dolgert A, et al. Forecasting life expectancy, years of life lost, and all-cause and cause-specific mortality for 250 causes of death: reference and alternative scenarios for 2016–40 for 195 countries and territories. Lancet, 2018, 392(10159): 2052-2090.
5.	劉琨, 張明洋, 李浩然, 等. 人工智能技術在泌尿系統腫瘤診斷中的研究現狀及展望. 生物醫學工程學雜志, 2021, 38(6): 1219-1228.
6.	He L, Long L R, Antani S, et al. Histology image analysis for carcinoma detection and grading. Comput Methods Programs Biomed, 2012, 107(3): 538-556.
7.	Li Y, Ping W. Cancer metastasis detection with neural conditional random field. arXiv preprint, 2018: 1806.07064.
8.	Tellez D, Litjens G, Van der Laak J, et al. Neural image compression for gigapixel histopathology image analysis. IEEE Trans Pattern Anal Mach Intell, 2019, 43(2): 567-578.
9.	Wang D, Khosla A, Gargeya R, et al. Deep learning for identifying metastatic breast cancer. arXiv preprint, 2016: 1606.05718.
10.	Wang Y, Kartasalo K, Weitz P, et al. Predicting molecular phenotypes from histopathology images: a transcriptome-wide expression–morphology analysis in breast cancer. Cancer Res, 2021, 81(19): 5115-5126.
11.	Zhang H, Kalirai H, Acha-Sagredo A, et al. Piloting a deep learning model for predicting nuclear BAP1 immunohistochemical expression of uveal melanoma from hematoxylin-and-eosin sections. Transl Vis Sci Technol, 2020, 9(2): 50.
12.	Zhang H, Meng Y, Qian X, et al. A regularization term for slide correlation reduction in whole slide image analysis with deep learning// Medical Imaging with Deep Learning. Lübeck: PMLR, 2021: 842-854.
13.	Myronenko A, Xu Z, Yang D, et al. Accounting for dependencies in deep learning based multiple instance learning for whole slide imaging// Medical Image Computing and Computer-Assisted Intervention-MICCAI 2021: 24th International Conference. Strasbourg: Springer International Publishing, 2021: 329-338.
14.	Srinidhi C L, Ciga O, Martel A L. Deep neural network models for computational histopathology: A survey. Med Image Anal, 2021, 67: 101813.
15.	Wang X, Chen H, Gan C, et al. Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE Trans Cybern, 2019, 50(9): 3950-3962.
16.	Zhou Z H. A brief introduction to weakly supervised learning. Natl Sci Rev, 2018, 5(1): 44-53.
17.	Lu M Y, Chen R J, Wang J, et al. Semi-supervised histology classification using deep multiple instance learning and contrastive predictive coding. arXiv preprint, 2019: 1910.10825.
18.	Meng Y, Zhang H, Zhao Y, et al. Spatial uncertainty-aware semi-supervised crowd counting// Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE, 2021: 15549-15559.
19.	Amores J. Multiple instance classification: Review, taxonomy and comparative study. Artif Intell, 2013, 201: 81-105.
20.	Chen Z, Chi Z, Fu H, et al. Multi-instance multi-label image classification: A neural approach. Neurocomputing, 2013, 99: 298-306.
21.	Dietterich T G, Lathrop R H, Lozano-Pérez T. Solving the multiple instance problem with axis-parallel rectangles. Artif Intell, 1997, 89(1-2): 31-71.
22.	Maron O, Lozano-Pérez T. A framework for multiple-instance learning// Neural Information Processing Systems (NeurIPS). Denver: NIPS Foundation, 1997.
23.	Ilse M, Tomczak J, Welling M. Attention-based deep multiple instance learning// International Conference on Machine Learning (LCML). Stockholm: PMLR, 2018: 2127-2136.
24.	Lerousseau M, Vakalopoulou M, Classe M, et al. Weakly supervised multiple instance learning histopathological tumor segmentation// Medical Image Computing and Computer-Assisted Intervention-MICCAI 2020: 23rd International Conference. Lima: Springer International Publishing, 2020: 470-479.
25.	Zhang H, Meng Y, Zhao Y, et al. DTFD-MIL: Double-tier feature distillation multiple instance learning for histopathology whole slide image classification// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE, 2022: 18802-18812.
26.	Shao Z, Bian H, Chen Y, et al. Transmil: Transformer based correlated multiple instance learning for whole slide image classification. Adv Neural Inf Process Syst, 2021, 34: 2136-2147.
27.	Yang S, Wang Y, Chen H. Mambamil: Enhancing long sequence modeling with sequence reordering in computational pathology// Medical Image Computing and Computer-Assisted Intervention-MICCAI 2024: 24th International Conference. Marrakesh: Springer Nature Switzerland, 2024: 296-306.
28.	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
29.	Xu G, Song Z, Sun Z, et al. Camel: A weakly supervised learning framework for histopathology image segmentation// Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE, 2019: 10682-10691.
30.	Bejnordi B E, Veta M, Van Diest P J, et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA, 2017, 318(22): 2199-2210.

1. He Y, Xu W, Xiao Y T, et al. Targeting signaling pathways in prostate cancer: mechanisms and clinical trials. Signal Transduct Target Ther, 2022, 7(1): 198.
2. Bray F, Ferlay J, Soerjomataram I, et al. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin, 2018, 68(6): 394-424.
3. James S L, Abate D, Abate K H, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet, 2018, 392(10159): 1789-1858.
4. Foreman K J, Marquez N, Dolgert A, et al. Forecasting life expectancy, years of life lost, and all-cause and cause-specific mortality for 250 causes of death: reference and alternative scenarios for 2016–40 for 195 countries and territories. Lancet, 2018, 392(10159): 2052-2090.
5. 劉琨, 張明洋, 李浩然, 等. 人工智能技術在泌尿系統腫瘤診斷中的研究現狀及展望. 生物醫學工程學雜志, 2021, 38(6): 1219-1228.
6. He L, Long L R, Antani S, et al. Histology image analysis for carcinoma detection and grading. Comput Methods Programs Biomed, 2012, 107(3): 538-556.
7. Li Y, Ping W. Cancer metastasis detection with neural conditional random field. arXiv preprint, 2018: 1806.07064.
8. Tellez D, Litjens G, Van der Laak J, et al. Neural image compression for gigapixel histopathology image analysis. IEEE Trans Pattern Anal Mach Intell, 2019, 43(2): 567-578.
9. Wang D, Khosla A, Gargeya R, et al. Deep learning for identifying metastatic breast cancer. arXiv preprint, 2016: 1606.05718.
10. Wang Y, Kartasalo K, Weitz P, et al. Predicting molecular phenotypes from histopathology images: a transcriptome-wide expression–morphology analysis in breast cancer. Cancer Res, 2021, 81(19): 5115-5126.
11. Zhang H, Kalirai H, Acha-Sagredo A, et al. Piloting a deep learning model for predicting nuclear BAP1 immunohistochemical expression of uveal melanoma from hematoxylin-and-eosin sections. Transl Vis Sci Technol, 2020, 9(2): 50.
12. Zhang H, Meng Y, Qian X, et al. A regularization term for slide correlation reduction in whole slide image analysis with deep learning// Medical Imaging with Deep Learning. Lübeck: PMLR, 2021: 842-854.
13. Myronenko A, Xu Z, Yang D, et al. Accounting for dependencies in deep learning based multiple instance learning for whole slide imaging// Medical Image Computing and Computer-Assisted Intervention-MICCAI 2021: 24th International Conference. Strasbourg: Springer International Publishing, 2021: 329-338.
14. Srinidhi C L, Ciga O, Martel A L. Deep neural network models for computational histopathology: A survey. Med Image Anal, 2021, 67: 101813.
15. Wang X, Chen H, Gan C, et al. Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE Trans Cybern, 2019, 50(9): 3950-3962.
16. Zhou Z H. A brief introduction to weakly supervised learning. Natl Sci Rev, 2018, 5(1): 44-53.
17. Lu M Y, Chen R J, Wang J, et al. Semi-supervised histology classification using deep multiple instance learning and contrastive predictive coding. arXiv preprint, 2019: 1910.10825.
18. Meng Y, Zhang H, Zhao Y, et al. Spatial uncertainty-aware semi-supervised crowd counting// Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE, 2021: 15549-15559.
19. Amores J. Multiple instance classification: Review, taxonomy and comparative study. Artif Intell, 2013, 201: 81-105.
20. Chen Z, Chi Z, Fu H, et al. Multi-instance multi-label image classification: A neural approach. Neurocomputing, 2013, 99: 298-306.
21. Dietterich T G, Lathrop R H, Lozano-Pérez T. Solving the multiple instance problem with axis-parallel rectangles. Artif Intell, 1997, 89(1-2): 31-71.
22. Maron O, Lozano-Pérez T. A framework for multiple-instance learning// Neural Information Processing Systems (NeurIPS). Denver: NIPS Foundation, 1997.
23. Ilse M, Tomczak J, Welling M. Attention-based deep multiple instance learning// International Conference on Machine Learning (LCML). Stockholm: PMLR, 2018: 2127-2136.
24. Lerousseau M, Vakalopoulou M, Classe M, et al. Weakly supervised multiple instance learning histopathological tumor segmentation// Medical Image Computing and Computer-Assisted Intervention-MICCAI 2020: 23rd International Conference. Lima: Springer International Publishing, 2020: 470-479.
25. Zhang H, Meng Y, Zhao Y, et al. DTFD-MIL: Double-tier feature distillation multiple instance learning for histopathology whole slide image classification// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE, 2022: 18802-18812.
26. Shao Z, Bian H, Chen Y, et al. Transmil: Transformer based correlated multiple instance learning for whole slide image classification. Adv Neural Inf Process Syst, 2021, 34: 2136-2147.
27. Yang S, Wang Y, Chen H. Mambamil: Enhancing long sequence modeling with sequence reordering in computational pathology// Medical Image Computing and Computer-Assisted Intervention-MICCAI 2024: 24th International Conference. Marrakesh: Springer Nature Switzerland, 2024: 296-306.
28. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
29. Xu G, Song Z, Sun Z, et al. Camel: A weakly supervised learning framework for histopathology image segmentation// Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE, 2019: 10682-10691.
30. Bejnordi B E, Veta M, Van Diest P J, et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA, 2017, 318(22): 2199-2210.

Journal of Biomedical Engineering

Latest ArticlesFeature distillation multiple instance learning method based on sequence reorganized Mamba

Abstract Full text Figures/Tables Video References Cited by

Format

Content