Review of application of U-Net and Transformer in colon polyp image segmentation_Journal of Biomedical Engineering

Authors：

SHI Yankun , SUN Shilei , LIU Jing , MA Jingang ,  LI Ming

College of Intelligence and Information Engineering, Shandong University of Traditional Chinese Medicine, Jinan 250355, P. R. China;

Corresponding?author：

Keywords：

Colorectal polyp; Medical image segmentation; U-Net; Transformer

DOI：

10.7507/1001-5515.202405039

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Colorectal cancer typically originates from the malignant transformation of colonic polyps, making the automatic and accurate segmentation of colonic polyps crucial for clinical diagnosis. Deep learning techniques such as U-Net and Transformer can effectively extract implicit features from medical images, and thus have significant potential in colonic polyp image segmentation. This paper first introduced commonly used evaluation metrics and datasets for colonic polyp segmentation. It then reviewed the application of segmentation models based on U-Net, Transformer, and their hybrid approaches in this domain. Finally, it summarized the improvement methods, advantages, and limitations of polyp segmentation algorithms, discussed the challenges faced by U-Net- and Transformer-based models, and provided an outlook on future research directions in this field.

1.	Maida M, Dahiya D S, Shah Y R, et al. Screening and surveillance of colorectal cancer: A review of the literature. Cancers, 2024, 16(15): 2746.
2.	李國威, 劉靜, 曹慧, 等. 深度學習在結腸息肉圖像分割中的研究綜述. 計算機科學與探索, 2025, 19(5): 1198-1216.
3.	Grion B A R, Fonseca P L C, Kato R B, et al. Identification of taxonomic changes in the fecal bacteriome associated with colorectal polyps and cancer: potential biomarkers for early diagnosis. Front Microbiol, 2024, 14: 1292490.
4.	尹藝曉, 馬金剛, 張文凱, 等. 從U-Net到Transformer: 混合模型在醫學圖像分割中的應用進展. 激光與光電子學進展, 2025, 62(2): 11-33.
5.	考文濤, 李明, 馬金剛. 卷積神經網絡在結直腸息肉輔助診斷中的應用綜述. 計算機科學與探索, 2024, 18(3): 627-645.
6.	孫福艷, 王瓊, 呂宗旺, 等. 深度學習在結腸息肉分割中的應用綜述. 計算機工程與應用, 2023, 59(23): 15-27.
7.	Xiao B, Hu J, Li W, et al. CTNet: Contrastive transformer network for polyp segmentation. IEEE T Cybern, 2024, 54(9): 5040-5053.
8.	Bernal J, Sánchez F J, Fernández-Esparrach G, et al. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Comput Med Imag Grap, 2015, 43: 99-111.
9.	Tajbakhsh N, Gurudu S R, Liang J. Automated polyp detection in colonoscopy videos using shape and context information. IEEE T Med Imaging, 2015, 35(2): 630-644.
10.	Jha D, Smedsrud P H, Riegler M A, et al. Kvasir-SEG: A segmented polyp dataset// International Conference on Multimedia Modeling. Cham: Springer International Publishing, 2019: 451-462.
11.	Vázquez D, Bernal J, Sánchez F J, et al. A benchmark for endoluminal scene segmentation of colonoscopy images. J Healthc Eng, 2017, 2017(1): 4037190.
12.	Xu Y, Quan R, Xu W, et al. Advances in medical image segmentation: A comprehensive review of traditional, deep learning and hybrid approache. Bioengineering, 2024, 11(10): 1034.
13.	劉佳偉, 劉巧紅, 李曉歐, 等. 一種改進的雙U型網絡的結腸息肉分割方法. 光學學報, 2021, 41(18): 72-80.
14.	Yeung M, Sala E, Sch?nlieb C B, et al. Focus U-Net: A novel dual attention-gated CNN for polyp segmentation during colonoscopy. Comput Biol Med, 2021, 137: 104815.
15.	Tomar N K, Jha D, Bagci U, et al. TGANet: Text-guided attention for improved polyp segmentation// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2022: 151-160.
16.	Wang H, Cao P, Wang J, et al. UCTransNet: rethinking the skip connections in U-Net from a channel-wise perspective with transformer// Proceedings of the AAAI Conference on Artificial Intelligence. Virtual Conference: AAAI, 2022: 2441-2449.
17.	Zhou Z, Rahman Siddiquee M M, Tajbakhsh N, et al. UNet++: A nested U-Net architecture for medical image segmentation// International Workshop on Deep Learning in Medical Image Analysis. Cham: Springer International Publishing, 2018: 3-11.
18.	Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4700-4708.
19.	Tomar N K, Jha D, Ali S, et al. DDANet: Dual decoder attention network for automatic polyp segmentation// International Conference on Pattern Recognition. Cham: Springer International Publishing, 2021: 307-314.
20.	Jha D, Smedsrud P H, Johansen D, et al. A comprehensive study on colorectal polyp segmentation with ResUNet++, conditional random field and test-time augmentation. IEEE J Biomed Health Inform, 2021, 25(6): 2029-2040.
21.	Ramzan M, Raza M, Sharif M I, et al. Gastrointestinal tract polyp anomaly segmentation on colonoscopy images using graft-U-Net. J Pers Med, 2022, 12(9): 1459.
22.	Punn N S, Agarwal S. Modality specific U-Net variants for biomedical image segmentation: a survey. Artif Intell Rev, 2022, 55(7): 5845-5889.
23.	Jha D, Tomar N K, Sharma V, et al. TransNetR: transformer-based residual network for polyp segmentation with multi-center out-of-distribution testing// Medical Imaging with Deep Learning. Virtual Conference: PMLR, 2024: 1372-1384.
24.	Guo Q, Fang X, Wang L, et al. Polyp segmentation of colonoscopy images by exploring the uncertain areas. IEEE Access, 2022, 10: 52971-52981.
25.	梁禮明, 何安軍, 朱晨錕, 等. 融合Transformer和跨級相位感知的結腸息肉分割方法. 生物醫學工程學雜志, 2023, 40(2): 234-243.
26.	Liu X, Song S. Attention combined pyramid vision transformer for polyp segmentation. Biomed Signal Proces, 2024, 89: 105792.
27.	Park K B, Lee J Y. SwinE-Net: Hybrid deep learning approach to novel polyp segmentation using convolutional neural network and Swin Transformer. J Comput Des Eng, 2022, 9(2): 616-632.
28.	梁禮明, 何安軍, 陽淵, 等. 基于Swin Transformer和圖形推理的結直腸息肉分割方法. 工程科學學報, 2024, 46(5): 897-907.
29.	Oukdach Y, Garbaz A, Kerkaou Z, et al. UViT-SEG: an efficient ViT and U-net-based framework for accurate colorectal polyp segmentation in colonoscopy and WCE images. J Imaging Inform Med, 2024, 37(5): 2354-2374.
30.	Pan S, Liu X, Xie N, et al. EG-TransUNet: a transformer-based U-Net with enhanced and guided models for biomedical image segmentation. BMC Bioinform, 2023, 24(1): 85.
31.	Duc N T, Oanh N T, Thuy N T, et al. Colonformer: An efficient transformer based method for colon polyp segmentation. IEEE Access, 2022, 10: 80575-80586.
32.	Zhang Y, Liu H, Hu Q. Transfuse: Fusing transformers and cnns for medical image segmentation// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer International Publishing, 2021: 14-24.
33.	Lin Y, Han X, Chen K, et al. CSwinDoubleU-Net: A double U-shaped network combined with convolution and Swin Transformer for colorectal polyp segmentation. Biomed Signal Proces, 2024, 89: 105749.
34.	Mahmood T, Rehman A, Saba T, et al. Recent advancements and future prospects in active deep learning for medical image segmentation and classification. IEEE Access, 2023, 11: 113623-113652.
35.	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks. Commun Acm, 2020, 63(11): 139-144.

1. Maida M, Dahiya D S, Shah Y R, et al. Screening and surveillance of colorectal cancer: A review of the literature. Cancers, 2024, 16(15): 2746.
2. 李國威, 劉靜, 曹慧, 等. 深度學習在結腸息肉圖像分割中的研究綜述. 計算機科學與探索, 2025, 19(5): 1198-1216.
3. Grion B A R, Fonseca P L C, Kato R B, et al. Identification of taxonomic changes in the fecal bacteriome associated with colorectal polyps and cancer: potential biomarkers for early diagnosis. Front Microbiol, 2024, 14: 1292490.
4. 尹藝曉, 馬金剛, 張文凱, 等. 從U-Net到Transformer: 混合模型在醫學圖像分割中的應用進展. 激光與光電子學進展, 2025, 62(2): 11-33.
5. 考文濤, 李明, 馬金剛. 卷積神經網絡在結直腸息肉輔助診斷中的應用綜述. 計算機科學與探索, 2024, 18(3): 627-645.
6. 孫福艷, 王瓊, 呂宗旺, 等. 深度學習在結腸息肉分割中的應用綜述. 計算機工程與應用, 2023, 59(23): 15-27.
7. Xiao B, Hu J, Li W, et al. CTNet: Contrastive transformer network for polyp segmentation. IEEE T Cybern, 2024, 54(9): 5040-5053.
8. Bernal J, Sánchez F J, Fernández-Esparrach G, et al. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Comput Med Imag Grap, 2015, 43: 99-111.
9. Tajbakhsh N, Gurudu S R, Liang J. Automated polyp detection in colonoscopy videos using shape and context information. IEEE T Med Imaging, 2015, 35(2): 630-644.
10. Jha D, Smedsrud P H, Riegler M A, et al. Kvasir-SEG: A segmented polyp dataset// International Conference on Multimedia Modeling. Cham: Springer International Publishing, 2019: 451-462.
11. Vázquez D, Bernal J, Sánchez F J, et al. A benchmark for endoluminal scene segmentation of colonoscopy images. J Healthc Eng, 2017, 2017(1): 4037190.
12. Xu Y, Quan R, Xu W, et al. Advances in medical image segmentation: A comprehensive review of traditional, deep learning and hybrid approache. Bioengineering, 2024, 11(10): 1034.
13. 劉佳偉, 劉巧紅, 李曉歐, 等. 一種改進的雙U型網絡的結腸息肉分割方法. 光學學報, 2021, 41(18): 72-80.
14. Yeung M, Sala E, Sch?nlieb C B, et al. Focus U-Net: A novel dual attention-gated CNN for polyp segmentation during colonoscopy. Comput Biol Med, 2021, 137: 104815.
15. Tomar N K, Jha D, Bagci U, et al. TGANet: Text-guided attention for improved polyp segmentation// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2022: 151-160.
16. Wang H, Cao P, Wang J, et al. UCTransNet: rethinking the skip connections in U-Net from a channel-wise perspective with transformer// Proceedings of the AAAI Conference on Artificial Intelligence. Virtual Conference: AAAI, 2022: 2441-2449.
17. Zhou Z, Rahman Siddiquee M M, Tajbakhsh N, et al. UNet++: A nested U-Net architecture for medical image segmentation// International Workshop on Deep Learning in Medical Image Analysis. Cham: Springer International Publishing, 2018: 3-11.
18. Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4700-4708.
19. Tomar N K, Jha D, Ali S, et al. DDANet: Dual decoder attention network for automatic polyp segmentation// International Conference on Pattern Recognition. Cham: Springer International Publishing, 2021: 307-314.
20. Jha D, Smedsrud P H, Johansen D, et al. A comprehensive study on colorectal polyp segmentation with ResUNet++, conditional random field and test-time augmentation. IEEE J Biomed Health Inform, 2021, 25(6): 2029-2040.
21. Ramzan M, Raza M, Sharif M I, et al. Gastrointestinal tract polyp anomaly segmentation on colonoscopy images using graft-U-Net. J Pers Med, 2022, 12(9): 1459.
22. Punn N S, Agarwal S. Modality specific U-Net variants for biomedical image segmentation: a survey. Artif Intell Rev, 2022, 55(7): 5845-5889.
23. Jha D, Tomar N K, Sharma V, et al. TransNetR: transformer-based residual network for polyp segmentation with multi-center out-of-distribution testing// Medical Imaging with Deep Learning. Virtual Conference: PMLR, 2024: 1372-1384.
24. Guo Q, Fang X, Wang L, et al. Polyp segmentation of colonoscopy images by exploring the uncertain areas. IEEE Access, 2022, 10: 52971-52981.
25. 梁禮明, 何安軍, 朱晨錕, 等. 融合Transformer和跨級相位感知的結腸息肉分割方法. 生物醫學工程學雜志, 2023, 40(2): 234-243.
26. Liu X, Song S. Attention combined pyramid vision transformer for polyp segmentation. Biomed Signal Proces, 2024, 89: 105792.
27. Park K B, Lee J Y. SwinE-Net: Hybrid deep learning approach to novel polyp segmentation using convolutional neural network and Swin Transformer. J Comput Des Eng, 2022, 9(2): 616-632.
28. 梁禮明, 何安軍, 陽淵, 等. 基于Swin Transformer和圖形推理的結直腸息肉分割方法. 工程科學學報, 2024, 46(5): 897-907.
29. Oukdach Y, Garbaz A, Kerkaou Z, et al. UViT-SEG: an efficient ViT and U-net-based framework for accurate colorectal polyp segmentation in colonoscopy and WCE images. J Imaging Inform Med, 2024, 37(5): 2354-2374.
30. Pan S, Liu X, Xie N, et al. EG-TransUNet: a transformer-based U-Net with enhanced and guided models for biomedical image segmentation. BMC Bioinform, 2023, 24(1): 85.
31. Duc N T, Oanh N T, Thuy N T, et al. Colonformer: An efficient transformer based method for colon polyp segmentation. IEEE Access, 2022, 10: 80575-80586.
32. Zhang Y, Liu H, Hu Q. Transfuse: Fusing transformers and cnns for medical image segmentation// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer International Publishing, 2021: 14-24.
33. Lin Y, Han X, Chen K, et al. CSwinDoubleU-Net: A double U-shaped network combined with convolution and Swin Transformer for colorectal polyp segmentation. Biomed Signal Proces, 2024, 89: 105749.
34. Mahmood T, Rehman A, Saba T, et al. Recent advancements and future prospects in active deep learning for medical image segmentation and classification. IEEE Access, 2023, 11: 113623-113652.
35. Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks. Commun Acm, 2020, 63(11): 139-144.

Journal of Biomedical Engineering

Latest ArticlesReview of application of U-Net and Transformer in colon polyp image segmentation

Abstract Full text Figures/Tables Video References Cited by

Format

Content