Research on surgical instrument segmentation algorithm based on frequency-domain adaptive feature decomposition and visual Mamba_Journal of Biomedical Engineering

Authors：

TAN Zili ^1,2 , HE Wenxuan ³ , CAO Yujie ^1,2 ,  CHI Ronghua ^1,2

1. School of Electronics and Information Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, P. R. China;
2. Jiangsu Province Engineering Research Center of Photonic Devices and System Integration for Communication Sensing Convergence, Wuxi University, Wuxi, Jiangsu 214105, P. R. China;
3. School of Medical Imaging, Xuzhou Medical University, Xuzhou, Jiangsu 221004, P. R. China;

Corresponding?author：

CHI Ronghua, Email: ronghchi@cwxu.edu.cn

Keywords：

Surgical instrument segmentation; U-shaped network; Visual Mamba; Fourier transform

DOI：

10.7507/1001-5515.202510015

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

For the endoscopic surgical instrument segmentation task, existing methods have failed to address the semantic gap caused by the mismatch between high-frequency spatial details and low-frequency semantic features in the U-shaped network (U-Net) architecture. This study proposes a U-Net algorithm based on frequency-domain adaptive feature decomposition and visual Mamba, namely frequency-domain decoupling Mamba U-Net (FDMUNet), for surgical instrument segmentation. This algorithm embeds a frequency-domain adaptive feature enhancement module into the skip connections, and decomposes features into high-frequency and low-frequency components through the Fourier transform and a learnable filter, followed by channel weighting and fusion, so as to enhance surgical instrument edge information and bridge the semantic gap between encoder and decoder features. On the Endoscopic Vision Challenge 2017 public dataset, FDMUNet achieved Intersection over Union, mean Intersection over Union, and mean class Intersection over Union scores of 70.79%, 74.25%, and 69.50%, respectively. In addition, the ablation experiment further verified the effectiveness of the proposed module. This method not only provides a new solution for instrument segmentation in complex scenes, but also provides a new research idea for the application of frequency-domain information in medical image segmentation.

Citation： TAN Zili, HE Wenxuan, CAO Yujie, CHI Ronghua. Research on surgical instrument segmentation algorithm based on frequency-domain adaptive feature decomposition and visual Mamba. Journal of Biomedical Engineering, 2026, 43(3): 554-561, 570. doi: 10.7507/1001-5515.202510015 Copy

1.	王勇, 湯紹濤. 達芬奇手術機器人輔助胸腔鏡手術治療小兒縱膈腫瘤1例. 臨床小兒外科雜志, 2017, 16(5): 518-520..
2.	Rivero-Moreno Y, Echevarria S, Vidal-Valderrama C, et al. Robotic surgery: a comprehensive review of the literature and current trends. Cureus, 2023, 15(7): e42370..
3.	Ahmed F A, Yousef M, Ahmed M A, et al. Deep learning for surgical instrument recognition and segmentation in robotic-assisted surgeries: a systematic review. Artif Intell Rev, 2024, 58(1): 1..
4.	Zhou Z, Alabi O, Wei M, et al. Text promptable surgical instrument segmentation with vision-language models//37th Conference on Neural Information Processing Systems (NIPS 2023), New Orleans: NIPS, 2023: 28611-28623..
5.	Sabri S A, York P J. Preoperative planning for intraoperative navigation guidance. Ann Transl Med, 2021, 9(1): 87..
6.	Consorti G, Monarchi G, Catarzi L. Presurgical virtual planning and intraoperative navigation with 3D-preformed mesh: a new protocol for primary orbital fracture reconstruction. Life, 2024, 14(4): 482..
7.	Teatini A, Pelanis E, Aghayan D L, et al. The effect of intraoperative imaging on surgical navigation for laparoscopic liver resection surgery. Sci Rep, 2019, 9(1): 18687..
8.	Hodel S, Uehara M. Navigation surgery in musculoskeletal disorders. BMC Musculoskelet Disord, 2024, 25(1): 814..
9.	Yanik E, Kruger U, Intes X, et al. Video-based formative and summative assessment of surgical tasks using deep learning. Sci Rep, 2023, 13(1): 1038..
10.	Berg M, Dahlin L B, Kjellman M. Overview of surgical training and assessment of surgical skills: a narrative review. Front Surg, 2025, 12: 1605495..
11.	Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks//26th Conference on Neural Information Processing Systems (NIPS 2012), Lake Tahoe: NIPS, 2012: 1097-1105..
12.	Abbas A, Abdelsamea M M, Gaber M M. Classification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network. Appl Intell, 2021, 51(2): 854-864..
13.	Zhu X, Zhang H, Hu B, et al. DSP-Net: a dynamic spectral-spatial joint perception network for hyperspectral target tracking. IEEE Geosci Remote Sens Lett, 2023, 20: 1-5..
14.	Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015, Munich: Springer International Publishing, 2015: 234-241..
15.	Zhao D, Zhang H, Huang K, et al. SASU-Net: hyperspectral video tracker based on spectral adaptive aggregation weighting and scale updating. Expert Syst Appl, 2025, 272: 126721..
16.	Hasan S M K, Linte C A. U-NetPlus: a modified encoder-decoder U-Net architecture for semantic and instance segmentation of surgical instruments from laparoscopic images//2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin: IEEE, 2019: 7205-7211..
17.	Khouy M, Jabrane Y, Ameur M, et al. Medical image segmentation using automatic optimized U-Net architecture based on genetic algorithm. J Pers Med, 2023, 13(9): 1298..
18.	Yuan Y, Cheng Y. Medical image segmentation with UNet-based multi-scale context fusion. Sci Rep, 2024, 14(1): 15687..
19.	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: transformers for image recognition at scale. arXiv, 2020: 2010.11929..
20.	Cao H, Wang Y, Chen J, et al. Swin-unet: Unet-like pure transformer for medical image segmentation//European Conference on Computer Vision (ECCV), Tel Aviv: Springer Nature Switzerland, 2023: 205-218..
21.	Tang F, Xu Z, Huang Q, et al. DuAT: dual-aggregation transformer network for medical image segmentation//Chinese Conference on Pattern Recognition and Computer Vision (PRCV), Xiamen: Springer Nature Singapore, 2023: 343-356..
22.	黃鯤鵬, 魏雨晴, 虞潔, 等. 基于Transformer的尺度自適應高光譜目標跟蹤器. 電光與控制, 2024, 31(12): 78-83..
23.	Gu A, Goel K, Ré C. Efficiently modeling long sequences with structured state spaces. arXiv, 2021: 2111.00396..
24.	Lu C, Schroecker Y, Gu A, et al. Structured state space models for in-context reinforcement learning//37th Conference on Neural Information Processing Systems (NIPS 2023), New Orleans: NIPS, 2023: 47016-47031..
25.	Somvanshi S, Islam M M, Mimi M S, et al. From s4 to Mamba: a comprehensive survey on structured state space models. arXiv, 2025: 2503.18970..
26.	Gu A, Dao T. Mamba: linear-time sequence modeling with selective state spaces. arXiv, 2023: 2312.00752..
27.	Liu Y, Tian Y, Zhao Y, et al. Vmamba: visual state space model//38th Conference on Neural Information Processing Systems (NIPS 2024), Vancouver: NIPS, 2024: 103031-103063..
28.	Ruan J, Li J, Xiang S. VM-UNet: vision Mamba UNet for medical image segmentation. arXiv, 2024: 2402.02491..
29.	Li M, Yan Z, Yan M, et al. PFESA: FFT-based parameter-free edge and structure attention for medical image segmentation//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2025, Daejeon: Springer Nature Switzerland, 2025: 470-480..
30.	Bouraya S, Belangour A. A comparative analysis of activation functions in neural networks: unveiling categories. Bull Electr Eng Inform, 2024, 13(5): 3301-3308..
31.	Mulindwa D B, Du S. An n-sigmoid activation function to improve the squeeze-and-excitation for 2D and 3D deep networks. Electronics, 2023, 12(4): 911..
32.	Yeung M, Sala E, Sch?nlieb C B, et al. Unified focal loss: generalising dice and cross entropy-based losses to handle class imbalanced medical image segmentation. Comput Med Imaging Graph, 2022, 95: 102026..
33.	武星, 陶晨杰, 李智, 等. 基于通道權重和數據效用特征的醫學圖像分割數據增強方法. 生物醫學工程學雜志, 2024, 41(2): 220-227,236..
34.	Mu Y, Nguyen T, Hawickhorst B, et al. The combined focal loss and dice loss function improves the segmentation of beta-sheets in medium-resolution cryo-electron-microscopy density maps. Bioinform Adv, 2024, 4(1): vbae169..
35.	Allan M, Shvets A, Kurmann T, et al. 2017 robotic instrument segmentation challenge. arXiv, 2019: 1902.06426..
36.	Chen L C, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation. arXiv, 2017: 1706.05587..
37.	González C, Bravo-Sánchez L, Arbelaez P. ISINet: an instance-based approach for surgical instrument segmentation//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2020, Lima: Springer International Publishing, 2020: 595-605..
38.	Wang J, Sun K, Cheng T, et al. Deep high-resolution representation learning for visual recognition. IEEE Trans Pattern Anal Mach Intell, 2020, 43(10): 3349-3364..
39.	Zhao Z, Jin Y, Heng P A. TraSeTR: track-to-segment transformer with contrastive query for instance-level instrument segmentation in robotic surgery//2022 International Conference on Robotics and Automation (ICRA), Philadelphia: IEEE, 2022: 11186-11193..
40.	Ayobi N, Pérez-Rondón A, Rodríguez S, et al. MATIS: masked-attention transformers for surgical instrument segmentation//2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), Cartagena: IEEE, 2023: 1-5..
41.	Yue W, Zhang J, Hu K, et al. SurgicalSAM: efficient class promptable surgical instrument segmentation//Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver: AAAI, 2024: 6890-6898..

1. 王勇, 湯紹濤. 達芬奇手術機器人輔助胸腔鏡手術治療小兒縱膈腫瘤1例. 臨床小兒外科雜志, 2017, 16(5): 518-520..
2. Rivero-Moreno Y, Echevarria S, Vidal-Valderrama C, et al. Robotic surgery: a comprehensive review of the literature and current trends. Cureus, 2023, 15(7): e42370..
3. Ahmed F A, Yousef M, Ahmed M A, et al. Deep learning for surgical instrument recognition and segmentation in robotic-assisted surgeries: a systematic review. Artif Intell Rev, 2024, 58(1): 1..
4. Zhou Z, Alabi O, Wei M, et al. Text promptable surgical instrument segmentation with vision-language models//37th Conference on Neural Information Processing Systems (NIPS 2023), New Orleans: NIPS, 2023: 28611-28623..
5. Sabri S A, York P J. Preoperative planning for intraoperative navigation guidance. Ann Transl Med, 2021, 9(1): 87..
6. Consorti G, Monarchi G, Catarzi L. Presurgical virtual planning and intraoperative navigation with 3D-preformed mesh: a new protocol for primary orbital fracture reconstruction. Life, 2024, 14(4): 482..
7. Teatini A, Pelanis E, Aghayan D L, et al. The effect of intraoperative imaging on surgical navigation for laparoscopic liver resection surgery. Sci Rep, 2019, 9(1): 18687..
8. Hodel S, Uehara M. Navigation surgery in musculoskeletal disorders. BMC Musculoskelet Disord, 2024, 25(1): 814..
9. Yanik E, Kruger U, Intes X, et al. Video-based formative and summative assessment of surgical tasks using deep learning. Sci Rep, 2023, 13(1): 1038..
10. Berg M, Dahlin L B, Kjellman M. Overview of surgical training and assessment of surgical skills: a narrative review. Front Surg, 2025, 12: 1605495..
11. Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks//26th Conference on Neural Information Processing Systems (NIPS 2012), Lake Tahoe: NIPS, 2012: 1097-1105..
12. Abbas A, Abdelsamea M M, Gaber M M. Classification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network. Appl Intell, 2021, 51(2): 854-864..
13. Zhu X, Zhang H, Hu B, et al. DSP-Net: a dynamic spectral-spatial joint perception network for hyperspectral target tracking. IEEE Geosci Remote Sens Lett, 2023, 20: 1-5..
14. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015, Munich: Springer International Publishing, 2015: 234-241..
15. Zhao D, Zhang H, Huang K, et al. SASU-Net: hyperspectral video tracker based on spectral adaptive aggregation weighting and scale updating. Expert Syst Appl, 2025, 272: 126721..
16. Hasan S M K, Linte C A. U-NetPlus: a modified encoder-decoder U-Net architecture for semantic and instance segmentation of surgical instruments from laparoscopic images//2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin: IEEE, 2019: 7205-7211..
17. Khouy M, Jabrane Y, Ameur M, et al. Medical image segmentation using automatic optimized U-Net architecture based on genetic algorithm. J Pers Med, 2023, 13(9): 1298..
18. Yuan Y, Cheng Y. Medical image segmentation with UNet-based multi-scale context fusion. Sci Rep, 2024, 14(1): 15687..
19. Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: transformers for image recognition at scale. arXiv, 2020: 2010.11929..
20. Cao H, Wang Y, Chen J, et al. Swin-unet: Unet-like pure transformer for medical image segmentation//European Conference on Computer Vision (ECCV), Tel Aviv: Springer Nature Switzerland, 2023: 205-218..
21. Tang F, Xu Z, Huang Q, et al. DuAT: dual-aggregation transformer network for medical image segmentation//Chinese Conference on Pattern Recognition and Computer Vision (PRCV), Xiamen: Springer Nature Singapore, 2023: 343-356..
22. 黃鯤鵬, 魏雨晴, 虞潔, 等. 基于Transformer的尺度自適應高光譜目標跟蹤器. 電光與控制, 2024, 31(12): 78-83..
23. Gu A, Goel K, Ré C. Efficiently modeling long sequences with structured state spaces. arXiv, 2021: 2111.00396..
24. Lu C, Schroecker Y, Gu A, et al. Structured state space models for in-context reinforcement learning//37th Conference on Neural Information Processing Systems (NIPS 2023), New Orleans: NIPS, 2023: 47016-47031..
25. Somvanshi S, Islam M M, Mimi M S, et al. From s4 to Mamba: a comprehensive survey on structured state space models. arXiv, 2025: 2503.18970..
26. Gu A, Dao T. Mamba: linear-time sequence modeling with selective state spaces. arXiv, 2023: 2312.00752..
27. Liu Y, Tian Y, Zhao Y, et al. Vmamba: visual state space model//38th Conference on Neural Information Processing Systems (NIPS 2024), Vancouver: NIPS, 2024: 103031-103063..
28. Ruan J, Li J, Xiang S. VM-UNet: vision Mamba UNet for medical image segmentation. arXiv, 2024: 2402.02491..
29. Li M, Yan Z, Yan M, et al. PFESA: FFT-based parameter-free edge and structure attention for medical image segmentation//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2025, Daejeon: Springer Nature Switzerland, 2025: 470-480..
30. Bouraya S, Belangour A. A comparative analysis of activation functions in neural networks: unveiling categories. Bull Electr Eng Inform, 2024, 13(5): 3301-3308..
31. Mulindwa D B, Du S. An n-sigmoid activation function to improve the squeeze-and-excitation for 2D and 3D deep networks. Electronics, 2023, 12(4): 911..
32. Yeung M, Sala E, Sch?nlieb C B, et al. Unified focal loss: generalising dice and cross entropy-based losses to handle class imbalanced medical image segmentation. Comput Med Imaging Graph, 2022, 95: 102026..
33. 武星, 陶晨杰, 李智, 等. 基于通道權重和數據效用特征的醫學圖像分割數據增強方法. 生物醫學工程學雜志, 2024, 41(2): 220-227,236..
34. Mu Y, Nguyen T, Hawickhorst B, et al. The combined focal loss and dice loss function improves the segmentation of beta-sheets in medium-resolution cryo-electron-microscopy density maps. Bioinform Adv, 2024, 4(1): vbae169..
35. Allan M, Shvets A, Kurmann T, et al. 2017 robotic instrument segmentation challenge. arXiv, 2019: 1902.06426..
36. Chen L C, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation. arXiv, 2017: 1706.05587..
37. González C, Bravo-Sánchez L, Arbelaez P. ISINet: an instance-based approach for surgical instrument segmentation//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2020, Lima: Springer International Publishing, 2020: 595-605..
38. Wang J, Sun K, Cheng T, et al. Deep high-resolution representation learning for visual recognition. IEEE Trans Pattern Anal Mach Intell, 2020, 43(10): 3349-3364..
39. Zhao Z, Jin Y, Heng P A. TraSeTR: track-to-segment transformer with contrastive query for instance-level instrument segmentation in robotic surgery//2022 International Conference on Robotics and Automation (ICRA), Philadelphia: IEEE, 2022: 11186-11193..
40. Ayobi N, Pérez-Rondón A, Rodríguez S, et al. MATIS: masked-attention transformers for surgical instrument segmentation//2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), Cartagena: IEEE, 2023: 1-5..
41. Yue W, Zhang J, Hu K, et al. SurgicalSAM: efficient class promptable surgical instrument segmentation//Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver: AAAI, 2024: 6890-6898..

Journal of Biomedical Engineering

Research on surgical instrument segmentation algorithm based on frequency-domain adaptive feature decomposition and visual Mamba

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content