The risk prediction of paroxysmal atrial fibrillation (PAF) is a challenge in the field of biomedical engineering. This study integrated the advantages of machine learning feature engineering and end-to-end modeling of deep learning to propose a PAF risk prediction method based on multimodal feature fusion. Additionally, the study utilized four different feature selection methods and Pearson correlation analysis to determine the optimal multimodal feature set, and employed random forest for PAF risk assessment. The proposed method achieved accuracy of (92.3 ± 2.1)% and F1 score of (91.6 ± 2.9)% in a public dataset. In a clinical dataset, it achieved accuracy of (91.4 ± 2.0)% and F1 score of (90.8 ± 2.4)%. The method demonstrates generalization across multi-center datasets and holds promising clinical application prospects.
Currently, the development of deep learning-based multimodal learning is advancing rapidly, and is widely used in the field of artificial intelligence-generated content, such as image-text conversion and image-text generation. Electronic health records are digital information such as numbers, charts, and texts generated by medical staff using information systems in the process of medical activities. The multimodal fusion method of electronic health records based on deep learning can assist medical staff in the medical field to comprehensively analyze a large number of medical multimodal data generated in the process of diagnosis and treatment, thereby achieving accurate diagnosis and timely intervention for patients. In this article, we firstly introduce the methods and development trends of deep learning-based multimodal data fusion. Secondly, we summarize and compare the fusion of structured electronic medical records with other medical data such as images and texts, focusing on the clinical application types, sample sizes, and the fusion methods involved in the research. Through the analysis and summary of the literature, the deep learning methods for fusion of different medical modal data are as follows: first, selecting the appropriate pre-trained model according to the data modality for feature representation and post-fusion, and secondly, fusing based on the attention mechanism. Lastly, the difficulties encountered in multimodal medical data fusion and its developmental directions, including modeling methods, evaluation and application of models, are discussed. Through this review article, we expect to provide reference information for the establishment of models that can comprehensively utilize various modal medical data.
Cervical intraepithelial neoplasia is the primary type of cervical precancerous lesion; however, manual clinical diagnosis is prone to bias and has limited grading accuracy. To achieve precise automated grading of CIN, this paper proposes a multimodal fusion Swin Transformer model and develops a corresponding computer-aided diagnosis system. This method employs three-channel fusion of raw images, cervical mask images, and directional gradient histogram features to enhance lesion texture and location information. Within the Swin Transformer backbone, an atrous spatial pyramid pooling module channel attention module and a convolutional feature extraction module are embedded to balance global semantic and local detail features. A focal loss function is adopted to address class imbalance in the dataset and improve the model’s ability to identify difficult-to-classify samples. On a dataset of 3 915 clinical colposcopy images, the model achieved an overall accuracy of 90.01%, precision of 87.55%, recall of 86.17%, F1 score of 89.13%, outperforming baseline models such as VGG, ResNet, and Swin Transformer. The developed system integrates image quality screening, lesion identification, and three-level classification functions, providing an effective tool for the rapid and objective screening of clinical cervical precancerous lesions.