In order to meet the need of autonomous control of patients with severe limb disorders, this paper designs a nursing bed control system based on motor imagery-brain computer interface (MI-BCI). In view of the low decoding performance of cross-subjects and the dynamic fluctuation of cognitive state in the existing MI-BCI technology, the neural network structure optimization and user interaction feedback enhancement are improved. Firstly, the optimized dual-branch graph convolution multi-scale neural network integrates dynamic graph convolution and multi-scale convolution. The average classification accuracy is higher than that of multi-scale attention temporal convolution network, Gram angle field combined with convolution long short term memory hybrid network, Transformer-based graph convolution network and other existing methods. Secondly, a dual visual feedback mechanism is constructed, in which electroencephalogram (EEG) topographic map feedback can improve the discrimination of spatial patterns, and attention state feedback can enhance the temporal stability of signals. Compared with the single EEG topographic map feedback and non-feedback system, the average classification accuracy of the proposed method is also greatly improved. Finally, in the four classification control task of nursing bed, the average control accuracy of the system is 90.84%, and the information transmission rate is 84.78 bits/min. In summary, this paper provides a reliable technical solution for improving the autonomous interaction ability of patients with severe limb disorders, which has important theoretical significance and application value.
ObjectiveTo investigate the feasibility and effectiveness of motor imagery based brain computer interface with wrist passive movement in chronic stroke patients with wrist extension impairment.MethodsFifteen chronic stroke patients with a mean age of (47.60±14.66) years were recruited from March 2017 to June 2018. At baseline, motor imagery ability was assessed first. Then motor imagery based brain computer interface with wrist passive movement was given as an intervention. Both range of motion of paretic wrist and Barthel index was assessed before and after the intervention.ResultsAmong the 15 chronic stroke patients admitted in the study, 12 finished the whole therapy, and 3 failed to pass the initial assessment. After the therapy, the 12 participants who completed the whole sessions of the treatment and follow up had improved ability of control electroencephalogram, in whom 9 regained the ability to actively extend the affected wrist, and the other 3 failed to actively extend their wrist (the rate of active extending wrist was 75%). The activity of daily life of all the participants did not change significantly before and after intervention, and no discomfort was found after daily treatment.ConclusionIn chronic stroke patients with wrist extension impairment, motor imagery based brain computer interface with wrist passive movement training is feasible and effective.
The effective classification of multi-task motor imagery electroencephalogram (EEG) is helpful to achieve accurate multi-dimensional human-computer interaction, and the high frequency domain specificity between subjects can improve the classification accuracy and robustness. Therefore, this paper proposed a multi-task EEG signal classification method based on adaptive time-frequency common spatial pattern (CSP) combined with convolutional neural network (CNN). The characteristics of subjects' personalized rhythm were extracted by adaptive spectrum awareness, and the spatial characteristics were calculated by using the one-versus-rest CSP, and then the composite time-domain characteristics were characterized to construct the spatial-temporal frequency multi-level fusion features. Finally, the CNN was used to perform high-precision and high-robust four-task classification. The algorithm in this paper was verified by the self-test dataset containing 10 subjects (33 ± 3 years old, inexperienced) and the dataset of the 4th 2018 Brain-Computer Interface Competition (BCI competition Ⅳ-2a). The average accuracy of the proposed algorithm for the four-task classification reached 93.96% and 84.04%, respectively. Compared with other advanced algorithms, the average classification accuracy of the proposed algorithm was significantly improved, and the accuracy range error between subjects was significantly reduced in the public dataset. The results show that the proposed algorithm has good performance in multi-task classification, and can effectively improve the classification accuracy and robustness.
Aiming at the problem of low recognition accuracy of motor imagery electroencephalogram signal due to individual differences of subjects, an individual adaptive feature representation method of motor imagery electroencephalogram signal is proposed in this paper. Firstly, based on the individual differences and signal characteristics in different frequency bands, an adaptive channel selection method based on expansive relevant features with label F (ReliefF) was proposed. By extracting five time-frequency domain observation features of each frequency band signal, ReliefF algorithm was employed to evaluate the effectiveness of the frequency band signal in each channel, and then the corresponding signal channel was selected for each frequency band. Secondly, a feature representation method of common space pattern (CSP) based on fast correlation-based filter (FCBF) was proposed (CSP-FCBF). The features of electroencephalogram signal were extracted by CSP, and the best feature sets were obtained by using FCBF to optimize the features, so as to realize the effective state representation of motor imagery electroencephalogram signal. Finally, support vector machine (SVM) was adopted as a classifier to realize identification. Experimental results show that the proposed method in this research can effectively represent the states of motor imagery electroencephalogram signal, with an average identification accuracy of (83.0±5.5)% for four types of states, which is 6.6% higher than the traditional CSP feature representation method. The research results obtained in the feature representation of motor imagery electroencephalogram signal lay the foundation for the realization of adaptive electroencephalogram signal decoding and its application.
The brain-computer interface (BCI) based on motor imagery electroencephalography (EEG) shows great potential in neurorehabilitation due to its non-invasive nature and ease of use. However, motor imagery EEG signals have low signal-to-noise ratios and spatiotemporal resolutions, leading to low decoding recognition rates with traditional neural networks. To address this, this paper proposed a three-dimensional (3D) convolutional neural network (CNN) method that learns spatial-frequency feature maps, using Welch method to calculate the power spectrum of EEG frequency bands, converted time-series EEG into a brain topographical map with spatial-frequency information. A 3D network with one-dimensional and two-dimensional convolutional layers was designed to effectively learn these features. Comparative experiments demonstrated that the average decoding recognition rate reached 86.89%, outperforming traditional methods and validating the effectiveness of this approach in motor imagery EEG decoding.
Convolutional neural networks (CNNs) are renowned for their excellent representation learning capabilities and have become a mainstream model for motor imagery based electroencephalogram (MI-EEG) signal classification. However, MI-EEG exhibits strong inter-individual variability, which may lead to a decline in classification performance. To address this issue, this paper proposes a classification model based on dynamic multi-scale CNN and multi-head temporal attention (DMSCMHTA). The model first applies multi-band filtering to the raw MI-EEG signals and inputs the results into the feature extraction module. Then, it uses a dynamic multi-scale CNN to capture temporal features while adjusting attention weights, followed by spatial convolution to extract spatiotemporal feature sequences. Next, the model further optimizes temporal correlations through time dimensionality reduction and a multi-head attention mechanism to generate more discriminative features. Finally, MI classification is completed under the supervision of cross-entropy loss and center loss. Experiments show that the proposed model achieves average accuracies of 80.32% and 90.81% on BCI Competition IV datasets 2a and 2b, respectively. The results indicate that DMSCMHTA can adaptively extract personalized spatiotemporal features and outperforms current mainstream methods.
Motor imagery electroencephalogram (EEG) signals are non-stationary time series with a low signal-to-noise ratio. Therefore, the single-channel EEG analysis method is difficult to effectively describe the interaction characteristics between multi-channel signals. This paper proposed a deep learning network model based on the multi-channel attention mechanism. First, we performed time-frequency sparse decomposition on the pre-processed data, which enhanced the difference of time-frequency characteristics of EEG signals. Then we used the attention module to map the data in time and space so that the model could make full use of the data characteristics of different channels of EEG signals. Finally, the improved time-convolution network (TCN) was used for feature fusion and classification. The BCI competition IV-2a data set was used to verify the proposed algorithm. The experimental results showed that the proposed algorithm could effectively improve the classification accuracy of motor imagination EEG signals, which achieved an average accuracy of 83.03% for 9 subjects. Compared with the existing methods, the classification accuracy of EEG signals was improved. With the enhanced difference features between different motor imagery EEG data, the proposed method is important for the study of improving classifier performance.
Motor imagery electroencephalogram (MI-EEG) decoding algorithms face multiple challenges. These include incomplete feature extraction, susceptibility of attention mechanisms to distraction under low signal-to-noise ratios, and limited capture of long-range temporal dependencies. To address these issues, this paper proposes a multi-branch differential attention temporal network (MDAT-Net). First, the method constructed a multi-branch feature fusion module to extract and fuse diverse spatio-temporal features from different scales. Next, to suppress noise and stabilize attention, a novel multi-head differential attention mechanism was introduced to enhance key signal dynamics by calculating the difference between attention maps. Finally, an adaptive residual separable temporal convolutional network was designed to efficiently capture long-range dependencies within the feature sequence for precise classification. Experimental results showed that the proposed method achieved average classification accuracies of 85.73%, 90.04%, and 96.30% on the public datasets BCI-IV-2a, BCI-IV-2b, and HGD, respectively, significantly outperforming several baseline models. This research provides an effective new solution for developing high-precision motor imagery brain-computer interface systems.
Clinical grading diagnosis of disorder of consciousness (DOC) patients relies on behavioral assessment, which has certain limitations. Combining multi-modal technologies and brain-computer interface (BCI) paradigms can assist in identifying patients with minimally conscious state (MCS) and vegetative state (VS). This study collected electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) signals under motor BCI paradigms from 14 DOC patients, who were divided into two groups based on clinical scores: 7 in the MCS group and 7 in the VS group. We calculated event-related desynchronization (ERD) and motor decoding accuracy to analyze the effectiveness of motor BCI paradigms in detecting consciousness states. The results showed that the classification accuracies for left-hand and right-hand movement tasks using EEG were 93.28% and 76.19% for the MCS and VS groups, respectively; the classification precisions using fNIRS were 53.72% and 49.11% for these groups. When combining EEG and fNIRS features, the classification accuracies for left-hand and right-hand movement tasks in the MCS and VS groups were 95.56% and 87.38%, respectively. Although there was no statistically significant difference in motor decoding accuracy between the two groups, significant differences in ERD were observed between different consciousness states during left-hand movement tasks (P < 0.001). This study demonstrates that motor BCI paradigms can assist in assessing the level of consciousness, with EEG being more sensitive for evaluating residual motor intention intensity. Moreover, the ERD feature of motor intention intensity is more sensitive than BCI classification accuracy.
Transfer learning is provided with potential research value and application prospect in motor imagery electroencephalography (MI-EEG)-based brain-computer interface (BCI) rehabilitation system, and the source domain classification model and transfer strategy are the two important aspects that directly affect the performance and transfer efficiency of the target domain model. Therefore, we propose a parameter transfer learning method based on shallow visual geometry group network (PTL-sVGG). First, Pearson correlation coefficient is used to screen the subjects of the source domain, and the short-time Fourier transform is performed on the MI-EEG data of each selected subject to acquire the time-frequency spectrogram images (TFSI). Then, the architecture of VGG-16 is simplified and the block design is carried out, and the modified sVGG model is pre-trained with TFSI of source domain. Furthermore, a block-based frozen-fine-tuning transfer strategy is designed to quickly find and freeze the block with the greatest contribution to sVGG model, and the remaining blocks are fine-tuned by using TFSI of target subjects to obtain the target domain classification model. Extensive experiments are conducted based on public MI-EEG datasets, the average recognition rate and Kappa value of PTL-sVGG are 94.9% and 0.898, respectively. The results show that the subjects’ optimization is beneficial to improve the model performance in source domain, and the block-based transfer strategy can enhance the transfer efficiency, realizing the rapid and effective transfer of model parameters across subjects on the datasets with different number of channels. It is beneficial to reduce the calibration time of BCI system, which promote the application of BCI technology in rehabilitation engineering.