Photoplethysmography (PPG) is often affected by interference, which could lead to incorrect judgment of physiological information. Therefore, performing a quality assessment before extracting physiological information is crucial. This paper proposed a new PPG signal quality assessment by fusing multi-class features with multi-scale series information to address the problems of traditional machine learning methods with low accuracy and deep learning methods requiring a large number of samples for training. The multi-class features were extracted to reduce the dependence on the number of samples, and the multi-scale series information was extracted by a multi-scale convolutional neural network and bidirectional long short-term memory to improve the accuracy. The proposed method obtained the highest accuracy of 94.21%. It showed the best performance in all sensitivity, specificity, precision, and F1-score metrics, compared with 6 quality assessment methods on 14 700 samples from 7 experiments. This paper provides a new method for quality assessment in small samples of PPG signals and quality information mining, which is expected to be used for accurate extraction and monitoring of clinical and daily PPG physiological information.