Objective To explore the association between different levels of physical activity and the risk of stroke in middle-aged and elderly people. Methods Middle-aged and elderly people from the China Health and Retirement Longitudinal Study in 2020 were selected as the research subjects. According to total physical (TPA) activity levels, the groups were divided into the inactive group, the low TPA group, the medium TPA group, and the high TPA group. This study used a logistic regression model to explore the impact of different levels of physical activity on the risk of stroke in middle-aged and elderly people. Results A total of 16 027 middle-aged and elderly population were included. Among them, 14 557 were physically active and 1 470 were inactive. Among the 14 557 physically active subjects, there were 4 852 in the high TPA group, 4 852 in the medium TPA group, and 4 853 in the low TPA group. 1 096 (6.8%) middle-aged and elderly people suffered from stroke. The research results showed that after correcting for covariates, medium and high TPA were significantly associated with a lower risk of stroke in middle-aged and elderly people (P<0.01). Furthermore, this study found a nonlinear relationship between TPA level and the prevalence of stroke (P for nonlinear=0.042). Conclusion Medium and high TPA levels are associated with a lower risk of stroke in middle-aged and elderly populations.