The existing control methods for the end effector of the manipulator have shortcomings such as poor stability and large swing amplitude differences. Therefore, this paper designs a control algorithm for the end effector of the manipulator based on the stochastic policy gradient algorithm. Firstly, a kinematic model of the end effector of the manipulator is established in the spatial range, and an adaptive sliding mode control value is constructed to implement dynamic control of the spatial motion process of the manipulator. The adaptive sliding mode controller is optimized by the K-means clustering stochastic gradient strategy to improve the robustness and stability of the control system. Based on the analysis of the mechanical quantities and joint disturbances of the end effector, an anti-swing control function is introduced to dynamically control and adjust the movement trajectory of the manipulator, and residual compensation is simultaneously implemented to make it closer to the theoretical trajectory. The experimental results show that the control based on the above method can avoid the side deviation of the end effector and reduce the swing of the manipulator during the movement, with the maximum swing value being only 2.3°. The ablation experiments also verify the effectiveness of the method.