Aiming at the problem that residual error gets smaller with the depth of propagation increasing and the bottom of DNN trains ineffective, by investigating the limitations of sigmoid activation function in DNN, a sigmoid activation function with two parameters is proposed. One parameter makes the input of sigmoid activation function concentrate in sides of the origin, and another parameter restrains the decreasing speed of residual error. The combination of two parameters enhances the training of DNN. Do number classification experiments on MNIST using deep belief networks(DBN), the results show that sigmoid activation function with two parameters can be used in DNN directly without pre-training and improve the performance in DNN with pre-training.