Aiming at the high resource consumption problem in deploying deep convolutional neural networks on edge devices, a collaborative optimization method combining knowledge distillation and low-bit quantization was studied; The key technology of joint guidance from quantization-aware training and distillation loss was adopted, effectively mitigating the accuracy loss of low-bit quantization through teacher model soft label supervision and projected gradient descent optimization; Experimental tests on CIFAR-10 and CIFAR-100 datasets showed that this method achieved 4-bit quantization of ResNet series networks, reaching 92.1% accuracy on CIFAR-10 with model size compressed to 0.41MB; Verified through FPGA edge deployment, the ResNet-20 inference latency was reduced from 82.3ms to 5.67ms, meeting the engineering requirements for low latency and high efficiency in edge computing; The research confirms that this method can significantly reduce resource overhead while maintaining accuracy, providing an effective solution for neural network deployment in resource-constrained environments.