Due to adverse factors such as coal dust interference, changes in illumination, equipment spatial position alterations and occlusions, as well as dynamic variations in the monitoring camera’s perspective, the automatic detection and recognition results of key target objects in mining face production scenes are prone to missed and false detections, especially for small target objects. This directly impacts the reliable application of intelligent video monitoring under work face production conditions. In order to improve the accuracy of automatic detection and recognition of key target objects, particularly small target objects, under actual production conditions and to enhance the effect of intelligent monitoring, this paper proposes an improved lightweight target object automatic detection algorithm model for mining faces based on YOLOv8. In the network of the algorithm model, the C2f module is first replaced with the C2f-Faster convolution module. Through structural lightweighting and computational optimization, model complexity is reduced and redundant computations as well as the number of parameters are minimized. Secondly, a SimAM attention mechanism is introduced, which uses the energy scale between features as a factor for assigning feature attention weights. This enhances the model's feature representation ability, achieving high detection accuracy for multi-scale objects, especially for small target objects, with minimal computational overhead. Furthermore, to address the degradation of the CIoU loss term during model training, which fails to reflect the true state of width and height regression, a new loss function called NIoU is proposed. This loss function enhances the sensitivity of the gradient during backpropagation to errors in the width-to-height ratio of the predicted target object boxes, thereby improving detection precision and recall. Experimental validation using actual data from multiple domestic mining face scenarios shows that, compared to the YOLOv8 model, the proposed algorithm model achieves a 39.13% reduction in the number of parameters, a 39.90% decrease in computational load, a 1.8% improvement in detection accuracy of target objects, and a 24% increase in inference speed. Finally, the proposed target detection algorithm model is applied to intelligent video monitoring of mining faces, demonstrating satisfactory performance that meets the real-time and reliability requirements for engineering applications.