Efficient Real-Time Multi-Class Object Tracking with YOLO11 and ByteTrack in Real-World Driving Scenes

Abstract

Accurate and real-time multi-object tracking (MOT) is essential for autonomous driving systems to ensure safe navigation and decision making in dynamic environments. This paper introduces a tracking-by-detection pipeline that integrates YOLOv11 a high speed, high-accuracy object detector with ByteTrack, a robust data association algorithm capable of lever-aging both high and low confidence detections. The proposed framework addresses key challenges in MOT such as frequent occlusions, fluctuating lighting, and dense traffic by combining efficient detection with motion-consistent identity tracking. Evaluated on the KITTI benchmark, our method demonstrates superior performance across multiple metrics, including HOTA, AssA, and MOTA, for both cars and pedestrians. Additionally, the system achieves an average runtime of 60.4 FPS, supporting its real-time applicability. The results confirm that the proposed YOLOv11 + ByteTrack integration provides a scalable, accurate, and deployment ready solution for complex urban driving scenarios.