Hybrid Transformer–YOLO Framework for Real-Time Object Detection and Tracking in Complex Surveillance Environments

Real-time object detection and tracking remain difficult in complex surveillance settings, where dense crowds, occlusion, variable illumination, and fast-moving targets are common. YOLO-based detectors run quickly but struggle to model long-range spatial dependencies, whereas transformer architectures capture global context at considerable computational expense. This paper introduces the Hybrid Transformer–YOLO (HT-YOLO) framework, which pairs a lightweight YOLO backbone with a compact transformer encoder through cross-attention fusion, and couples this detector to a tracking stage built on Kalman-filter motion prediction and appearance-based re-identification (Re-ID). Evaluated on the MOT17 and MOT20 benchmarks, HT-YOLO runs at 42 FPS on an RTX 4090 GPU while delivering gains of 4.6 points in mAP@0.5 and 6.1 points in MOTA over a YOLOv8 baseline. The results highlight the contribution of the cross-attention fusion module and the Re-ID-aware tracker, especially under occlusion, and suggest that hybrid CNN–transformer designs can support accurate, low-latency detection and tracking for surveillance applications.