An Enhanced Hybrid Transformer–YOLO Framework for Real-Time Object Detection and Tracking in Complex Surveillance Environments
Contributors
Dr.Jayaram C V
Dr. SUDHAKAR K
Keywords
Proceeding
Track
General Track
License
Copyright (c) 2026 Sustainable Global Societies Initiative

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Abstract
Detecting and tracking objects in real time within challenging surveillance footage, where crowds are dense, objects are occluded or tiny, and motion causes blur, is still an open problem. YOLO detectors built on CNNs run quickly but lack wider context, while Transformers supply global context only by paying a heavy computational price. Extending our previous proposal, this work implements and evaluates an improved Hybrid Transformer–YOLO (HT-YOLO) framework in which a YOLOv11 backbone is paired with a Vision Transformer (ViT) encoder via a Dual-Path Attention Fusion module, and an identity-preserving multi-object tracking module follows. When compared with YOLOv8, ViT, and YOLO+DeepSORT baselines, HT-YOLO attains 74.1% mAP (+13.1 points over YOLOv8) at 35 FPS, lifts IDF1 from 63% (DeepSORT) to 78%, and lowers identity switches from 42 to 19. Step-by-step ablations reveal that every component improves accuracy, speed, or tracking stability, suggesting that a lightweight CNN–Transformer hybrid offers a workable path toward precise, real-time surveillance analytics.