An Enhanced Hybrid Transformer–YOLO Framework for Real-Time Object Detection and Tracking in Complex Surveillance Environments


Date Published : 21 November 2026

Contributors

Dr.Jayaram C V

LUCM
Author

Dr. SUDHAKAR K

Nitte Meenakshi Institute of Technology, NITTE (Deemed to be University)
Author

Keywords

Object detection; multi-object tracking; YOLOv11; vision transformer; dual-path attention fusion; smart surveillance; real-time computer vision; occlusion handling; ablation study.

Proceeding

Track

General Track

License

Copyright (c) 2026 Sustainable Global Societies Initiative

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Abstract

Detecting and tracking objects in real time within challenging surveillance footage, where crowds are dense, objects are occluded or tiny, and motion causes blur, is still an open problem. YOLO detectors built on CNNs run quickly but lack wider context, while Transformers supply global context only by paying a heavy computational price. Extending our previous proposal, this work implements and evaluates an improved Hybrid Transformer–YOLO (HT-YOLO) framework in which a YOLOv11 backbone is paired with a Vision Transformer (ViT) encoder via a Dual-Path Attention Fusion module, and an identity-preserving multi-object tracking module follows. When compared with YOLOv8, ViT, and YOLO+DeepSORT baselines, HT-YOLO attains 74.1% mAP (+13.1 points over YOLOv8) at 35 FPS, lifts IDF1 from 63% (DeepSORT) to 78%, and lowers identity switches from 42 to 19. Step-by-step ablations reveal that every component improves accuracy, speed, or tracking stability, suggesting that a lightweight CNN–Transformer hybrid offers a workable path toward precise, real-time surveillance analytics.

References

No References

Downloads

How to Cite

Dr.Jayaram C V, D. C. V., & Dr. SUDHAKAR K, D. S. K. (2026). An Enhanced Hybrid Transformer–YOLO Framework for Real-Time Object Detection and Tracking in Complex Surveillance Environments. Sustainable Global Societies Initiative, 1(15). https://vectmag.com/sgsi/paper/view/1428