Hybrid BLIP-CLIP Framework for Semantically Enhanced Automatic Image Caption Generation

Automatic image caption generation stands at the crossroads of computer vision and natural language processing, requiring systems that can simultaneously understand visual content and produce fluent, semantically accurate textual descriptions. Existing single-model approaches—including Bootstrapping Language-Image Pre-training (BLIP) and Contrastive Language-Image Pre-training (CLIP)—demonstrate complementary strengths and limitations. BLIP produces linguistically diverse candidate captions yet occasionally generates semantically inconsistent descriptions, while CLIP offers robust visual-semantic alignment but lacks a generative capability. This paper presents a two-stage hybrid architecture that combines BLIP's generative fluency with an ensemble CLIP re-ranking mechanism to achieve consistently high-quality image captions. BLIP generates multiple candidate descriptions through beam-search decoding; a multi-variant CLIP ensemble then scores each candidate on image-text semantic similarity and selects the optimal output. Evaluation on a diverse image corpus encompassing natural scenes, indoor environments, and complex multi-object compositions demonstrates significant improvements over single-model baselines across all standard captioning metrics. The hybrid system attains BLEU-4 of 0.89, METEOR of 0.91, CIDEr of 1.18, SPICE of 0.84, and ROUGE-L of 0.88—gains of 7–10% over BLIP alone. A web-based deployment platform enables real-time caption generation with interactive performance visualization. These results confirm that ensembling generative and discriminative vision-language models represents a practical and effective strategy for improving automatic captioning quality.