A Unified Framework for Evaluating Text Summarization Techniques
Contributors
Ann Baby
Keywords
Proceeding
Track
General Track
License
Copyright (c) 2026 Sustainable Global Societies Initiative

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Abstract
Text summarization has become an essential capability for managing the rapid growth of unstructured textual data across domains such as recruitment, legal analysis, and document management. Research has progressed along three broad paradigms — extractive, transformer-based abstractive, and large language model (LLM)-based summarization — yet these are rarely evaluated together under conditions that reflect real-world deployment. This paper proposes a unified, deployable framework that integrates all three paradigms into a single, CPU-compatible system with standardized ROUGE-1/2/L evaluation, using resume summarization as a representative application. The system was built through an iterative three-stage process — a single-model baseline, a CPU-deployable multi-paradigm framework, and a proposed GPU-oriented advanced design — and deployed as an interactive Hugging Face Space. Quantitative evaluation shows consistently moderate-to-strong lexical overlap with the source text (ROUGE-1 F1 = 0.5459, ROUGE-2 F1 = 0.3467, ROUGE-L F1 = 0.5459), while qualitative analysis exposes clear, practically consequential trade-offs: extractive summarization guarantees factual traceability at the cost of redundancy; abstractive summarization (DistilBART) delivers the strongest balance of coherence and computational efficiency; and LLM-based summarization (TinyLLaMA) achieves the greatest fluency and cross-sectional synthesis, at a measurably higher risk of hallucination. These results demonstrate that no single summarization paradigm dominates across deployment contexts, and that a unified comparative framework has direct, actionable value for practitioners who must choose a summarization strategy under real constraints of accuracy, speed, and computational budget.