A Unified Framework for Evaluating Text Summarization Techniques


Date Published : 20 November 2026

Contributors

Ann Baby

Rajagiri College of Social Sciences, Kochi, India
Author

Keywords

Text Summarization Extractive Summarization Abstractive Summarization Large Language Models Transformer Models ROUGE Evaluation Metrics Hugging Face Deployment Resume Summarization Comparative Analysis Natural Language Processing.

Proceeding

Track

General Track

License

Copyright (c) 2026 Sustainable Global Societies Initiative

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Abstract

Text summarization has become an essential capability for managing the rapid growth of unstructured textual data across domains such as recruitment, legal analysis, and document management. Research has progressed along three broad paradigms — extractive, transformer-based abstractive, and large language model (LLM)-based summarization — yet these are rarely evaluated together under conditions that reflect real-world deployment. This paper proposes a unified, deployable framework that integrates all three paradigms into a single, CPU-compatible system with standardized ROUGE-1/2/L evaluation, using resume summarization as a representative application. The system was built through an iterative three-stage process — a single-model baseline, a CPU-deployable multi-paradigm framework, and a proposed GPU-oriented advanced design — and deployed as an interactive Hugging Face Space. Quantitative evaluation shows consistently moderate-to-strong lexical overlap with the source text (ROUGE-1 F1 = 0.5459, ROUGE-2 F1 = 0.3467, ROUGE-L F1 = 0.5459), while qualitative analysis exposes clear, practically consequential trade-offs: extractive summarization guarantees factual traceability at the cost of redundancy; abstractive summarization (DistilBART) delivers the strongest balance of coherence and computational efficiency; and LLM-based summarization (TinyLLaMA) achieves the greatest fluency and cross-sectional synthesis, at a measurably higher risk of hallucination. These results demonstrate that no single summarization paradigm dominates across deployment contexts, and that a unified comparative framework has direct, actionable value for practitioners who must choose a summarization strategy under real constraints of accuracy, speed, and computational budget.

References

No References

Downloads

How to Cite

Baby, A. (2026). A Unified Framework for Evaluating Text Summarization Techniques. Sustainable Global Societies Initiative, 1(13). https://vectmag.com/sgsi/paper/view/1488