A Comprehensive Benchmarking Framework for Evaluating Traditional and Large Language Model-Based Text Summarization Techniques

The surge of unstructured textual data on various platforms such as recruitment websites, legal databases, and enterprise management systems has further driven the need for automation in text summarization. There are a variety of summarization techniques that fall into one of three categories: extractive, abstractive, and large language model (LLM) based summarization. This paper introduces a unified and deployable framework that unifies and places extractive, abstractive and LLM-based summarization methods in a single standardized evaluation pipeline. It uses a common input repository, a domain-specific resume corpus, the same pre-processing for all paradigms, and benchmarks each approach using ROUGE-1, ROUGE-2 and ROUGE-L F1 scores on a CPU-based deployment served via a Hugging Face Spaces interface. Three progressive model variants – an abstractive summarizer, a multi-paradigm summarizer mode, and an advanced resume intelligence model – were implemented and assessed in terms of model performance, resource consumption, cost-effectiveness, and interpretability. The macro-average ROUGE F1 score ranges around 0.55 for paradigms, and extractive summarization has the best lexical overlap between summary and reference, while LLM-based summarization has the most contextually fluent summary but the least lexically anchored one. The proposed framework enables the reproducible comparative evaluation and also brings practical guidelines for choosing summarization paradigms in a real world deployment scenario, e.g., resume screening and recruitment analytics in a resource constrained setting.