Explainable AI in Healthcare Analytics and Clinical Decision Support Systems: A Systematic Literature Review and Research Framework

Artificial Intelligence (AI) has successfully provoked a paradigm shift in healthcare analytics with its ability to improve diagnosis, prognosis, patient risk assessment and clinical decisions. Many machine learning and deep learning models are high in predictive accuracy, but they are also black-box models, with insufficient transparency, interpretability and clinician trust. Explainable Artificial Intelligence (XAI) is a promising solution that will give understandable explanations for AI-generated predictions while maintaining predictive performance. In this survey, the authors comprehensively examine the latest developments in XAI for health analytics and Clinical Decision Support Systems (CDSS) based on studies published primarily between 2020 and 2026. The selected literature is analyzed with respect to healthcare applications, datasets, AI models, explainability methods, clinical usefulness, and existing limitations. Furthermore, the research directions, unmet challenges and gaps in knowledge on interpretability, fairness, trustworthiness and clinical adoption are broadly discussed. The survey gives a thorough overview of the current state of the art in XAI in healthcare and highlights future research directions to make intelligent healthcare systems more transparent, reliable, and clinically acceptable.