A Hybrid Vision Transformer–CNN Diagnosis Network with Explainable AI for Multi-Class Skin Cancer Prediction

Accurate multi-class classification of dermoscopic skin lesions remains a persistent diagnostic bottleneck, driven by high intra-class visual variability, strong inter-class similarity between malignant and benign lesions, and severe class imbalance in public benchmark datasets. This paper presents a hybrid diagnosis network that integrates vision transformers with convolutional feature extraction and explainable artificial intelligence (XAI) to address these challenges. The proposed pipeline combines Segment Anything Model (SAM)-guided lesion segmentation and artifact removal with an SMOTE and Focal Cross-Entropy Loss-based class-imbalance mitigation strategy. A dual-backbone architecture fuses a Swin Transformer, which captures global contextual dependencies across the dermatoscopic field, with a ConvNeXt branch that preserves fine-grained local texture, unified through a cross-attention fusion module that classifies seven distinct lesion categories. Evaluated on the HAM10000 benchmark, the network achieved 96.15% overall multi-class accuracy and a 98.14% Dice coefficient in lesion segmentation, outperforming standalone Vision Transformer and ResNet-50 baselines by 3.75% and 7.35%, respectively. Score-CAM-based explainability further demonstrated that the model's activation heatmaps align with the ABCDE clinical criteria used by dermatologists, supporting clinical trust and enabling lightweight, quantized deployment on portable dermatoscope-coupled devices. These results indicate that combining global transformer context, local convolutional detail, and interpretable visual explanations offers a viable path toward reliable, deployable computer-aided diagnosis (CAD) systems for skin cancer screening.