Explainable Robustness Against Localized Malware Obfuscation: A Hybrid CNN-ViT Approach with Comparative Attention Analysis

Abstract

The rapid proliferation of obfuscated and zero-day malware variants poses a critical challenge to modern cybersecurity defences. Traditional Convolutional Neural Network (CNN) classifiers, while effective on clean binary images, exhibit catastrophic vulnerability when adversaries apply localised packing or byte-level scrambling to evade detection. In this paper, a hybrid deep learning architecture that combines a lightweight CNN feature extractor with a Vision Transformer (ViT) encoder to achieve explainable robustness against localised malware obfuscation. The CNN extracts hierarchical texture patterns from grayscale binary images, while the ViT models long-range spatial dependencies through multi-head self-attention across 196 image patches. Crucially, this approach injects Spatial Dropout (nn.Dropout2d) into the CNN layers and employs Focal Loss to handle severe class imbalance, forcing the model to learn from partially corrupted feature maps during training itself. Evaluated on the Malimg benchmark (9,339 samples, 25 families), the hybrid model achieves 93.80% accuracy with a Macro AUC of 0.9980. Under simulated obfuscation with 30% local byte noise, the hybrid architecture maintains 81.82% accuracy, while an equivalent pure CNN baseline using the same CNN feature extractor collapses to 52.62%, a gap of 29.20 percentage points. Through comparative attention heatmaps, this paper provides visual, interpretable proof that while CNN activations scatter across noise artifacts under obfuscation, the ViT’s self-attention dynamically shifts to focus on persistent global structural payloads, explaining why the architecture survives where CNNs fail.