Light-SustainFormer: A Lightweight Temporal Fusion Transformer-Based CNN–BiGRU Framework for Sustainable Smart Building Energy Forecasting

Abstract

 Energy consumption forecasting is central to the design of smart, sustainable, and cost-efficient buildings. Recent deep learning and Transformer-based models have improved forecasting accuracy, yet most such models prioritise predictive performance over computational cost and model size. Full Transformer and Temporal Fusion Transformer architectures capture long-range temporal patterns, but their large dimensions and multiple attention blocks require high computational cost. This paper introduces Light-SustainFormer, a lightweight deep learning framework for short-term energy consumption forecasting in smart buildings. We propose a Lite-Temporal Fusion Transformer attention module that streamlines the conventional TFT by using fewer attention heads and a single attention block with a lightweight feed-forward layer. Instead of applying an attention block directly to raw time-series data, the proposed model first passes the inputs through CNN layers to capture short-term energy fluctuations, followed by Bi-GRU layers to produce compact temporal representations. This CNN–BiGRU-based compression reduces computational cost without sacrificing the ability to model long-range energy usage patterns. Light-SustainFormer learns energy consumption behaviour across weather, occupancy, time of day, and seasonal variations. On experimental benchmarks, the Light-SustainFormer achieves an MAE of 0.118 kWh, an RMSE of 0.164 kWh, a MAPE of 4.82%, and an R² of 0.961. These results demonstrate that a lightweight, attention-based architecture can deliver strong forecasting accuracy at low computational cost, providing a practical solution for energy management in smart buildings.