Deep Learning Framework for Indian Heritage Site Classification and Virtual Tour Generation using Transfer Learning

Abstract

The Indian heritage sites represent several centuries of history, architecture, and diversity of various cultures. Creative approaches are required to secure accessibility and awareness, and to maintain and promote these places. Traditional methods of heritage interpretation often fail to deliver a detailed, scalable, and automated experience because they rely on human guides, fixed displays, or limited mobile applications. This paper proposes a deep learning-based system that integrates a virtual tour system and heritage site classification. We predict the 55 Indian monuments using the datasets (Kaggle + GitHub) with transfer learning using EfficientNetB0 and ResNet50. To improve generalisation, a unified dataset of 1,091 images was processed and augmented. The trained models achieved good classification performance and high validation accuracy (EfficientNetB0: 99.39%; ResNet50: 99.08%). To enhance cultural interaction, we combine the classifier with a virtual tour module that captures the inside and outside views of the architectural site, along with textual descriptions in JSON files, with each landmark having its own. Users can upload an architectural image, label the monument, and automatically navigate its interior via a structured virtual tour using a web-based GUI (React frontend + FastAPI backend). The proposed system addresses gaps in existing systems by being automated, scalable, and accessible. Future developments for this system include multilingual support, AR/VR support, and implementation as an offline mobile/web application to increase digital heritage preservation.