CookIS: Your AI-Powered Cooking Companion

Abstract

This paper presents CookIS, an innovative AI-powered cooking assistant that delivers a hands-free, personalised, and context-aware culinary experience. The system integrates wearable glasses with a camera and microphone to capture real-time visual and audio inputs, which are processed using a Raspberry Pi and a home server running advanced AI models. The setup employs YOLOv5 for ingredient recognition, Google Speech Recognition for voice commands, and a Retrieval-Augmented Generation (RAG)-enhanced LLaMA model to generate step-by-step cooking instructions. Output is delivered to the user through Google Text-to-Speech (gTTS), providing seamless audio guidance. CookIS addresses key challenges in conventional cooking, such as manual recipe searching and difficulty following instructions. Experimental results demonstrate the system’s effectiveness in enabling real-time guidance, adaptability to available ingredients, and enhanced user experience, particularly for beginners and elderly users. By combining wearable technology, computer vision, and large language models, CookIS contributes toward accessible, sustainable, and efficient smart kitchen solutions