A framework for low cost, ubiquitous and interactive smart refrigerator

Abstract

Internet of Things (IoT) and Artificial Intelligence (AI)-enabled technologies are essential in developing innovative environments and intelligent applications. IoT and AI-enabled appliances are entering our kitchens, adding more comfort and usability. However, these appliances are not economical and are beyond the reach of a commoner with a moderate income. An intelligent fridge is one such appliance. This paper proposes a design for developing a cost-effective, ubiquitous, and intelligent refrigerator. Unlike existing approaches, the proposed method identifies and predicts the fridge items based on Night Vision images and provides minimal natural language interaction with the fridge. The proposed design aims to convert any standard refrigerator into its more intelligent counterpart with minimal hardware and software requirements. The design allows users to view fridge contents on the go using a mobile application and interact with it using natural language. The transfer learning technique enables us to use a YOLOv5n model for object detection. As there are no publicly available Night Vision image datasets of fridge items, we created a custom dataset of Night Vision images to train and validate the object recognition model. Our model for object detection achieved a mAP of 97.1% compared to the YOLOv3-tiny and YOLOv4-tiny models, whose mAP are 94.8% and 96.3%, respectively. The overall cost of the refrigerator after deployment of the module is less than $300, making it an affordable option. The proposed framework meets most of the requirements of a low-cost, ubiquitous, interactive smart refrigerator. © 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.